FERMI: A FEmtocell Resource Management System forInterference Mitigation in OFDMA Networks

Tech. Report for Paper ID: 260

ABSTRACTThe demand for increased spectral efficiencies is driving thenext generation broadband access networks towards deploy-ing smaller cells (femtocells) and sophisticated air interfacetechnologies (Orthogonal Frequency Division Multiple Ac-cess or OFDMA). The dense deployment of femtocells how-ever, makes interference and hence resource managementboth critical and extremely challenging. In this paper, wedesign and implement one of the first resource managementsystems, FERMI, for OFDMA-based femtocell networks.As part of its design, FERMI (i) provides resource isola-tion in the frequency domain (as opposed to time) to lever-agepower poolingacross cells to improve capacity; (ii) usesmeasurement-driven triggers to intelligently distinguish cli-ents that require just link adaptation from those that requireresource isolation; (iii) incorporates mechanisms that enablethe joint scheduling of both types of clients in the sameframe; and (iv) employs efficient, scalable algorithms to de-termine a fair resource allocation across the entire networkwith high utilization. We implement FERMI on a prototypefour-cell WiMAX femtocell testbed and show that it yieldssignificant gains over conventional approaches.

1. INTRODUCTIONThe demand for higher data rates and increased spectral

efficiencies is driving the next generation broadband accessnetworks towards deploying smaller cell structures (calledfemtocells) with OFDMA [1]. They are installed in enter-prises and homes, and operate using the same spectrum andtechnology as macrocells, while connecting to the core net-work through cable or DSL backhaul. In addition to the in-creased user throughput from short ranges, the smaller sizeof femtocells increases the system capacity by enabling spa-tial reuse. This allows broadband access service providersto(i) improve coverage and service quality, (ii) effectivelybal-ance load by offloading traffic from macrocell to femtocells,and (iii) reduce operational expenses and subscriber churn.

To retain the aforementioned benefits, femtocells have tointer-operate with and use the same access technology asmacrocells. Hence, resource management solutions for fem-tocells cannot be designed from scratch. Although thereare methods proposed to alleviate the macro-femto interfer-ence [2], interference mitigation between femtocells has notdrawn much attention and thus forms the focus of this work.

There are several key aspects that make the resource man-agement problem both challenging and unique in OFDMAfemtocells. We articulate these aspects below.

Femtocells versus Macrocells: Femtocell deployments aresignificantly more dense compared to the planned deploy-ments of macrocells. Hence, while interference is localizedat cell edges in macrocells, it is less predictable and morepervasive across femtocells. This renders Fractional Fre-quency Reuse (FFR) solutions (proposed for macrocells) in-adequate in mitigating interference in femtocells.

Femtocells versus WiFi: In femtocell networks, OFDMAuses a synchronous transmission access policy across cells,on a licensed spectrum. In contrast, with WiFi unlicensedspectrum is accessed asynchronously. This affects resourcemanagement (interference mitigation) in the two systems ina fundamental way. In a typical WiFi system, interferingcells are either tuned to operate on orthogonal channels oruse carrier sensing to arbitrate medium access on the samechannel. In an OFDMA femtocell system, there is no carriersensing. Interfering cells can either operate on orthogonalparts of the spectrum, or directly project interference on theclients of each other. In OFDMA femtocells, transmissionsto different clients are multiplexed in each frame. Since ev-ery client may not need spectral isolation, simply operatingadjacent cells on separate parts of the spectrum comes at thecost of underutilization of the available capacity. In otherwords, resource isolation in OFDMA femtocells needs to beadministrated with care. In a WiFi system, since an accesspoint transmits data to a single client at a time (using theentire channel assigned to it), this challenge does not arise.

Our contributions in brief: We design and implementone of the first resource management systems, FERMI, forOFDMA-based femtocell networks. FERMI decouples re-source management across the network from scheduling wi-thin each femtocell and addresses the former. This allowsresource allocation across femtocells to be determined by acentral controller (CC) at coarse time scales. Frame schedul-ing within each femtocell can then be executed indepen-dently on the allocated set of resources. The four key corner-stones of FERMI’s resource management solution include:

• Frequency Domain Isolation:It isolates resources for cli-ents in each femtocell, in frequency (as opposed to time).This allows forpower poolingto jointly mitigate interfer-ence and increase system capacity (discussed later).• Client Categorization:It employs proactive, measurement-

driven triggers to intelligently distinguish clients in eachfemtocell that require just link adaptation from those thatrequire resource isolation with an accuracy of over 90%.• Zoning: It incorporates a frame structure that supports

the graceful coexistence of clients that can reuse the spec-

trum and the clients that require resource isolation.• Resource Allocation and Assignment:It employs novel

algorithms to assign orthogonal sub-channels to interfer-ing femtocells in a near-optimal fashion.

We have implemented a prototype of FERMI on an exper-imental four-cell WiMAX femtocell testbed. FERMI pro-vides a complete resource management solution while beingstandards compatible; this enables its adoption on not onlyexperimental platforms but also on commercial femtocellsystems. To the best of our knowledge, we report the first re-source management solution implemented on an actual OF-DMA femtocell testbed. Comprehensive evaluations showFERMI’s resource management to yield significant gains insystem throughput over conventional approaches.

Organization: We describe background and related workin §2. Experiments that motivate FERMI’s design are in§3.The building blocks of FERMI are described in§4. In §5,we describe the resource allocation algorithms that are partof FERMI. We evaluate FERMI in§6. We conclude in§7.

2. BACKGROUND AND RELATED WORKIn this section, we describe relevant related work. We then

provide a brief background on WiMAX femtocell systems.Macrocellular Systems:While broadband standards em-

ploying OFDMA (WiMAX, LTE) are relatively recent, re-lated research has existed for quite some time [3]. Thereis research that addresses problems pertaining to single cell[4] and multi-cell [5] OFDMA systems. Several efforts havelooked at the interference between macrocells and femto-cells [2, 6] leveraging thelocalizedinterference coupled withplannedcell layouts. However, the interferenceamongfem-tocells remains in question since femtocells lack the featuresof localized interference and planned deployments of macro-cells. There have been some recent works [7] that address in-terference among femtocells via distributed mechanisms butare restricted to theoretical studies. In contrast, we imple-ment a centralized resource management system to mitigateinterference among femtocells.

Spectrum Allocation: There has been research address-ing resource allocation using graph coloring for WiFi [8,9, 10]. The main objective in these studies is to allocatea minimum number of orthogonal contiguous channels toeach interfering AP. Instead, our objective is to realize aweighted max-min fair allocation while utilizing as manysub-channels as possible. In addition, resource allocation isjust one component of our work; we implement a novel re-source management system with several enhancements tai-lored to OFDMA. There are also approaches that allocatespectrum chunks to contending entities [11, 12]. However,these studies rely on asynchronous random access and asso-ciated sensing capabilities. We address a more challengingproblem in OFDMA synchronous access systems and satisfyrequirements that are specific to OFDMA femtocells.

WiMAX Preliminaries: While our study applies to multi-cell OFDMA femto networks in general, our measurements

are conducted on a WiMAX (802.16e [13]) femtocell testbed.In WiMAX, OFDMA divides the spectrum into multiple

tones (sub-carriers) and several sub-carriers are groupedtoform a sub-channel. A WiMAX frame is a two-dimensionaltemplate that carries data to multiple mobile stations (MSs)across both time (symbols) and frequency (sub-channels).The combination of a symbol and a sub-channel constitutesa tile (the basic unit of resource allocation at the MAC). Datato users are allocated as rectangular bursts of tiles in a frame.

In OFDMA femtocells, frame transmissions are synchro-nized both between the BS1 and MSs as well as across BSs(by virtue of synchronizing to the macro BS [14]). An ex-ample of a WiMAX TDD (time division duplexing) frameis shown in Fig. 1(a); the transmissions from the BS to aMS (downlink) and those from the MS to the BS (uplink)are separated in time. The frame consists of the preamble,control and data payload. While the preamble is used by theMS to lock on to the BS, the control consists of FCH (framecontrol header) and MAP. MAP conveys the location of thedata burst for a MS in a frame and consists of both the down-link and uplink MAPs. A BS schedules the use of resourcesboth on the downlink and the uplink. The DL-MAP indi-cates where each burst is placed in the frame, which MS it isintended for, and what modulation (MCS) decodes it. Sim-ilarly the UL-MAP indicates where the MS should place itsdata on the uplink frame. The uplink frame has dedicatedsub-channels for HARQ which is used by the MSs to explic-itly acknowledge (ACK/NACK) the reception of each burst.

3. DESIGN ASPECTS OF FERMITo derive the right design choices for interference mitiga-

tion, we perform extensive measurements on our femtocelltestbed.

Experimental Setup: Our testbed consists of four fem-tocells (cells 1-4) deployed in an indoor enterprise environ-ment (Fig. 1(b)). We use PicoChip’s [15] femtocells that run802.16e (WiMAX). Our clients are black boxes using com-mercial WiMAX cards from Accton [16]. The cells operateon a 8.75 MHz bandwidth with the same carrier frequencyof 2.59 GHz. For this frequency, an experimental license hasbeen obtained to transmit WiMAX signals on the air.

We consider downlink UDP traffic from the BSs to the cl-ients generated byiperf. The traffic rate is set large enoughto saturate the available resources. Each data point corre-sponds to an interference topology and is obtained by run-ning an experiment for 7 minutes, measuring the throughputand averaging it over several such runs. We generate differ-ent interference topologies by varying the locations of theclients (along the path shown in Fig. 1(b)). Moving theclients provides a finer control on the inter-BS interferencemagnitude as opposed to changing the locations of the BSs.More importantly, note here that we only need to accountfor whether or not a client of a BS is interfered by anotherBS. This is unlike in WiFi, where in dense deployments, a1We use the terms femtocell, BS, cell interchangeably.

2

PREAMBLE

Sub-channels

FCH

DL-MAP

UL-MAP

User 1

DL Burst

User 2

DL Burst

User 3

DL Burst

Symbol Duration

User 1

UL Burst

User 2

UL Burst

User 3

UL Burst

Transition Gap

(a) WiMAX Frame Structure

1

3

2

Femto Client locations

4

(b) Testbed Deployment

Time (symbols)

Frequency

BS

1

BS

2

Frequency

Time (symbols)

(a) TDI

Time (symbols)

Frequency

BS

1

BS

2

Frequency

Time (symbols)

(b) FDI

PREAMBLE CONTROL DATA EMPTY

(c) Illustration of TDI and FDI

Figure 1: Illustrations for frame structure, deployment and resource isolation alternatives.

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10 12 14 16 18

CD

F

Throughput(Mbps)

BaselineTDIFDI

(a) Resource Isolation

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

CD

F

Throughput Gain

2 Cells3 Cells

(b) TDI vs FDI

Figure 2: Benefits of FDI over TDI.

WiFi AP can preclude the transmissions of a nearby AP dueto carrier sensing. In other words, in an OFDMA setting, thelocations of the clients (rather than the BSs) are important.Thus, we believe that our setup captures a reasonable set ofscenarios that could arise in practical deployments.

The baseline strategy for our measurements is one wherea BS operates on its entire spectrum, while performing anideal link adaptation (MCS selection) for its clients. For eachof our data points, we run the experiment over all MCS levelsand record the one that delivers the highest throughput.

Coping with Interference: There are two approaches tocoping with interference in OFDMA. Switching to a lowerMCS level via link adaptation (rate control) could suffice ifthe received signal quality is above the threshold requiredby the lower MCS level. With strong interference (typicalin dense deployments), the received SINR could be evenlower than that required for the lowest MCS operation. Iso-lating the resources utilized by interfering cells helps alle-viate the effects, but it results in a reduced set of transmis-sion resources in each cell. Clearly, the choice between linkadaptation and resource isolation must be made dependingon the nature of interference. In a two-dimensional WiMAXframe, resources can be isolated among BSs either in time(symbols) or in frequency (sub-channels) as depicted in Fig.1(c). Time domain isolation (TDI) isolates resources in timeby leaving empty (guard) symbols to prevent collisions; fre-quency domain isolation (FDI) allocates orthogonal sets ofsub-channels to different BSs for their transmissions.

Our goal is to answer:Does link adaptation suffice incoping with interference or is resource isolation needed? Ifneeded, should resource isolation be in time or in frequency?

Towards this, we experiment with three strategies: (a) thebaseline strategy where, BSs operate using all resources, (b)TDI and (c) FDI where, BSs operate using half of the (or-thogonal) available set of symbols and sub-channels, respec-tively, in each frame (Fig. 1(c)). All strategies employ linkadaptation via cycling through MCS levels. We first considercells 1 and 2, and present the CDF (over the client locations)of the aggregate throughput in Fig. 2(a). We see that re-source isolation provides significant gains over the baselineand that FDI outperforms TDI in aggregate throughput byabout 20%. We repeat these experiments with cells 1, 2 and3. Each cell operates on a third of the resources with TDI orFDI. In Fig. 2(b), we see that the median percentage thro-ughput gain of FDI over TDI increases from about 17% fortwo cells to about 60% for three cells.

This interesting observation is due to what we refer to aspower pooling, only possible with FDI. The energy trans-mitted by a BS is split over its constituent sub-channels inOFDMA. With a smaller subset of sub-channels, the aver-age power per sub-channel increases, potentially allowingthe cell to operate using a higher MCS. As more cells areactivated, the number of (orthogonal) sub-channels availableper cell decreases; this however, increases the average powerand hence the throughputper sub-channel. Eventually, thehigher per sub-channel throughput in each cell contributestothe higher network throughput capacity. We notice that theMCS supported by client 1 is indeed higher with FDI thanTDI. The average MCS difference between FDI and TDI is1 level in the 2 cell topology and almost 2 levels with 3 cells.

Accommodating Heterogeneous Clients:As discussedearlier, for clients in close proximity to their BS link adap-tation may be sufficient to cope with interference. Invok-ing resource isolation for such clients will underutilize re-sources. Given that OFDMA multiplexes multiple clienttransmissions in a frame (to saturate resources), it becomesnecessary to accommodate clients with heterogeneous re-quirements (link adaptation vs. resource isolation) in thesame frame.

Towards this, we propose to usezoning, where an OF-DMA two-dimensional frame is divided into two data trans-mission zones. The first zone operates on all sub-channelsand is used to schedule clients that need just link adaptation(hereafter referred to asreuse zone). The second zone uti-

3

-0.5

0

0.5

1

0.1 0.2 0.3 0.4 0.5

Thr

ough

put G

ain

Fraction of Distance Between Cells

Reuse ClientAggregate

(a) Zoning Gain

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

CD

F

Throughput Difference

InterferenceSingle Cell

(b) Calibration Measurements

Figure 3: Motivation for zoning (a) and calibrating mea-surements for categorization (b).

lizes only a subset of sub-channels (determined by FDI) andhere, clients that require resource isolation are scheduled (re-ferred to asresource isolation zone). Link adaptation is alsoperformed for clients in this zone.

We perform an experiment with two cells to understandthe benefits of zoning. Cell 2 causes interference while cell1 transmits data to its clients. Cell 1 schedules two clients:one by reusing all sub-channels (reuse client) and the otherone by isolating resources (from cell 2). The reuse clientis moved from the proximity of cell 1 towards cell 2; theother client is static. We compare the throughput that cell 1achieves against a scheme where there is no reuse (both cli-ents are scheduled by isolating resources). As one might ex-pect, as long as the reuse client does not experience apprecia-ble interference from cell 2, reusing sub-channels providesa throughput gain over the pure resource isolation scheme.We plot these throughput gains in Fig. 3(a) as a function ofthe reuse client’s distance from cell 1. Interestingly, signif-icant gains (at least 20%) from reusing sub-channels can beavailed even when the client is at 40% of the distance be-tween the interfering cells. Beyond this distance, the inter-ference from cell 2 degrades throughput. We revisit zoningwhen we describe the algorithms in FERMI in Sec. 5.

Although zoning holds promise, it only dictates how toaccommodate heterogeneous clients; it does not provide acomplete resource management solution. Several challengesremain in achieving this goal. Specifically, for each BS, weneed to(a) determine the size (in symbols) of the reuse zone(b) determine the subset of sub-channels allocated to the re-source isolation zone, and(c) adapt both these zones to thedynamics of the network in a scalable manner. FERMI in-corporates novel algorithms to address these challenges.

4. BUILDING BLOCKS OF FERMIWe depict the relationship between the blocks of FERMI

in Fig. 4. In a nutshell, thecategorizationof clients al-lows the BS to determine how the frame should be dividedinto zones, from its perspective (block 1). The BS then de-termines the set of BSs that cause interference on those ofits clients that require resource isolation (block 2). Thisin-formation is then fed to the central controller (CC) alongwith cell-specific load parameters. The CC then constructs

Categorize

clients using

dedicated

measurement

zones

Determine set of

interfering BSs

via passive

measurements

Client Categorization Conflict Graph Generation

Resource

Allocation

at CC

1 2

Load parameters

List of Interferers

Figure 4: The building blocks of FERMI.

an interference map and computes the network wide sub-channel allocation and zoning parameters (details in Sec. 5).It disseminates this information back to the BSs, which usethese operational parameters until the next resource alloca-tion update. Note that the CC is similar to the notion ofa self-organizing network (SON) server [14] maintained bythe service provider. Next, we explain the client categoriza-tion and the interference map generation components.

Client Categorization at Femto BSs:The first buildingblock categorizes clients into two classes; the first needs onlylink adaptation (class 1) while the second needs resource iso-lation (class 2). To understand how clients are to be catego-rized as either class 1 or class 2, we perform calibration ex-periments. We consider two cells each with a single client.We experiment over a large set of client locations to generatea plurality of scenarios. We first consider a cell in isolation(i.e., no interference). At each client location, we experi-ment by sequentially allocating two spectralparts (of equalsize) of the frame to the client. Since, the fading effects onthe two sets of assigned sub-channels are likely to be differ-ent, the client will receive different throughputs with thetwodifferent allocations.We notice however, that the differencebetween the two allocations is at most 25 % in more than90 % of the considered client locations(Fig. 3(b)). We nowrepeat the experiment, but with interference. In one of theallocations (i.e. parts), the second cell projects interferenceon the client; in the other the operations are without interfer-ence (via resource isolation).We observe that in this case,there is a throughput difference of over 25 % (in many cases,significantly higher) in more than 90 % of the topologies.

These results suggest that the throughput (per unit resource)difference at a client between an interference-free allocationand an allocation with interference can be used to categorizeit as class 1 or class 2. If this difference is less than a thresh-old (referred to asα later), link adaptation suffices for thisclient. However, if it is larger than the threshold, one cannotimmediately determine if the client needs resource isolation.This is because the above experiments were done by allocat-ing equal resources to the client in the settings with and with-out interference. If such a client is categorized as class 2 andallocated a smaller set of isolated resources (based on cell’sload), the throughput it achieves may in fact only be similarto what it would achieve by being a class 1 client. Unfortu-nately, it is difficult to know the cell loads a priori and hence

4

0

3

6

9

12

0 20 40 60 80 100 120 0

0.2

0.4

0.6

0.8

1Th

roug

hput

(Mbp

s)

Burs

t Del

iver

y R

atio

Time (sec)

ThroughputBDR

(a) BDR Estimate vs. Throughput

PREAMBLE CONTROL DATA EMPTY

Frequency

Time

OCCUPIED

FREE

REUSE

RESOURCE

ISOLATION

(b) Measurement Zones

0

2

4

6

8

10

12

14

0.25 0.5 0.75 1 1.5 2

Ave

rag

e E

rro

r (%

)

Alpha

500 samples250 samples125 samples

50 samples25 samples

(c) Class 2 Ground Truth

0

10

20

30

40

50

60

0.25 0.5 0.75 1 1.5 2

Ave

rag

e E

rro

r (%

)

Alpha

500 samples250 samples125 samples

50 samples25 samples

(d) Class 1 Ground Truth

Figure 5: Client Categorization Components (a-b) and Accuracy Results (c-d).

one cannot make a clear determination of whether to catego-rize these clients as class 1 or class 2. Thus, FERMI takes aconservative approach and categorizes all of such clients asclass 2. We find that this helps accommodate fluctuations inthe load and interference patterns.

Although a BS does not have access to the throughput ata client, it is informed about the reception of each burst viaACKs on the uplink. We define Burst Delivery Ratio (BDR)to be the ratio of successfully delivered bursts to the totalnumber of transmitted bursts. The BS canestimateBDRby taking the ratio of the number of ACKs received to thetotal number of feedbacks received. Since the feedback itselfmight practically get lost on the uplink, this is an estimateofthe actual BDR. We perform experiments to understand ifthe BDR estimate at the BS can provide an understanding ofthe throughput at the client. Fig. 5(a) shows that indeed theBS can very accuratelytrack the client throughput using theBDR estimates.

To achieve categorization in practice, FERMI introducestwo measurement zones in the frame as indicated in Fig.5(b), namely theoccupiedand free zones. Every BS op-erates on all sub-channels in theoccupiedzone. Schedulinga client in this zone enables the BS to calculate the BDRin the presence of interference from other cells. Schedulinga client in thefree zone to calculate the BDR without in-terference is slightly tricky. Given a set of interfering BSs,all BSs but for one must leave thefree zone empty in anyframe. Allowing only one of the interfering BSs to sched-ule its clients in thefreezone, will enable it to measure thenon-interference BDR at its clients. Hence, a random ac-cess mechanism with probabilityγ

nis used to decide access

to the free zone, wheren is the number of interfering BSsandγ ≥ 1 is a constant parameter set by the CC. Note thatclients associate with BSs at different instants and hence itis unlikely that all interfering BSs will categorize their cli-ents at the same time. Hence,γ is used to increase the ac-cess probability to thefreezone. FERMI schedules regulardata bursts in the measurement zones to calculate the BDR(a good throughput estimator), thereby keeping the processtransparent to clients and retaining standards compatibility.While theoccupiedzone can be used as an extension to thereuse zone when categorization of the clients is completed,this is not possible for thefreezone, whose utility is towards

categorization in other cells. Here, the central controller thatkeeps track of client (dis)associations, triggers the use of thefreezone (cast as a data zone) solely for the purpose of cate-gorization in relevant parts of the network and disables it tominimize overhead once the procedure is complete.

The accuracy of client categorization is evaluated in Figs.5(c) and 5(d). We again consider two cells; clients 1 and 2belong to the two cells, respectively. We generate multipletopologies by varying the location of client 1 in the pres-ence of interfering cell 2. First, the throughput of client 1ismeasured for both zones (freeandoccupied) to identify theground truth at each location; here leveraging our previousmeasurements, we conclude that if the throughput differenceis less than 25%, client 1 is at a location where it only needslink adaptation. Otherwise, the particular scenario is deemedas one that needs resource isolation. After the ground truthisestablished, cell 1 collects BDR samples from both measure-ment zones to decide on the client category. The decision ismade based on these samples: if the averagefreezone BDRis at leastα% higherthan the averageoccupiedzone BDR,then the client category is class 2. Here, arbitrating the ac-cess to thefree zone is a factor that reduces the accuracyof estimation. If two BSs schedule their clients in this zoneat the same time, rather than getting a BDR sample with-out interference, they both could get a sample that indicatesinterference. Averaging the BDR over multiple samples isused to alleviate this.

The categorization accuracy when the ground truth is (a)resource isolation and (b) link adaptation is plotted in Figs.5(c) and 5(d), respectively. In corroboration with our mea-surement based inference, it can be seen that increasingαbeyond 0.25 decreases the accuracy of detecting resourceisolation but conversely it increases the accuracy of detect-ing link adaptation. Further, while increasing the number ofsamples over whichα is measured can help improve accu-racy, the benefits are not significant. Hence, it pays to usefewer samples to categorize clients (towards reducing over-head). Thus, FERMI uses anα of 1 with 25 frame samplesto obtain an accuracy greater than 90%.

Interference Map Collection at CC: The CC in FERMIgenerates an interference (conflict) map between BSs thatnot only incorporates point-to-point but also cumulative in-terference at clients. Note that interference is client depen-

5

dent and since multiple clients are scheduled in tandem ineach OFDMA frame, the interference patterns between BSsvary from one frame to another. This makes it impossi-ble for any practical resource management scheme to gatherschedule-dependent interference information, determineanallocation and disseminate it to the BSs for execution in ev-ery frame. Hence, the goal of the resource managementscheme in FERMI is to allocate resources at a coarse timescale granularity (over hundreds of frames) by collectingaggregateinterference statistics from each BS. This decou-ples resource allocation from frame scheduling in each BS,thereby allowing a conflict graph approach to adequatelycapture interference dependencies for our purpose.

In addition to client categorization, the measurement zonesin FERMI also help in deciphering interference relations. Ifa BS causes interference to the clients of another BS so asto require resource isolation, then an edge is added betweenthe two BSs in the conflict graph. Note that the interferencerelations need to be determined only for class 2 clients.

Measurements in theoccupiedzone are used as the ba-sis to categorize a client as class 2. Note however, thatallBSs operate in this zone and thus, the client experiences thecumulative interference from all interfering BSs. Addingan edge to each of these neighboring cells in the conflictgraph would be overly conservative; some of them may onlyproject weak levels of interference on the client. Hence, weneed to determine the minimum set of interference edges thatneed to be added in the conflict graph to eliminate interfer-ence through resource isolation. Towards this, we use thefollowing procedure following the initial categorization.

Consider a femtocellA and a class 2 clientcl of A. cl pas-sively measures the received power from neighboring BSs(available during handover between BSs). If the power froma neighboring BS (B) exceeds a threshold, thenB is addedto cl’s list of strong interferers.cl reports this list toA, whichthen consolidates it and reports the set of conflict edges (foreach strong interferer) that must be added to the conflictgraph to CC. CC uses this information for making the initialresource allocation decision. While this accounts for point-to-point interference, some clients may not see any individ-ual strong interferer but the cumulative power from a subsetof neighbors could be strong enough to require resource iso-lation. Such clients will continue to see interference after theinitial resource allocation. These clients can be identified bycomparing the BDR achieved on the assigned sub-channelswith that seen in thefreezone.

To illustrate, let us consider one such client. The clientreports a list of neighbor BSs (not already reported) in de-creasing order of received power as potential cumulative in-terferers to its BS. Now, FERMI needs to identify the small-est subset of these neighbors to whom adding a conflict edgewill eliminate interference through resource isolation. Toachieve this two filters are applied: (i) from the cumulativeinterference list sent by the client, the BS first removes thoseneighbors from the list to whom an edge was already added

by one of its other clients; (ii) the pruned list is sent to theCC, where neighbors whose current resource allocation areorthogonal to that of the client’s cell are further removed asthey could not have caused interference. The CC then looksat the final pruned lists and adds a conflict edge to the neigh-bor that appears first in multiple lists. With the updated inter-ference graph, resource allocation is determined once againand disseminated to femtocells for the next resource alloca-tion epoch. If the BDR for the client is sufficiently improvedand is now withinα% of what is observed in thefreezone,the process is complete. If not, the next strongest interferingBS is added to the conflict graph (again subject to filteringbased on the current resource allocation) and so on. In ad-dressing cumulative interference, conflict edges are addedonly one by one in each epoch by the CC. This is becausemost of the interference experienced by clients is strong indense femtocell deployments. Hence, aggressively addingconflict edges to address cumulative interference will onlyresult in under-utilization of resources. Thus, using bothpas-sive and active measurements at clients, FERMI accounts forboth strong and cumulative interference in its resource allo-cation decisions in each epoch.

Why Dedicated Measurements?: One could argue thatusing only the passive received power measurements frominterfering BSs is an easier approach to categorize clients.Here, if a client receives a signal from an interfering BS thatis higher than a threshold, it is categorized as class 2; oth-erwise, it is a class 1 client. However, for this method towork well in practice, a lot of calibration is needed to findaccurate, often scenario dependent, threshold values. In ad-dition, the received power does not necessarily give an in-dication of the throughput observed at the clients. To avoidthese practical issues, FERMI relies on highly accurate di-rect measurements for client categorization, which allowsitto have coarse thresholds for identification of strong interfer-ers. Having categorized the clients and identified the inter-ference dependencies between femtocells, we are now readyto present the resource allocation algorithms at the CC.

5. ALGORITHMS IN FERMIThe goal of resource management at CC is to determine

for each femtocell (i) the size of thereusezone and, (ii) thespecific subset of sub-channels for operations in theresourceisolationzone to obtain an efficient and fair allocation acrossfemtocells. While the joint determination of parameters forboth the zones is the optimal approach, this depends on thro-ughput information that changes in each frame, thereby cou-pling resource allocation with per-frame scheduling deci-sions. Since, as discussed in Sec. 4, per-frame resourceallocation is infeasible due to practical constraints, FERMIperforms resource allocation at coarse time scales.

Each femtocell reports two parameters to the CC to facil-itate resource allocation: (i) load (number of clients) in itsresource isolationzone, and (ii) desired size (in time sym-bols) of itsreusezone. Alternative definitions for load can be

6

adopted but number of clients is sufficient for our purposes(as in [9]). Note that a femtocell does not have the completepicture of interference dependencies across cells; it onlyhasa localized view. Thus, it simply provides the load in itsresource isolation zone and expects the CC to allocate re-sources proportional to its load. Each femtocell determinesthe desired size of its reuse zone based on the relative load inthe two zones. Since class 2 clients will be scheduled imme-diately after the reuse zone (see Fig. 5(b)), if two interferingcells have different sizes for their reuse zones, then the cellwith the larger reuse zone will cause interference to the class2 clients of the other cell. Hence, an appropriate size for thereuse zone of each cell also needs to be determined by theCC based on the reported desired values. Next, we presentthe algorithm at the CC to determine the sub-channel allo-cation and assignment to each femtocell, followed by theselection of their reuse zone sizes.

5.1 Allocation and AssignmentThe goal of the sub-channel allocation component in FERMI

is to allocate and assign sub-channels to the resource isola-tion zone in each femtocell so as to maximize utilization ofsub-channels in the network subject to a weighted max-minfairness model. The reasons for the choice of the weightedmax-min fairness are two fold: (i) weights account for vari-ations in load across different cells; and (ii) max-min allowsfor an almost even split of sub-channels betweens cells ina contention region, which in turn maximizes the benefitsfrom power pooling (see Sec. 3). Thus, given the load forthe resource isolation zone from each femtocell along withthe conflict graph constructed, the CC’s goal is to determinea weighted (load based) max-min allocation of sub-channelsto femtocells (i.e. vertices in the graph).

THEOREM 1. The sub-channel allocation and assignmentproblem in FERMI is NP-hard.

PROOF. Consider the simpler version of the problem, wherewe are interested only in the optimum objective (max-min)value and not in the specific allocation to all the cells. Hence,we are interested only in determining the largest number ofsub-channelsx such that all cells can at least be given anallocation ofx. This in turn can be determined as follows.For a given integeri, replace each vertex (femtocell) in theconflict graphG = (V, E) by a clique of sizei and addedges between all vertices across the cliques if there existedan edge between the vertices corresponding to the cliques inthe original graphG. Let the resulting graph beGi. Now,determining the maximumi such thatGi is colorable withNcolors (N being the # of sub-channels) yields the desired ob-jective value. However, determining ifGi is colorable withN colors is the problem of multi-coloringG, where eachvertex requires as assignment ofi colors. Thus, the max-min value to our simpler problem isx if and only if G canbe x-fold colored withN colors. Since multi-coloring isNP-hard and forms a special case of a simpler version of ourproblem, the hardness automatically carries over.

While the allocation problem in FERMI may seem sim-ilar to multi-coloring at the outset, this is not the case. Infact, multi-coloring can only provide an assignment of sub-channels for a specified allocation. However, in FERMI weare also interested in determining a weighted max-min allo-cation in addition to the assignment, which makes the prob-lem much more challenging. Further, every contiguous setof sub-channels allocated to a cell is accompanied by an in-formation element in the control part of the frame (MAP),describing parameters for its decoding at the clients. Thisconstitutes overhead, which in turn increases with the num-ber of discontiguous sets allocated to a cell. Therefore, ourgoal is toreduceoverhead due to discontiguous allocations,while ensuring an efficient allocation. We presentA3, the al-location and assignment algorithm in FERMI that achievesthe aforementioned objectives.Overview ofA3: Any resource allocation algorithm attemptsto allocate shared resources between entities in a contentionregion subject to a desired fairness. Each contention re-gion corresponds to a maximal clique in the conflict graph.However, a given femtocell may belong to multiple con-tention regions and its fair share could vary from one re-gion to another. This makes it hard to obtain a fair alloca-tion, for which it is necessary to identify all maximal cliquesin the conflict graph. However, there are an exponentialnumber of maximal cliques in general conflict graphs withno polynomial-time algorithms to enumerate them. Hence,we propose an alternate, novel approach to resource alloca-tion in A3, which is both polynomial-time as well as pro-vides near-optimal fair allocations with minimal discontigu-ity (overhead). The three main steps inA3 are as follows.

Algorithm 1 Allocation and Assignment Algorithm:A3

1: Triangulate: A3 first transforms the given conflictgraphG into a chordal graphG′ by adding a minimalset of virtual interference edges toG = (V, E).

2: Allocate and Assign: A3 computes a provablyweighted max-min allocation on the chordal graphG′.

3: Restore:A3 removes the virtual edges fromG′ and up-dates the allocation to the vertices carrying the virtualedges to account for under-utilization on the originalgraphG.

A chordal graph does not contain cycles of size four ormore. Very efficient algorithms for important problems likemaximum clique enumeration can be applied on chordal graphs[17]. The key idea inA3 is to leverage the power of chordalgraphs in obtaining a near-optimal allocation. In the interestof space, we do not present details of some aspects of thealgorithm, whose solutions exist in literature; the readercanperuse the cited references. We now present details of thethree steps inA3 along with a running example in Fig. 6.Triangulation: The process of adding edges to triangulate(chordalize) a graph is known asfill-in . Since adding edgesto the conflict graph would result in a conservative alloca-

7

Triangulation

B (1)

C (2)

A (1)

D (1)

F (1)

E (2)

Vertex Initial Allocation

C min(8, 10, 10) = 8

D min(6, 5) = 5

G 15

E 8

A min(7,6) = 6

F 4

Clique Tree

Restoration

N/A

N/A

N/A

[12:19]

N/A

Final Allocation

8

5

15

8

14

4G (3)

DG

ACD

ABC CEF

Assignment Benchmark

B 6

[1 : 8]

[1 : 5] + [20]

[1 : 5]

[12 : 19]

[6 : 20]

[6 : 11]

[9 : 11] + [20]

N/A

none

6

8

5

15

8

10

4

10

[a : b] denotes the set of sub-channels from a to b (inclusive)

Figure 6: Illustration of A3 algorithm for 20 sub-channels in the spectrum. The vertex loads are included in parentheses.

tion than is required, the goal is to add theminimumnum-ber of edges needed for triangulation. While this is a NP-hard problem in itself,A3 employs a maximum cardinalitysearch based algorithm [18] that is guaranteed to produce aminimal triangulation and runs in timeO(|V ||E|)). Fig. 6depicts a fill-in edge between verticesA andC. As we shallsubsequently see, the restoration (third) step inA3 is used toalleviate the under-utilization introduced by triangulation.Allocation: A3 uses the following algorithm to determinethe weighted max-min allocation on the triangulated graphG′. Once the graph is triangulated, all its maximal cliques

1: INPUT: G′ = (V, E′) and loadℓi, ∀vi ∈ V2: Allocation:3: Un-allocated verticesU = V , Allocated verticesA = ∅4: Determine all the maximal cliquesC = {C1, . . . , Cm}

in G′ using perfect elimination ordering5: Resource:Rj = N , Net load:Lj =

∑

i:vi∈Cjℓi, ∀Cj

6: Determine tuples:si = maxj:vi∈Cj{Lj},

ti =∑

j 1vi∈Cj, ∀vi

7: Determine initial allocation:

Ai = minj:vi∈Cj

⌊

ℓiRjP

k:vk∈Cjℓk

+ 0.5

⌋

, ∀vi ∈ U

8: while U 6= ∅ do9: Pick un-allocated vertex with maximum lexico-

graphic rank:vo = arg maxi:vi∈U (si, ti)10: AllocateAo sub-channels tovo; U ← U\vo,

A ← A∪ vo

11: Update remaining resource:Rj = Rj −Ao,∀j : vo ∈ Cj

12: Removevo from cliques:Cj ← Cj\{vo}, ∀j : vo ∈Cj ; UpdateLj ∀j and(si, ti) ∀vi ∈ U

13: Update allocation:

Ai = minj:vi∈Cj

⌊

ℓiRjP

k:vk∈Cjℓk

+ 0.5

⌋

, ∀vi

14: end while

are listed in linear time (O(|V |)) by determining a perfectelimination ordering (PEO) [18].A3 determines the net loadon each maximal clique (step 5) and for every un-allocatedvertex (cell,vi), it determines a tuple(si, ti), wheresi indi-cates the highest load in the cliques thatvi belongs to andti

is the number of cliques that it belongs to (step 6).A3 thendetermines its weighted fair share in each of the maximalcliques it belongs to and determines its minimum (rounded)share amongst all its member cliques (step 7). It picks thevertex (vo) with the highest lexicographic rank and allocatesthe computed share of sub-channels to it (vertexC is pickedfirst with sc = 5 and tc = 3). vo is then removed fromthe list of un-allocated vertices (steps 8-10). The allocatedvertex is removed from its member cliques, and the cliqueload, resource and vertex tuples are correspondingly updated(steps 11,12). The weighted share for the remaining set ofun-allocated vertices in each of the maximal cliques thatvo

belongs to is updated based on the remaining resources inthose cliques (step 13). The process is repeated untill allvertices receive allocation and runs in timeO(|V |2).Assignment: After the vertices get their weighted max-minallocation, the next step is to provide an actual assignmentofsub-channels to satisfy the allocations.A3 leverages cliquetrees for this purpose. A clique tree for a chordal graphGis a tree whose nodes are maximal cliques inG. Further, itsatisfies some useful properties (as we show later).

A3 generates a clique tree for the chordal graphG′ (de-picted in Fig. 6) in linear time by building on top of a PEOor by constructing a maximum spanning tree [17]. It picksan arbitrary node in the clique tree as its root and starts sub-channel assignment proceeding from the root to its leaves.At every level in the tree, it assigns sub-channels to un-assigned vertices in each of the nodes (maximal cliques)based on their allocation (vertexD is assigned first with sub-channels [1:5]). When assigning sub-channels to a vertex, acontiguous set of sub-channels that is disjoint with existingassignments to other vertices in the same clique is achieved.When contiguous assignment is not possible, assignment ismade to minimize fragmentation (vertexB is assigned twofragments). Since a vertex may belong to multiple cliques,once its assignment is made, it is retained in all subsequentlevels of the tree. We establish later that the above proce-dure that runs inO(|V |) can yield a feasible assignment ofsub-channels to satisfy the allocation.Restoration: Fill-in edges could result in conservative (under-utilized) allocation of resources. While the triangulation inA3 attempts to reduce the addition of such edges, we still

8

need a final step to restore potential under-utilization.A3 re-visits vertices carrying fill-in edges and removes such edgesone by one. When a fill-in edge is removed, the removalof a conflict may free up some sub-channels at each of thevertices carrying the edge. If so, the largest set of such sub-channels (that do not conflict with the assignment of neigh-bor vertices) are directly assigned to those vertices (for ver-tex A, sub-channels [12:19] are freed after the conflict re-moval with C and can be re-assigned toA). This can bedone inO(|V |).

To summarize, given the exponential number of cliquesin the original graph,A3 intelligently transforms the graphinto a chordal graph with only a linear number of cliques andoptimally solves the allocation and assignment problem.A3

keeps the potential under-utilization due to virtual edgesto aminimum with its triangulation and restoration components.Thus, it provides near-optimal performance for most of thetopologies with a net running time ofO(|V ||E|). We nowestablish two key properties ofA3.

PROPERTY 1. A3 produces a weighted max-min alloca-tion on the modified graphG′.

PROOF. Before presenting the proof, we recap the defini-tion of max-min fairness, which needs to be slightly modi-fied given that fractional channel allocations are not possi-ble. An allocation vector is said to bemax-minif it is notpossible to increase the allocation (x) of an element with-out decreasing that of another element with an allocation ofx + 1 or lesser. The correspondingweighted max-mindef-inition requires that an increase in the allocation (x) of anelement with weightwi is accompanied by a decrease in theallocation of another elementj (with weightwj) with an al-location of wjx

wi+ 1 or lesser.

Note that a weighted max-min allocation onG′, where theweights correspond to the load on the vertex, is equivalentto a max-min allocation on a transformed graphG∗, whereeach vertexvi in G′ is replaced by a clique withℓi nodes(sub-vertices) inG∗ and an edge between two vertices inG′

translates to an edge between all sub-vertices of the two ver-tices inG∗. A cliqueCm with m vertices inG′ now trans-lates to a clique with

∑

i:vi∈Cmℓi sub-vertices inG∗. Thus,

to showA3’s allocation mechanism yields a weighted max-min allocation onG′, it is sufficient to show that it yields amax-min allocation inG∗ where each sub-vertex has unityload (weight). We will show this by contradiction.

Assume a given allocation is not max-min. Hence, con-sider two sub-verticesva, vb in a maximal cliqueC1 withrespective allocations beingx andx + 2. For the allocationto not be max-min, we should be able to increaseva’s al-location either in isolation or by decreasingvb’s allocation.Sinceva’s allocation was restricted to begin with, it must be-long to a bigger clique thanC1, namelyC2. This is because,when all the loads are unity, the allocation mechanism picksvertices belonging to bigger cliques first. Now, ifva’s allo-cation can be increased by one sub-channel, then this must

also not exceed the resource capacity ofC2. There are threepossible cases.(i) If va was allocated after other sub-vertices inC2, thenva’s allocation would have already expanded to occupy theremaining capacity ofC2, which means thatva’s allocationcannot be increased further.(ii) If va was not the last sub-vertex inC2 to be allocated,then it might be that the sub-vertices that followedva werebottlenecked in other larger cliques and could only receivean allocation< x, thereby allowingva’s allocation to be fur-ther increased to use up the remaining capacity inC2. How-ever, this is not possible since the sub-vertices that follow va

in A3 can only belong to equivalent or smaller cliques andwill hence be able to receive an allocation≥ x, thereby usingupC2’s capacity completely. Thus,va’s allocation cannot beincreased in this case as well.(iii) Similar to vb, let there by another sub-vertexvc in C2

that has an allocation ofx + 2 or more. In this case,va’sallocation can be increased at the cost ofvb’s andvc’s al-locations in the two cliques. However, note that ifvc wasallocated beforeva, then its allocation will be≤ x. Further,sinceva is bottlenecked inC2, if vc is allocated afterva, thenits allocation can at most bex + 1. Thus, in either of thesecases,va’s increase will have to come at the cost of anothersub-vertex with an allocation≤ x + 1.

PROPERTY 2. A3 always produces a feasible assignmentof sub-channels for its allocation.

PROOF. A3 generates a clique tree forG′ and starts as-signing sub-channels to vertices in each of the nodes (cliques)in the clique tree starting from the root.A3 can run intoassignment problems if it encounters an un-assigned vertexvi belonging to multiple cliques at the same level with con-flicts such that it prevents a feasible assignment tovi. This iswhere theclique intersection propertyof clique trees comeinto play.

The clique intersection property states that for every pairof distinct cliquesC1, C2, the set of vertices inC1 ∩C2 willbe contained in all the cliques on the path connectingC1 andC2 in the tree. Hence, ifvi appears at multiple cliques at alevel in the tree, it must have appeared in isolation at somehigher level in the tree. Given that all vertices, when en-countered first, appear in a single clique at a level, afeasibleassignment to the vertex is always possible since the alloca-tions always satisfy the capacity in all cliques. Further, oncea vertex is assigned sub-channels, its assignment carries overto other cliques containing it in subsequent levels. Thus, byusing a clique tree for assignment,A3 is able to ensure a fea-sible assignment of sub-channels given a weighted max-minallocation.

Based on these two properties, we have the following result.

THEOREM 2. If G is chordal, thenA3 produces an opti-mal weighted max-min allocation.

Through our comprehensive evaluations in Section 6, weshow that over 70% of the topologies are chordal to begin

9

with for which A3 yields an optimal allocation. For the re-maining topologies,A3 sub-optimality is within 10%, indi-cating its near-optimal allocation.

Other possible comparative approaches: While greedyheuristics for multi-coloring do not address our allocationproblem, to understand the merits ofA3, we propose andconsider two extensions to such heuristics that also performallocation and assignment (coloring).

The first heuristic isprogressive(labeledprog); alloca-tions and assignments are made in tandem one channel ata time. The vertex with the smallest weighted allocation(allocation

load = Ai

ℓi) is chosen and assigned the smallest index

channel that is available in its neighborhood. By assigningchannels one at a time, this heuristic is able to achieve goodfairness. However, its running time isO(|V |2N), where itsdependence onN (number of sub-channels) makes it pseudo-polynomial, thereby affecting its scalability. Further, it re-sults in a highly fragmented assignment of channels to ver-tices, which in turn increases the control overhead in frames.

Another heuristic that avoids the pseudo-polynomial com-plexity, is interference degreebased (labeleddeg). The shareto every vertex is determined based on its weight and the re-maining resources (after removing allocated vertices) in its

interference neighborhood and is (ℓi(N−

P

j:(vi,vj)∈E,vj∈A)

P

j:(vi,vj)∈E,vj∈Uℓj

).

Then the vertex with the min. share is allocated as contigu-ous of a set of sub-channels as possible. This heuristic runsin O(|V |2) and also keeps the overhead low. However, itsfairness is significantly worse.

By adopting a greedy approach, heuristics derived frommulti-coloring either achieve low complexity and overheadat the cost of fairness but not both.A3 however, deciphers in-terference dependencies with good accuracy to provide bothnear-optimal fairness and reduced complexity and overhead.Further, since the allocation and assignment is effected onthe chordal graphG′, dynamics in the form of arrival/departureof clients/cells (addition/deletion of edge conflicts) canbeeasily accommodated in a purely localized manner throughincremental schemes [19]. This in turn allowsA3 to scalewell to network dynamics unlike other heuristics.

Benchmarking: To understand how closeA3 is to theoptimum, we need to obtain the weighted max-min alloca-tion on the original graphG. This requires listing of all themaximal cliques, which are exponential in number. This isachieved in a brute-force manner (exponential complexity).Once all the maximal cliques are obtained onG, the allo-cation procedure ofA3 can be directly applied to obtain aweighted max-min allocation onG.

5.2 ZoningWe addressed the assignment of sub-channels to the re-

source isolation zone of each cell. Our next step is to de-termine the size of the reuse zone (in symbols) for each cellbased on their desired sizes. There arise three challengesin determining the reuse zone size (sr). (i) If two inter-

Frequency

Allocation

BS

1

BS

2

BS

3

Desired

Reuse Zone

Size (symbols)

10 15 5

Operational

Zones

Marked in Time

10 5 5

Band A Band B Band A

A

B

A

B

Frequency

Frequency

Frequency

Time Time Time

Reuse Isolation ReuseIsolation

Reuse Isolation

(15)

Class 1 Clients (introduced zone)

Figure 7: Illustration of zoning mechanism of FERMI.

fering cells use two differentsr ’s, the one with the largersr will cause interference to the class 2 clients of the othercell. Hence, a common reuse zone is required among in-terfering cells. (ii) Since allocation and zoning are meantto operate at coarse time scales (decoupled from per-framescheduling), the commonsr among interfering cells cannotbe determined based on throughput. Hence, the choice of thecommonsr is restricted to either the minimum or maximumof the desired zone sizes of the neighboring cells. (iii) If eachcell belongs to a single contention region (clique), choosingthe commonsr is easy. However, since cells may belong tomultiple cliques, this will result in a commonsr (minimumor maximum) propagate to the entire network. Cells with adesired zone size less than the commonsr may not have suf-ficient data for their class 1 clients to fill up to thesr, whilecells with a larger desired zone size will have to perform iso-lation (without reusing sub-channels). Either case results inunder-utilization, which is exacerbated when a single com-monsr is used in the network.

FERMI addresses the above challenge as follows (illus-tration in Fig. 7). For each cell, the CC determines the min-imum of the advertised (desired)sr ’s of all the cell’s neigh-bors and uses that as its operationalsr (e.g. 10 symbolsfor BS1, 5 symbols for BS2). The cell schedules its class1 clients in the reuse zone till the operationalsr (using allsub-channels). It continues to schedule class 1 clients in thesecond zone between its operationalsr and its desiredsr.However, these are scheduled only in the band allocated tothe cell byA3 (the scheduling of BS2 between the5th andthe15th symbols). The class 2 clients are scheduled in theresource isolation zone (after the desiredsr) using the sub-channels (band) allocated byA3.

Introducing a zone (marked on Fig. 7) that schedules class1 clients between the operational and desiredsr ’s (using theband given byA3), provides a graceful transition betweenthe reuse and resource isolation zones. Since the chance forunder-utilization is more when the operationalsr exceedsthe desiredsr, the minimum of the desiredsr ’s in the neigh-borhood is used as the operationalsr for a cell. Further,

10

CC

Client

Femtocell

GPS

Antenna

GPS

Unit

(a) WiMAX Equipment

Categorization

Zoning

Application

Data

Burst

Tracking

Interface

from / to

CC

(a)

(b) (c) (d)(e)

MAC / PHY API

Burst Packing

Frame

Controller

HARQ

Scheduler

Rate

Adaptation

CINR

feedback

from MS

(b) Implementation of FERMI

Topology 1

BS3BS2BS1 BS3

BS2BS1

BS4

Topology 2

BS3

BS2BS1

BS4

Topology 3

BS3BS2BS1 BS4

Topology 4

BS3

BS2BS1

BS4Topology 5

(c) Topologies

Figure 8: WiMAX equipment (a), implementation of FERMI (b) a nd topologies for prototype evaluation (c).

since each cell computes its operationalsr only based onthe desiredsr’s of its neighbors and not their operationalsr ’s, propagation of a single commonsr in the network (andthe resulting under-utilization) is avoided. As an example,this would correspond to every BS having the samesr (i.e.global min.) of 5 symbols in Fig. 7. Using the minimumof the desiredsrs of neighbors (i.e. local min.) avoids thispropagation for BS1 and allows it to have asr of 10 symbols.Hence, different regions of the network can have differentsr

values, which increases the potential for sub-channel reuse.Further, cells that belong to multiple contention regions withdifferent operationalsr ’s in the different cliques (e.g. BS2 inFig. 7) will not see interference to their class 2 clients, sincethe operationalsr of all their cliques will be less than theirdesiredsr, while they schedule only class 1 clients in the re-gion between their operational and desiredsr. As we shallshow in our evaluations, zoning provides significant thro-ughput gains as long as thesr values in different cliquescan be decoupled (i.e. a single globally minimum desiredsr

does not propagate).

6. SYSTEM EVALUATIONTo evaluate FERMI, we both conduct experiments on our

testbed as well as simulations. Simulations help evaluate thescalability and the relative performance of each algorithmwith parameters that are not easy to adapt in practice.

6.1 Prototype EvaluationsImplementation Details: Fig. 8(a) shows a picture of

our testbed equipment. Given that we do not have a macroBS at our disposal, we use external GPS modules to achievesynchronization among femtocells. The GPS modules areplaced next to windows with cables providing a 1 pulse persecond (pps) signal to each femtocell (antenna and cable de-picted). The clients are USB dongles connected to laptops.

FERMI is implemented on the PicoChip platform whichprovides abase reference designimplementation of the Wi-MAX standard. The reference design does not involve so-phisticated scheduling routines and provides just aworkinglink between the BS and the MS. Since the clients are off-the-shelf WiMAX MSs (with no possibility of modification),it is a challenge to realize a working implementation of vari-

ous components such as categorization and zoning. Some ofthese challenges were to keep our implementation within theboundariesof the rigid WiMAX frame structure and to inte-grate commercial clients with our experimental testbed. Wesignificantly extend (shown as colored components in Fig.8(b)) the reference design to implement FERMI. Specifi-cally, our implementation operates as follows.

(a) When data from higher layers is passed onto the MAC,we first route the data based on what MS it is intended forand whether that MS is already categorized (as in Sec. 4) ornot. (b) If the MS is already categorized, its data is packedin the relevant zone of the frame that the MS needs (reuse vs.resource isolation). If not, its data is packed in the measure-ment (recall free and occupied) zones introduced for catego-rization. The burst packing component implements a rectan-gular alignment of the data of both MSs that have been cate-gorized before as well as MSs that are being categorized.(c)After packing, the data is passed onto the frame controllerwhich prepares the control payload before the frame is trans-mitted on the air.(d) Burst tracking component keeps an in-formation tuple for measurement zones for the MSs that arebeing categorized. It tracks the ACK status of each measure-ment burst (populated as list of tuples). After enough BDRsamples are collected, it decides on the client category andinforms the burst packing component about the decision.(e)Interface with the CC leverages kernel sockets to communi-cate the load and conflict information to the CC and receivesoperational parameters for zoning and allocation (used bythe burst packing component).

Experimental Evaluations: We evaluate the performanceof each algorithm using our testbed. We create five topolo-gies as shown in Fig. 8(c). The dotted edge between BS1and BS3 in Topology 2 is thefill-in edge introduced byA3

(other topologies are already chordal). In generating thesetopologies, we leverage our WiMAX testbed (see Fig. 1(b))by changing the client locations for each BS. We measure thefairness of each algorithm relative to the optimal allocation(benchmark) as the normalized distance to the benchmarkd =

√∑

i∈V (ti − si)2 /√

∑

i∈V (si)2 [20] whereti andsi denote the number of sub-channels assigned to vertexi byan algorithm and the benchmark, respectively.

Throughput and Fairness: In our WiMAX deploy-ment, the BSs have 30 sub-channels available in the spec-

11

Topology 1 Topology 2 Topology 3 Topology 4 Topology 5

Algorithm A3 dist deg BM A3 dist deg BM A3 dist deg BM A3 dist deg BM A3 dist deg BM

BS1 (1) 15 15 20 15 20 10 20 15 10 7 7 10 15 15 20 15 20 15 23 20BS2 (2) 15 10 10 15 10 10 10 15 10 10 16 10 15 10 10 15 10 7 7 10BS3 (3) 15 15 20 15 20 10 20 15 10 7 7 10 15 10 10 15 10 10 11 10BS4 (2) - - - - 10 10 10 15 10 10 16 10 15 15 20 15 10 10 12 10

Utilization 45 40 50 45 60 40 60 60 40 34 46 40 60 50 60 60 50 42 53 50

Throughput (Mbps) 20.87 18.36 21.80 - 29.04 19.73 27.86 - 19.61 15.87 20.76 - 26.79 22.72 27.08 - 23.95 19.94 25.06 -

Table 1: Throughput and utilization of each algorithm along with individual allocations (for equal load) for the BS.

0

0.1

0.2

0.3

0.4

0.5

0.6

1 2 3 4 5

Dis

tanc

e fr

om o

ptim

al

Topology

A3

distdeg

(a) Fairness (equal load)

0

0.1

0.2

0.3

0.4

0.5

0.6

1 2 3 4 5D

ista

nce

from

opt

imal

Topology

A3

distdeg

(b) Fairness (variable load)

0

10

20

30

40

50

60

1 2 3 4 5Agg

rega

te T

hrou

ghpu

t(M

bps)

Topology

BaselineA3

A3+Zoning

(c) Throughput

0

5

10

15

20

25

30

2 6 10 14Agg

rega

te T

hrou

hgpu

t (M

bps)

Demand differences

Zoning-LocalZoning-Global

No Zoning

(d) Global vs Local Min.

Figure 9: Fairness and Zoning Benefits ofA3.

trum. Each BS has two clients (one class 1, one class 2).When there is no zoning employed, we schedule both cl-ients in the same set of sub-channels allocated to the BS.For scenarios with zoning, the specific zoning strategy de-termines the size of the reuse zone and the resource isolationzone.We perform experiments for each topology with the al-location determined by each algorithm (assuming equal loadin each BS). Here, we introduce a heuristic (labeleddist) thatdecides the share of a vertex based on its weight and the re-sources in the neighborhood (without removing the allocatedvertices). The share of a vertexi becomes ℓiN

P

j:(vi,vj)∈E ℓj. It

mimics a distributed degree-based allocation and helps usunderstand the importance of having a centralized approach.

Table 1 summarizes the number of sub-channels allocatedto each BS along with utilization and aggregate throughputmeasurements from the experiments. We observe thatdisthas the lowest utilization and therefore the lowest aggregatethroughput. This is because it over-accounts for interferenceby just considering the vertex degrees in allocation.degin-herently penalizes vertices with high degree and allocatesmore resources to the others; it slightly outperformsA3 inutilization and throughput (albeit at the cost of fairness). Fig.9(a) and 9(b) plot the fairness for equal load and variableload (listed next to each BS in parentheses in Table 1), re-spectively. It is seen thatA3 consistently outperforms theother algorithms except topology 2 (equal load case) whereit requires a fill-in edge. However, the restoration step ofA3 can account for under-utilization (due to the fill-in edge)achieving the same utilization as the benchmark. In all othertopologies,A3 achieves the exact allocation as the bench-mark (BM in Table 1) since they are naturally chordal.

Zoning Benefits: We present two measurements forA3

- with and without zoning in Fig. 9(c). The baseline strat-egy is where all femto BSs operate on all available resourceswith link adaptation. We observe that even without zoning,

A3 has significant gains over the baseline. The gains arefurther pronounced when zoning is employed, givingA3 athroughput increase of 50% on average.

Next, we quantify the benefits of decoupling reuse zonedemands in the network (local min.) against having a singlereuse demand propagate to each contention region (globalmin). For this experiment, we use topology 1 in Figure 8(c).We set equal reuse zone demands for BS1 and BS 2 (var-ied in each measurement) and a fixed demand of 4 symbolsfor BS3. Demand difference is defined as the difference be-tween the common demand of BS1 and BS2 and the demandof BS3 (4 symbols). As we vary the demand difference from2 to 14, we measure the aggregate throughput and presentit in Fig. 9(d). It is seen that both global min. and localmin. zoning have increasing throughput as the demand dif-ference increases. For the global min. zoning, although theoperational size is the same (4), the high demand of BS1 andBS2 allow them to schedule their class 1 clients over a largerset of resources (recall the transition zone in Sec. 5). Notethat since class 1 clients are likely to support a higher MCSthan class 2 clients, having a large demand contributes tothroughput gains (as compared to scheduling class 2 clientsin the transition zone). For local min. zoning, the opera-tional size for BS1 is significantly higher as compared to theglobal min. resulting in an increasing throughput gain overthe global min. strategy. This shows FERMI’S benefits fromdecoupling desired reuse zone sizes between different con-tention regions in the network.

6.2 Evaluations with SimulationsSystem Model and Metrics: We implement a simula-

tor to evaluate FERMI in comparison to its alternatives. Thesimulator incorporates a channel model proposed by the IEEE802.16 Broadband Wireless Access Working Group for fem-tocell simulation methodologies [21]. This model captureswireless effects such as log-distance path loss, shadow fad-

12

0

50

100

150

200

250

0 10 20 30 40 50 60 70Tot

al S

ub-c

hann

els

Allo

cate

d

Range (meters)

A3prog.deg.

benchmark

(a) Utilization

0

5

10

15

20

25

30

35

0 10 20 30 40 50 60 70

Tot

al T

hrou

ghpu

t (M

bps)

Range (meters)

A3prog.deg.

benchmark

(b) Throughput

Figure 10: Effect of Range and Throughput and Utilization.

ing and penetration loss, typical of indoor deployments. TheSNR from the model is mapped to a MCS using a rate tablefrom our real WiMAX testbed to compute throughput.

The simulation area is a 7x7 grid where the distance be-tween each grid point is 12 meters. In addition, the widthand height of this area is 100 meters. We simulate a deploy-ment in this area by randomly choosing grid locations foreach femtocell. We then randomly generate a client loca-tion for each femtocell and determine the conflict graph. Wemeasure the overhead of each algorithm as the number ofcontiguous sub-channel chunks allocated per femtocell. Inaddition to overhead, we define thefill-in edge ratio to bethe ratio of number of fill-in edges to the edges that are al-ready present in the conflict graph. If the conflict graph ischordal, the fill-in edge ratio is 0.

Simulation results: Next, we present our simulation re-sults. Each measurement point is an average over resultsfrom 100 randomly generated topologies.

Effect of Femto Range: Fig. 10(a) shows the utiliza-tion for 10 femtocells with 30 sub-channels available in thespectrum and with variable load. It is seen thatdegslightlyoutperforms the other algorithms in total allocated numberof sub-channels. The reason behind this is that the degree-based heuristic inherently penalizes high degree verticeswith-out any concern for fairness and allocates more resources toother vertices. It is also seen thatA3 can reach the sameutilization as the benchmark; although the triangulation re-sults in potential under-utilization, the restoration step canaccount for this and restore utilization (trading off some levelof fairness). However, as we will show later,A3 has perfectfairness (i.e. exact allocation as the benchmark) for mostof the topologies. In addition, for topologies where it re-quires restoration due to fill-in edges, it is within 0.1 dis-tance from the benchmark. We show the total throughput ofeach algorithm in Figure 10(b). We observe a similar trendas in Fig. 10(a) wheredegoutperforms the others. As ex-pected, the throughput curves follow those of utilization andthe same observations continue to hold. Fig. 11(a) plots theeffect of range on fairness. We observe that both heuristics

(prog and deg) consistently deviate more from the bench-mark as range increases. With increasing range, the numberof sub-channels that a femto is assigned decreases (resourcesare shared among more femtos). Recalling the fairness for-mula, a given difference in allocations (between the bench-mark and the heuristics) becomes more pronounced with lessnumber of sub-channels assigned by the benchmark (si). In-terestingly,A3 exhibits an improvement in fairness after aparticular range (while maintaining less than 0.15 distancefrom the benchmark). We find that the fill-in edge ratio is themain factor that affectsA3’s fairness (plotted in Fig. 11(b)).For small ranges, the graph contains some isolated vertices(very few cycles) andA3 does not introduce fill-in edges.As range increases, cycles start to form andA3 adds fill-inedges to maintain chordality. However for further ranges,increased connectivity turns in favor ofA3 since the cycleshappen rarely and fill-in edge ratio decreases again.

Effect of Number of Femtos: We fix a number of sub-channels (5) and a range (20 m.) and vary the number offemtocells. From Fig. 11(c) we see thatA3 consistentlyoutperforms the other heuristics in terms of fairness and iswithin 0.1 distance of the benchmark due to the rare needfor fill-in edges. The distance increases with the numberof femtos due to increased likelihood of cycles; the trendagain follows that of the fill-in edge ratio (plotted in Fig.11(d)). Fordegandprog, the distance also increases withthe number of femtos because of a reduced number of sub-channels per femto (si).

Effect of Number of Sub-channels: We simulate 30femtocells with 10 m. range. In Fig. 12(a), it is seen thatA3

exhibits a constant distance from the optimal. SinceA3’sperformance is mainly influenced by fill-in edge ratio, thenumber of sub-channels does not have a significant effect onA3’s fairness. It is also seen that the distance forprog anddegdecreases with increased number of sub-channels due tothe increase in number of sub-channels per femto (si). Thismakes the differences in allocations (between the heuristicand the benchmark) less pronounced as compared to whenthere are fewer sub-channels. Fig. 12(b) shows the effect of

13

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

0 10 20 30 40 50 60 70

Dis

tanc

e fro

m o

ptim

al

Range (meters)

A3

prog.deg.

(a) Fairness

0

0.02

0.04

0.06

0.08

0.1

0 10 20 30 40 50 60 70

Fill-

in E

dge

Rat

io

Range (meters)

(b) Fill-in Edge Ratio

0

0.1

0.2

0.3

0.4

0.5

10 15 20 25 30 35 40

Dis

tanc

e fro

m o

ptim

al

Number of Femtocells

A3

prog.deg.

(c) Fairness

0

0.02

0.04

0.06

0.08

0.1

10 15 20 25 30 35 40

Fill-

in E

dge

Rat

io

Number of Femtocells

(d) Fill-in Edge Ratio

Figure 11: Effect of Range (a-b) and Effect of Number of Femtos (c-d).

0

0.1

0.2

0.3

0.4

0.5

0 5 10 15 20 25 30

Dis

tanc

e fro

m o

ptim

al

Number of Sub-channels

A3

prog.deg.

(a) Fairness

0 1 2 3 4 5 6 7 8

0 5 10 15 20 25 30

Bur

sts

per F

emto

Number of Sub-channels

A3

prog.deg.

(b) Overhead

0

50

100

150

200

250

0.2 0.3 0.4 0.5 0.6 0.7 0.8

Tota

l Thr

ough

put (

Mbp

s)

Fraction of High Demand Femtos

no zoningglobal min

local min

(c) Local vs Global Min.

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

CD

F

Distance from optimal

A3

prog.deg.

(d) Fairness

Figure 12: Effect of Number of Sub-channels (a-b), Effect ofZoning (c) and Overall Fairness (d).

number of sub-channels on overhead. The overhead forA3

anddegis very close to 1 and does not change with numberof sub-channels. This shows that they can assign a singlecontiguous set of sub-channels to the femtocells. However,prog tends to have an increasing overhead. Sinceprog al-locates a fragmented set of sub-channels, the overhead in-creases with increasing number of sub-channels per femto.

Effect of Zoning: Each femtocell has two clients: onethat requires resource isolation (class 2) and one that requiresjust link adaptation (class 1). We simulate 40 femtocellswith range 10 m, 30 sub-channels and 30 symbols in theframe. We have three different types of reuse zone demands:i) high-demand femtos that randomly generate a reuse de-mand between 15 and 20 symbols ii) moderate demand fem-tos that generate a demand between 10 and 15 symbols andiii) low-demand femtos with generated demand between 5and 10 symbols. We experiment by varying the fraction ofthe high-demand femtocells. Fig. 12(c) shows the total thro-ughput achieved for each zoning strategy. It is seen that asthe fraction of high-demand femtos increases, the through-put for both zoning strategies increases. However, the gainof local min. over global min. is more with higher fractionof high-demand femtocells. This is a natural artifact of ver-tices converging to a higher local demand value as opposedto the global minimum demand which is the same on av-erage (generated by the low-demand vertices). The resultsreinforce FERMI’s benefits of decoupling the reuse zone de-mands in different contention regions of the network (pre-venting a single demand from propagating).

Overall Fairness: Finally, we present the CDF of thedistance from the optimal as a cumulative set of all previ-ously described experiments for the three algorithms consid-ered, in Fig. 12(d). We use the results with range 10 m. and20 m., as these represent a more realistic deployment (giventhe entire area is 100x100 meters). These provide an under-

standing of how fair a given algorithm is in practical deploy-ments with a large set of variables (# femtos, # sub-channels,zoning etc.). It is seen thatA3 is able to reach the exactbenchmark allocation in about 70% of the topologies whichis far superior to the performance of the other heuristics.deghas the worst performance and reaches the benchmark allo-cation in only 10% of the topologies.prog does better thandegbut still significantly underperforms compared toA3.

7. CONCLUSIONSIn this paper, we design and implement FERMI, one of

the first resource management systems for OFDMA femto-cell networks. Resource management in femtocells offers aset of unique practical challenges (posed by the requirementfor standards compatibility) that to the best of our knowledgehad not been addressed before. Our approach is based on aset of design decisions derived with extensive experimentson a four-cell WiMAX femtocell testbed. It is composed oftwo functional modules. The first module uses coarse-levelmeasurements to classify clients into two categories, thosethat need resource isolation and those that do not. The sec-ond module assigns OFDMA sub-channels to the differentfemto-cells in a near-optimal fashion. We implement ourapproach on our testbed to show its superiority compared toconventional approaches. We also perform simulations toshowcase its scalability and efficacy in larger scale settings.

8. REFERENCES[1] R. Van Nee and R. Prasad, “OFDM for Wireless Multimedia

Communications,”Artech House, 2000.[2] D. Lopez-Perez, G. Roche, A. Valcarce, A. Juttner, and J.Zhang,

“Interference Avoidance and Dynamic Frequency Planning forWiMAX Femtocells Networks,” inProc. of IEEE ICCS, 2008.

[3] 3GPP, “Technical specification group radio access networks; 3Ghome NodeB study item technical report (release 8),”TR 25.820V1.0.0 (2007-11), Nov 2007.

14

[4] S. Kittipiyakul and T. Javidi, “Subcarrier Allocation in OFDMASystems: Beyond water-filling,” inProc. of Signals, Systems, andComputers, 2004.

[5] T. Quek, Z. Lei, and S. Sun, “Adaptive Interference Coordination inMulti-cell OFDMA Systems,” inIEEE PIMRC, 2009.

[6] J. Yun and S. Kang, “CTRL: A Self-Organizing FemtocellManagement Architecture for Co-Channel Deployment,” inACMMOBICOM, Sept 2010.

[7] K. Sundaresan and S. Rangarajan, “Efficient Resource Managementin OFDMA Femto Cells,” inProc. of ACM MOBIHOC, May 2009.

[8] A. Mishra, S. Banerjee, and W. Arbaugh, “Weighted coloring basedchannel assignment for WLANs,” inACM SIGMOBILE MobileComputing and Communications Review, July 2005, vol. 9.

[9] T. Moscibroda, R. Chandra, Y. Wu, S. Sengupta, P. Bahl, andY. Yuan, “Load-aware spectrum distribution in wireless lans,” inIEEE ICNP, 2008.

[10] A. Mishra, V. Brik, S. Banerjee, A. Srinivasan, and W. Arbaugh,“Client-driven channel management for wireless lans,” inIEEEInfocom, 2006.

[11] L. Yang, W. Hou, L. Cao, B. Zhao, and H. Zheng, “SupportingDemanding Wireless Applications with Frequency-agile Radios,” inUSENIX NSDI, 2010.

[12] K. Tan, J. Fang, Y. Zhang, S. Chen, L. Shi, J. Zhang, and Y.Zhang,“Fine-grained Channel Access in Wireless LAN,” inACMSIGCOMM, Aug. 2010.

[13] IEEE 802.16e 2005 Part 16, “Air Interface for Fixed and MobileBroadband Wireless Access Systems ,”IEEE 802.16e standard.

[14] WMF-T33-118-R016v01, “Femtocells Core Specification,” .[15] PicoChip, “http://www.picochip.com,” .[16] Accton, “http://www.accton.com,” .[17] J. R. S. Blair and B. W. Peyton, “An introduction to chordal graphs

and clique trees,” inhttp://www.ornl.gov/info/reports/1992/3445603686740.pdf.

[18] A. Berry, J. R. S. Blair, P. Heggernes, and B. W. Peyton, “Maximumcardinality search for computing minimal triangulations of graphs,”in Journal Algorithmica, May 2004, vol. 39.

[19] A. Berry, P. Heggernes, and Y. Villanger, “A vertex incrementalapproch for dynamically maintaining chordal graphs,” inAlgorithmsand Computation, 14th Int. Symp. (ISAAC), December 2003.

[20] R. Jain, A. Durresi, and G. Babic, “Throughput fairnessindex: Anexplanation,” inATM Forum Document Number: ATM Forum /990045, February 1999.

[21] S. Yeh and S. Talwar, “Multi-tier simulation methodology ieeec802.16ppc-10/0039r1,” 2010.

15