two examples of submodularity in wireless communications

Two Examples ofSubmodularity in WirelessCommunications

Ni Ding

13 June 2017

www.data61.csiro.au

Outline

IntroductionSubmodular FunctionTarski Fixed Point TheoremSubmodular (Set) Function Minimization

Adaptive Modulation in Network-coded Two-way Relay ChannelSystemTwo-player Game ModelPure Strategy Nash EquilibriumCournot Tatonnement

Communication for OmniscienceSystemMinimum Sum-ratePrincipal Sequence of PartitionsModified Decomposition AlgorithmExtensions: Secret Capacity, Clustering and Data Compression

Conclusion

2 | Two Examples of Submodularity in Wireless Communications | Ni Ding

Submodularity

• a property of functions defined on lattice [1]1 [2]2.

I lattice: fundamental algebraic structure on partial order.

• applications: economics, machine learning, operations research.

Study on Machine Learning in [3]3:

Submodularity imposes a structure which allows much strongermathematical results than we would be able to achieve without it.

• submodularity on

I vector lattice: discrete convexity and comparative staticsI set lattice: combinatorial optimization, e.g., in graph theory

1Topkis 20012Murota 20053Vondrak 2007


Lattice

Poset

For a set L and a binary order , (L,) is a poset (partially ordered set)if either a b or a 6 b,∀a, b ∈ L.

examples: (RN ,≤), (1, . . . , 4N ,≤), (2V ,⊆) and (1, 1, 2, 3,⊆)

Lattice

A poset (L,) is a lattice with notation (L,∨,∧) ifa ∨ b = supa, b ∈ L and a ∧ b = infa, b ∈ L,∀a, b ∈ L with sup andinf w.r.t.

• maximum∨L = supL and minimum

∧L = inf L exist;

examples: (RN ,∨,∧), (1, . . . , 4N ,∨,∧) with r ∨ r′ = (maxri , r ′i : i ∈ 1, . . . ,N)and r ∧ r′ = (minri , r ′i : i ∈ 1, . . . ,N); (2V ,∪,∩) and (1, 1, 2,∪,∩)


Submodular Function

Submodularity

f : (L,∨,∧) 7→ R is submodular if

f (a) + f (b) ≥ f (a ∨ b) + f (a ∧ b), ∀a, b ∈ L.

f is supermodular if −f is submodular.


Tarski Fixed Point TheoremN-player game model Ω = N , Ai , ci (a)i∈N witha ∈ A = ×i∈NAi ⊆ RN :

• pure strategy Nash equilibrium (PSNE): best response functionψ : A 7→ A with ai ∈ Ai , a−i ∈ ×i ′∈N\iAi ′ and

ψi (a−i ) ∈ arg minci (ai , a−i ) : ai ∈ Ai

, ∀i ∈ N .

a∗ is an PSNE if a∗ = ψ(a∗), i.e., a∗ is a fixed point of ψ.

• question: PSNE exists for discrete A? supermodular game withstrategic complements: ψ : (A,∨,∧) 7→ (A,∨,∧) is non-decreasingif ci is submodolar ∀i .

Tarski Fixed Point Theorem [4]4

The fixed points of non-decreasing ψ : (A,∨,∧) 7→ (A,∨,∧) form a(nonempty) lattice.

4Tarski et. al 1955


Submodular (Set) FunctionMinimization

For f : (2V ,∪,∩) 7→ R, consider

minf (X ) : X ⊆ V (1)

combinatorial optimization: NP-complete or NP-hard in general

SFM (submodular function minimization) algorithm

If f is submodular, i.e.,

f (X ) + f (Y ) ≥ f (X ∪ Y ) + f (X ∩ Y ), ∀X ,Y ⊆ V ,

(1) can be solved in polynomial time and the minimizers form a lattice:⋃argminf (X ) : X ⊆ V and

⋂argminf (X ) : X ⊆ V exist [5]5.

5Fujishige 2005


Adaptive Modulation inNetwork-coded Two-way RelayChannel

network-coded two-way relay channel (NC-TWRC): two userscommunicate via a center node, relay ‘R’.

user 1wireless

Rwireless

user 2

physical-layer network coding (PNC): messages x1 and x2 transmittedsimultaneously in phase I, the superposition z broadcast in phase II.

user 1x1

Rx2

user 2

phase I: multiple access (MAC)

user 1z

Rz

user 2

phase II: amplify and forward (AF)


Adaptive Modulation inNC-TWRC

assumption: m-quadrature amplitude modulation (m-QAM) adopted byeach user:

• constellation size mi = 2ai of user i with ai ∈ 0, 1, . . . , thenumber of bits/symbol, determined by user i

Strategic Complements

increasing best response:

• spectral efficiency: one tends to transmit while the other does so

• equal share of the channel: one increases ai while the other−i = 1, 2 \ i does so

proposal: two-player game model parameterized by user-to-user channelsignal-to-noise ratios (SNRs)


Two-player Game ModelΩγ = N , Γ, Ai , ci (γi , a)i∈N :• N = 1, 2;• γ = (γ1, γ2) ∈ Γ = Γ1 × Γ2 with γi ∈ Γi being SNR of user i-to-user−i channel determined by PNC scheme;

• a = (a1, a2) ∈ A = A1 ×A2 = 0, 1, . . . ,Am2 with Am being themaximum number of bits/symbol

• cost function: ci : Γi ×A 7→ R+

ci (γi , a) = ce(γi , ai ) + cr (a)

with the cost associated with transmission error rate

ce(γi , ai ) =− ln(5Pb)(2ai − 1)

1.5γi

and the cost associated with spectral efficiency and fairness

cr (a) =a−i + 1

ai + 1


Pure Strategy Nash Equilibrium(PSNE)

Submodularity

ci : (A,∨,∧) 7→ R+ is submodular, i.e.,

ci (γi , a) + ci (γi , a′) ≥ ci (γi , a ∨ a′) + ci (γi , a ∧ a′),

for all a, a′ ∈ (A,∨,∧), i ∈ N and γ ∈ Γ.

Tarski Fixed Point Theorem =⇒ Existence of PSNE

Pure strategy θ : Γ 7→ A, where θ(γ) = (θ1(γ), θ2(γ)) with θi (γ) ∈ Ai

being the pure strategy of user i when SNRs are γ = (γ1, γ2):

• PSNE θ∗ exists

• The largest PSNE θ∗

and the smallest PSNEs θ∗ exist


Cournot Tatonnementdetermine extremal PSNEs: Cournot tatonnement [6]6

• Let ψ : Γ×A 7→ A and ψ : Γ×A 7→ A be the maximal andminimal best response functions, respectively, with

ψi (γ, a−i ) =∨

arg minai∈Ai

ci (γi , ai , a−i )

ψi(γ, a−i ) =

∧arg min

ai∈Ai

ci (γi , ai , a−i )

• recursions with θ(0)

(γ) =∨A and θ(0)(γ) =

∧A:

θ(γ) := ψ(γ,θ(γ))

θ(γ) := ψ(γ,θ(γ))

Convergence

θ(k)(γ) and θ(k)(γ) converge monotonically downward and upward

to θ∗(γ) and θ∗(γ), respectively, for all γ.

6Vives 1990


Experiment Iperformance of extremal PSNEs:A = 0, 1, . . . , 92 and simulation lasts for 104 symbol durations:

extremal PSNEs θ∗

and θ∗ are compared to the single-agent adaptivemodulation (Single-AM) and 2-QAM scheme.

−6 −4 −2 0 2 4 610−5

10−4

10−3

10−2

10−1

100

γ(dB)

biterrorrate

(BER)

θ∗

θ∗

Single-AM2-QAM

(a) bit error rate (BER)

−6 −4 −2 0 2 4 60

2

4

6

8

10

γ(dB)

bits

per

symbol

duration

θ∗

θ∗

Single-AM2-QAM

(b) spectral efficiency


Experiment IIexample of Cournot tatonnement

A = 0, 1, . . . , 92 and the sequences θ(k)

1 (γ) and θ(k)1 (γ) generated

by Cournot tatonnement for certain γ

1 2 3 40

2

4

6

8

iteration index k

strategy

ofschedu

ler1 θ

(k)1 (γ)

θ∗1(γ)

θ(k)1 (γ)θ∗1(γ)


Communication for Omniscience

indices of users: a finite ground set V with |V | > 1discrete correlated random source: ZV = (Zi : i ∈ V )

• user i observes an i.i.d. n-sequence Zni of Zi in private

communication for omniscience (CO) [7]7:

• users exchange Zi s directly over noiseless broadcast channels

• goal: attain omniscience, the state that each user recovers ZnV

Minimum Sum-rate Problem

how to attain omniscience with RCO(V ), the minimum total number oftransmissions: value of RCO(V ) and an optimal rate vectorr∗V = (r∗i : i ∈ V )

7Csiszar et. al 2004: CO formulated based on the study on secret capacity


Example: Coded CooperativeData Exchange (CCDE)

client 1

Z1 = [Wa,Wb,Wc ,Wd ,We ]ᵀ

client 2

Z2 = [Wa,Wb,Wf ]ᵀ

client 3

Z3 = [Wc ,Wd ,Wf ]ᵀ

3-mobile clients in V = 1, 2, 3; Zi : partial observation of a packet setwith Wj denoting a packet

Solutions to Minimum Sum-rate Problem

RCO(V ) = 72 and r∗V = (r∗1 , r

∗2 , r∗3 ) = ( 5

2 ,12 ,

12 ): by packet-splitting

Wj =⇒W(1)j ,W

(2)j ; transmit (r1, r2, r3) = (5, 1, 1) with ri denote the

number of linear combinations of packet chunks W(k)j .


Omniscience-achievabilityFor X ⊆ V : r(X ) =

∑i∈X ri for rV = (ri : i ∈ V )

H(X ): the amount of randomness in ZX measured by Shannon entropy

Omniscience-achievability [7]8

An omniscience-achievable rV satisfies the Slepian-Wolf (SW) constraint:r(X ) ≥ H(X |V \ X ) = H(V )− H(V \ X ),∀X ( V .

achievable rate vector set:

R(V ) = rV ∈ R|V | : r(X ) ≥ H(X |V \ X ),∀X ( V

minimum sum-rate:

RCO(V ) = minr(V ) : rV ∈ R(V )

constant sum-rate set: Rα(V ) = rV ∈ R(V ) : r(V ) = αoptimal rate vector set: RRCO(V )(V )

8Csiszar et. al 2004


Nonemptiness of BasePolyhedron

For α ∈ R+, let

fα(X ) =

H(X |V \ X ) X ( V

α X = V.

polyhedron: P(fα,≥) = rV ∈ R|V | : r(X ) ≥ fα(X ),∀X ⊆ V base polyhedron: B(fα,≥) = rV ∈ P(fα,≥) : r(V ) = fα(V ) = α

• B(fα,≥) = Rα(V ) 6= ∅ ⇐⇒ ∃ achievable rV with r(V ) = α

dual set function: f #α (X ) = fα(V )− fα(V \ X )

• B(fα,≥) = B(f #α ,≤) [5]9;

why consider B(f #α ,≤)? f #

α is intersecting submodular, i.e.,

f #α (X ) + f #

α (Y ) ≥ f #α (X ∪ Y ) + f #

α (X ∩ Y ), ∀X ,Y : X ∩ Y 6= ∅.9Fujishige 2005


Minimum Sum-rateΠ(V ): the set of all partitions of V and Π′(V ) = Π(V ) \ V achievability of α: B(f #

α ,≤) 6= ∅ iff α = minP∈Π(V )

∑C∈P f #

α (C ) [5]10

Minimum Sum-rate

RCO(V ) = maxP∈Π′(V ) φ(P) with the finest maximizer P∗. Here,

φ(P) =∑C∈P

H(V \ C |C )

|P| − 1.

interpretation: ∀C ∈ P, the cut C ,V \ C imposes SW constraintr(V \ C ) ≥ H(V \ C |C ) so that∑

C∈P

r(V \ C ) = (|P| − 1)r(V ) ≥∑C∈P

H(V \ C |C )

A multi-way cut P ∈ Π′(V ) imposes r(V ) ≥ φ(P).

10Fujishige 2005: minP∈Π(V )

∑C∈P f #

α (C) is called the Dilworth truncation of f #α .


Principal Sequence of Partitions

Principal Sequence of Partitions (PSP) [8]11:

minP∈Π(V )

∑C∈P f

#α (C ) is a piecewise linear increasing curve in α

that is fully characterized by p ≤ |V | − 1 critical points

H(V ) = α0 > α1 > α2 > . . . > αp ≥ 0.

Let Pj be the finest minimizer of minP∈Π(V )

∑C∈P f

#α (C ).

P0 P1 P2 . . . Pp

where P P ′ denotes P ′ is strictly finer than P.

The first critical point determines the solutions to the minimumsum-rate problem: RCO(V ) = α1,P∗ = P1.

11Nagano et. al 2010


Example of PSP

client 1

Z1 = [Wa,Wb,Wc ,Wd ,We ]ᵀ

client 2

Z2 = [Wa,Wb,Wf ]ᵀ

client 3

Z3 = [Wc ,Wd ,Wf ]ᵀ

0 1 2 3 4 5 6

−5

0

5

10

α1 = 72,P1 = 1, 2, 3

α0 = 0,P0 = 1, 2, 3

α

minP∈Π

(V)

∑ C∈P

f# α(C

)

minP∈Π(V )

∑C∈P f

#α (C )

α

PSP results:α0 > α1 and P0 P1:

• No omniscience-achievablerV if α < α1, because α 6=minP∈Π(V )

∑C∈P f #

α (C );

• RCO(V ) = α1 andP∗ = 1, 2, 3 = P1


Properties of φ(P) in PSP

αj and Pj :

• If j = 1, αj = φ(Pj);

• When j > 1, let α = φ(Pj) and Pj′ be the finest minimizer ofminP∈Π(V )

∑C∈P f #

α (C ). Then,

αj < α < α1

j ′ < j =⇒ Pj′ Pj

Suggestion: A Recursive Algorithm

• iteratively updates α and P, the estimation of α1 = RCO(V ) andP∗ = P1;

• terminate when α = φ(P).


Modified DecompositionAlgorithm

recursion in modified decomposition algorithm (MDA):

α := φ(P)

P(n) is the finest minimizer of minP∈Π(V )

∑C∈P f #

α(n) (C ) and

P(0) = i : i ∈ V

Optimality of the MDA algorithm

α(n) and P(n) converge monotonically towards α1 = RCO(V ) and

P1 = P∗, respectively. Also returns rV ∈ B(f #RCO(V ),≤) = RRCO(V )(V )

• minP∈Π(V )

∑C∈P f #

α(n) (C ) reduces to⋂argminf #

α(n) (X )− r(X ) : i ∈ X ⊆ V ,∀i ∈ V , SFM due to the

intersecting submodularity of f #α(n)

• complexity: O(|V |2 · SFM(|V |))12

12SFM(|V |): the complexity of minimizing submodular function f : 2V 7→ R.


ExperimentV = 1, . . . , 5: Wm is an independent uniformly distributed random bit:

Z1 = (Wb,Wc ,Wd ,Wh,Wi ),

Z2 = (We ,Wf ,Wh,Wi ),

Z3 = (Wb,Wc ,We ,Wj ),

Z4 = (Wa,Wb,Wc ,Wd ,Wf ,Wg ,Wi ,Wj ),

Z5 = (Wa,Wb,Wc ,Wf ,Wi ,Wj ),

0 1 2 3 4 5 6 7 8 9 10

−20

−10

0

10

P3 = 1, 2, 3, 4, 5

P2 = 4, 5, 1, 2, 3

P1 = 1, 4, 5, 2, 3 = P∗P0 = 1, 2, 3, 4, 5

α

minP∈Π

(V)

∑ C∈P

f# α(C

)

minP∈Π(V )

∑C∈P f

#α (C )

α

(c) PSP

0 1 2 3

5.8

6

6.2

6.4

6.6

iteration index

α

α(n)RCO(V )

(d) α(n) by MDA algorithm


Extensions of CO: SecretCapacity

secret capacity CS(V ): the maximum rate at which the secret key can begenerated by the users in V with results in [7]13:

• dual relationship: RCO(V ) = H(V )− CS(V )

• mutual dependence upper bound on CS(V ):

CS(V ) ≤ I (V ) = minP∈Π′(V )

∑C∈P H(C )− H(V )

|P| − 1︸︷︷︸mutual dependence in ZV

tightness [9]14: CS(V ) = I (V ) = H(V )− RCO(V )question: how to achieve CS(V )? with interactive communication rater(V ) = RCO(V )? silly! CS(V ) can be attained with r(V ) ≤ RCO(V ) [7]

13Csiszar et. al 200414Chan et. al 2015


Extensions of CO: Clustering

Inspired by the name ‘mutual dependence’

I (V ) = minP∈Π′(V )

∑C∈P H(C )− H(V )

|P| − 1

is proposed in [9]15 as a generalization of Shannon’s mutual informationto multivariate case: I (V ) = H(1) + H(2)− H(1, 2) whenV = 1, 2.

• realization: I (V ) is the similarity measure of more than two rvs.

I limitation in existing clustering algorithms: pairwisesimilarity/dissimilarity measure

I agglomerative clustering result given by PSP determined inO(|V |2 · SFM(|V |)) time

question: can PSP clustering frame work provide more objective overviewof the dataset?

15Chan et. al 2015


Extensions of CO (Digiscape):Source Coding with SideInformation

1

Zn1

2

Zn2

. . . . . . |V |

Zn|V |

T

sensor nodes i ∈ V = 1, . . . , |V | reveal all information to sink T .

• for lossless data compression/aggregation, SW constraints:

r(X ) ≥ H(X |V \ X ),∀X ( V , r(V ) = H(V ) =⇒ RH(V )(V )

• an extreme, one of the unfairest, rV ∈ RH(V )(V ) can be determinedin O(|V |) time

• question: how to find a fair rate allocation in RH(V )(V ) efficiently?27 | Two Examples of Submodularity in Wireless Communications | Ni Ding

Conclusion

two examples of submodularity in wireless communications:

• vector lattice: the existence of PSNEs in a game modeledadaptive modulation problem in NC-TWRC

• set lattice: polynomial time algorithm for solving CO problem

future:

• vector lattice: more applications of discrete convexity, e.g.,the energy-delay trade-off in data aggregation tree inDigiscape, and monotone comparative statics

• set lattice: improving efficiency for determining PSPI less call of SFM algorithmI improving complexity SFM(|V |): SFM belongs to worst

polynomial algorithm category, e.g., |V |5 to |V |8


Bibliography I

D. M. Topkis, Supermodularity and complementarity. Princeton: PrincetonUniversity Press, 2001.

K. Murota, “Note on multimodularity and l-convexity,” Math. Oper. Res.,vol. 30, no. 3, pp. 658–661, Aug. 2005.

J. Vondrak, “Submodularity in combinatorial optimization,” Ph.D. dissertation,Dept. Appl. Math., Charles Univ., Prague, 2007.

A. Tarski et al., “A lattice-theoretical fixpoint theorem and its applications,”Pacific J. Math., vol. 5, no. 2, pp. 285–309, 1955.

S. Fujishige, Submodular functions and optimization, 2nd ed. Amsterdam, TheNetherlands: Elsevier, 2005.

X. Vives, “Nash equilibrium with strategic complementarities,” J. Math. Econ.,vol. 19, no. 3, pp. 305 – 321, 1990.

I. Csiszar and P. Narayan, “Secrecy capacities for multiple terminals,” IEEETrans. Inf. Theory, vol. 50, no. 12, pp. 3047–3061, Dec. 2004.


Bibliography II

K. Nagano, Y. Kawahara, and S. Iwata, “Minimum average cost clustering,” inProc. Advances in Neural Inf. Process. Syst., Vancouver, Candada, 2010, pp.1759–1767.

C. Chan, A. Al-Bashabsheh, J. Ebrahimi, T. Kaced, and T. Liu, “Multivariatemutual information inspired by secret-key agreement,” Proc. IEEE, vol. 103,no. 10, pp. 1883–1913, Oct. 2015.


two examples of submodularity in wireless communications

Documents