lattices which are good for (almost) everything

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 51, NO. 10, OCTOBER 2005 3401

Lattices Which Are Good for (Almost) EverythingUri Erez, Simon Litsyn, Senior Member, IEEE, and Ram Zamir, Senior Member, IEEE

Abstract—We define an ensemble of lattices, and show that forasymptotically high dimension most of its members are simul-taneously good as sphere packings, sphere coverings, additivewhite Gaussian noise (AWGN) channel codes and mean-squarederror (MSE) quantization codes. These lattices are generated byapplying Construction A to a random linear code over a primefield of growing size, i.e., by “lifting” the code to .

Index Terms—Coding for unconstrained additive white Gauss-ian noise (AWGN) channel, lattice codes, Loeliger ensemble, mean-squared error (MSE) quantization, Minkowski bound, Poltyrev ex-ponent, sphere covering, sphere packing.

I. INTRODUCTION

I N this work, we consider the problem of existence of latticesin Euclidean space that are simultaneously asymptotically

good in several coding-related contexts. We begin with a de-scription of the binary counterpart which motivated this study.

In -dimensional binary Hamming space, the problems ofsphere packing, sphere covering, channel coding, and quanti-zation are well known. The first two are of a combinatorial na-ture. In the packing problem, we are interested in packing thegreatest possible number of nonintersecting Hamming spheresof a given radius. This is equivalent to maximizing the numberof codewords for a given minimum distance of the code. For acomprehensive survey see e.g., [28], [31]. The covering problemasks for a minimum size collection of spheres of a given radius,such that every point of space belongs to at least one sphere.This corresponds to covering codes, see, e.g., [6]. The othertwo problems are of a probabilistic or information-theoreticalnature. The channel coding problem for the binary-symmetricchannel (BSC) asks for the best arrangement of points in Ham-ming space such that for a given number of codewords, theprobability of error of the maximum-likelihood decoder is min-imized. Of particular importance is the question of the greatestpossible rate of the code that still enables this error probability tovanish asymptotically as the dimension (length of the code)goes to infinity. This leads to the concept of channel capacityand error exponent. For a survey see, e.g., [17]. The quantiza-tion problem seeks to minimize the required number of code-words such that the average distance of the points in Hamming

Manuscript received June 2, 2003; revised June 7, 2005. The work of S. Litsynwas supported in part by the ISF and the Binational Science Foundation underGrant BSF1999-099. The work of R. Zamir was supported in part by the IsraelAcademy of Science Research Fund under Grant ISF 65-01. The material in thispaper was presented in part at the IEEE Information Theory Workshop, Paris,France, April 2003.

The authors are with the Department of Electrical Engineering–Systems, Tel-Aviv University, Ramat-Aviv 69978, Tel-Aviv, Israel (e-mail: [email protected];[email protected]; [email protected]).

Communicated by R. Urbanke, Associate Editor for Coding Techniques.Digital Object Identifier 10.1109/TIT.2005.855591

space from their nearest codeword is not greater than a specifiedtarget distortion, see, e.g., [2]. In what follows, we deal with theasymptotic case of high dimension . The bounds on the param-eters are usually exponential in the dimension. For our purposesall bounds with the same exponent are considered equivalent.

All these problems are typically treated in information theoryby random coding arguments. Basically, all that is needed is todraw at random -tuples according to a uniform distribution.The best known bounds, asymptotic in , are obtained in thismanner. For the packing problem it is the Gilbert bound [19],for the covering problem it is the Goblick–Cohen bound [20],[7]. For the channel coding problem the bound is given by theShannon capacity [36] and the Elias–Gallager exponent [12].For the quantization problem the bound is the Shannon rate-distortion bound [2].

The obtained codes, however, lack structure. It turns out thateven if we impose a requirement that the code be linear, it is stillpossible to achieve the same asymptotic bounds. Indeed, for thepacking problem this was shown by Varshamov [40], for thecovering problem (from which the claim for quantization alsofollows) by Cohen [7], and for the channel coding problem byGallager and Dobrushin [16], [11].

Moreover, the mentioned bounds hold true in a strongersense, namely, for almost all linear codes. This was shownfor the packing problem by Pierce [30], for the coveringproblem (from which the claim for quantization also follows)by Blinovskii [3] (for the nonlinear case see [10]), and for thechannel coding problem by Gallager and Dobrushin [16], [11].In fact, as a code for the BSC, the random linear ensemble isin a sense better than the totally random ensemble for channelcoding. A typical code of the linear ensemble achieves theexpurgated bound. This is not the case for a typical code ofthe totally random ensemble where expurgation of a relativelysmall number of codewords is necessary, see, e.g., [1], [4].

A random linear ensemble is produced by a randomly chosengenerator matrix with independent equiprobable binary

entries. Since the same ensemble of linear codes generates a so-lution to all four problems, it follows (by a union bound argu-ment) that there exist binary linear codes that are good in all foursenses simultaneously.

In this paper, we prove a similar result for analogous prob-lems in Euclidean space, namely, that there exist lattices that aresimultaneously good for sphere packing and covering, channelcoding and quantization. While the sphere packing/coveringproblems carry over in a straightforward manner from theHamming to the Euclidean space, the corresponding extensionsof the transmission/quantization problems require a word ofcaution. The standard scenarios of channel/source coding inEuclidean space are based on bounded codebooks [36]. Inchannel coding this is due to a transmitter power constraint,

0018-9448/$20.00 © 2005 IEEE

3402 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 51, NO. 10, OCTOBER 2005

while in source coding this is the result of a finite rate con-straint. Nevertheless, for our discussion of lattice codes it ismore convenient to consider unbounded codebooks. Poltyrev’snotion of “unconstrained channels” provides a meaningfuldefinition for the capacity of such codebooks in the channelcoding context [32] (see also [14]), while “high resolution”quantization theory provides a corresponding notion of codingrate in the quantization framework [18], [45], [43]. Similarly tothe binary Hamming case, we address here only asymptotic, inthe space dimension, bounds. The bounds are again exponentialin the dimension and are the best known bounds also for non-lattice constellations (though subexponential terms may vary).We show that asymptotically there exist lattices that achievethese bounds simultaneously.

For the Euclidean packing problem the best known boundis the Minkowski bound [29], for the covering problem itis the Rogers bound [33], [34], for the channel coding andmean-squared error (MSE) quantization problems the boundsare due to Poltyrev [32], [42] (which are related to Shannon’sbounds for the power-constrained channel and rate distortionof Gaussian sources, see [37], [36]). We say that a sequenceof lattices is good for packing if it asymptotically achieves theMinkowski bound. A similar terminology is used when theRogers and Poltyrev bounds are used.

It is well known that a lattice that is good with respect to anyof the four criteria has a Voronoi region (see definition below)that in some sense is “close” to spherical. However, the corre-sponding notions of closeness differ for the various problems.Furthermore, for a fixed dimension, an optimal lattice in onesense need not necessarily be optimal in another. For example,in three-dimensional Euclidean space , the optimal lattice forpacking is the face-centered cubic (FCC) lattice while the op-timal lattice for covering is the body-centered cubic (BCC) lat-tice, see, e.g., [8], [15]. Nonetheless, in this work we prove thatasymptotically (in dimension) a lattice may be optimal in allfour senses.

In order to prove our result we would like to have an ensembleof lattices that in a sense induces a uniform distribution of pointsin the Euclidean space. Unlike the binary case though, it is notself-evident what a natural ensemble of lattices should be. Infact, there are various alternative approaches that could be taken.Also, one must address how to keep a constant “rate” of thelattice as a function of the dimension .

The approach we take is based on Construction A as pro-posed by Loeliger [27]. The unit cube is discretized into atimes finer grid, and then a random linear code is drawnover the resulting group of size . For a given dimension , thedensity (or “rate”) of the lattice is thus determined by two pa-rameters and rather than one. We find conditions on these“free” parameters such that the ensemble is good, for each ofthe four problems. These conditions vary, however, for the dif-ferent problems. The question is therefore whether they are notin contradiction. In effect, the essence of the proof that there ex-ists an ensemble simultaneously good for all four problems, liesin showing that there is a nonvoid intersection in the conditionsimposed by the different problems.

Loeliger has previously taken this approach of defining anensemble of lattices in [27]. Using this technique, he proved

the existence of lattices that achieve both the Minkowski boundand the Poltyrev capacity of the additive white Gaussian noise(AWGN) channel. We extend here the analysis to error expo-nents as well as treat also the covering and quantization prob-lems.

In [5], Butler proved that there exist lattices that are simul-taneously good for packing and covering. His proof utilizes thetechniques of Rogers [35]. In the current work, we re-derive thisclaim as well as extend it to the channel coding and quantizationsettings. Moreover, our proof is significantly simpler.

The paper is organized as follows. Section II introduces thenecessary notation and defines the four problems of interest. InSection III, the random ensemble of lattices is introduced. Sec-tions IV–VII each contain the proof that the defined ensembleis good for one of the four problems; simultaneous goodness isproved in Section VIII. The work is summarized in Section IX.Technical proofs are relegated to the Appendices.

II. LATTICE PROPERTIES

In order to give precise definitions of the problems we use thefollowing notation.

• GRID: cubic grid (lattice) of step size .• : -dimensional lattice nested in GRID, i.e., GRID.• : Euclidean norm, i.e., .1

• : set sum. If and are sets in then

• : Fundamental Voronoi region of , i.e,

(1)

and

(2)

• : volume of a closed set in .• where

and means rounding to the nearest nongreaterinteger.

• CUBE• CUBE, where is any set in .• GRID GRID CUBE.• : -dimensional closed sphere of radius centered at

the origin, unit sphere.• : volume of sphere of radius , i.e.,

• : rounding to the nearest greater prime.• : lattice constellation. CUBE generates by

Construction A.• : size of lattice constellation.• : a point in , .

A. Packing Problem

Consider a lattice with Voronoi region . For a given radius, the set is a packing in Euclidean space if for all lattice

points we have

1Extension to other norms is considered in the discussion.

EREZ et al.: LATTICES WHICH ARE GOOD FOR (ALMOST) EVERYTHING 3403

Fig. 1. Geometric picture.

That is, the spheres do not intersect. Define the packing radiusof the lattice by

is a packing (3)

Denote by the “effective radius” of the Voronoi region,meaning the radius of a sphere having the same volume, so that

is defined by

(4)

or equivalently

(5)

where (the volume of a unit sphere) is given by (see, e.g.,[8])

(6)

Fig. 1 gives the geometric picture of and with respectto the Voronoi region, as well as the other radii to be definedlater. Define the packing efficiency of a lattice by

(7)

We note that the packing efficiency is by definition notgreater than one. We wish to be as large as possible.The density of the packing, i.e., the proportion of space takenup by the spheres is .

Let denote an -dimensional lattice. Define the optimalasymptotic packing efficiency by

The best known lower bound for is given by theMinkowski–Hlawka theorem [35]

(8)

Since from volume arguments, the bound implies aradius half as large as we could hope for. In fact, it is known

that no efficient packing exists, in the sense that is strictlyless than one. The best known upper bound was found by Ka-batiansky and Levenshtein [23] and is given by (see [8])

(9)

Therefore, it is not known whether the Minkowski bound istight. We say that a sequence of lattices is asymptotically goodfor packing if it achieves the Minkowski bound.

We note that for small dimension, the optimal packing effi-ciency is a rather irregular (nonmonotonic) function of . In-deed, for we have . Sphere packing isefficient in the one-dimensional case but cannot be efficient forany larger dimension. A similar irregular behavior at small di-mensions is also true in the covering problem but as we shallsee, as the optimal covering is efficient.

B. Covering Problem

The associated notions for the covering problem are definedsimilarly to their packing counterparts. The set is a cov-ering of Euclidean space if

That is, each point in space is covered by at least one sphere.Define the covering radius of the lattice by

is a covering

Define the covering efficiency of a lattice by

We note that the covering efficiency is by definitionnot less than one. We wish to be as small as possible.The density of the covering, i.e., the average number of spherescovering a point is .

Define the optimal asymptotic covering efficiency by

It is a result of Rogers [34] that satisfies

This means that covering (in contrast to packing) may be asymp-totically efficient, i.e., every point in space can be covered byat most a subexponential number of spheres. See standard text-books on packing and covering such as Rogers [35] and Conwayand Sloane [8].

It will be shown later that simultaneously good packings andcoverings exist, as previously shown by Butler [5]. We say thata sequence of lattices is good for covering if it satisfies Rogers’bound, i.e., if it allows for efficient covering.

C. MSE Quantization

In MSE quantization, we associate with a nearest neighborquantizer such that

(10)

where ties are broken in a systematic manner. Equivalently

if (11)


The second moment, , of is defined as the secondmoment per dimension of a uniform distribution over

(12)

A figure of merit of a lattice quantizer with respect to the MSEdistortion measure is the normalized second moment

(13)

The minimum possible value of over all lattices in isdenoted by . The normalized second moment of a sphere, de-noted by , approaches as the dimension goes to infinity.The isoperimetric inequality implies that forall . We also have .

The operational significance of this figure of merit comesfrom a result due to Gersho [18] (see also [41], [21], [24]) thatstates that under quite general conditions for high-resolution lat-tice quantization of a source the distortion satisfies

(14)

where denotes the distortion, is the quantization rate innats, and is the dimension-normalized differ-ential entropy of the source (also in nats).

From rate-distortion theory we have that the ultimate perfor-mance of a random code, defined by the Shannon lower bound,would yield the same formula for the distortion with re-placed by , see [26]. Thus, we refer to a sequence of latticesas good for MSE quantization if tends to . A resultof Zamir, Feder, and Poltyrev [42] states that

(15)

i.e., that there exist good lattice quantizers.We note that unlike in the AWGN channel coding problem,

here it is not interesting to investigate the rate-distortion errorexponent for rates . As a consequence of the factthat we have a good covering lattice, the probability that thereis any point with distortion exceeding ( being arbi-trarily small) may be made zero for sufficiently large dimension,corresponding to an infinite exponent, in contrast to the situa-tion we encounter later for channel coding. We will see belowthat in the channel coding problem the error exponent is finite.Thus, we observe that the quantization problem is fundamen-tally different from the “dual” AWGN channel coding problemdiscussed in the followng subsection when “second-order ef-fects” (like error exponents) are considered.

D. Coding for the Unconstrained AWGN

Of the four applications of lattices considered in this workthe precise notion of good lattices for coding over the AWGNchannel was the most recently introduced. It appears in a 1994paper by Poltyrev [32].

The AWGN channel model is given by the input/output rela-tion

(16)

where is independent and identically distributed (i.i.d.)Gaussian noise of variance . We denote by an i.i.d. vector

of length of noise random variables. We define the “effectiveradius” of the noise vector by

(17)

Note that by the law of large numbers as, so that with high probability. Traditionally, the

scenario is such that the transmitter is subject to a power con-straint, i.e., the input must satisfy . This meansthat only a finite subset of the lattice points is used as a code-book, typically the intersection of the lattice with a sphere (or athin spherical shell) of radius and the transmission rate isthe logarithm of this number normalized by the dimension. Thisapproach originated in the work of De Buda [9] and was subse-quently refined in [25], [27], [39]. When such an approach istaken, the maximum-likelihood (ML) decoding regions are notthe Voronoi regions of the lattice due to the boundedness of thecodebook. Moreover, the decoding regions are not identical (upto translation), and in fact, some are not bounded. This breaksthe symmetry of the lattice structure in the decoding process andmakes the definition of goodness of a lattice highly dependenton the precise mode of transmission.

An alternative approach was taken by Poltyrev [32]. He con-sidered the problem of coding for the unconstrained AWGNchannel. In this scenario, any point of a lattice may be trans-mitted, corresponding to infinite power and transmission rate.For a given lattice, the ML decoder will search for the latticepoint that is nearest to the received vector. Therefore, the prob-ability of decoding error is the probability that the noise leavesthe Voronoi region of the transmitted lattice point

(18)

Since the rate of transmission is infinite, performance is mea-sured with respect to the ratio of the radius of the Voronoi re-gion and the “radius” of the noise. More precisely, define theVoronoi-to-noise effective radius ratio

(19)

Using the relation (see e.g., [42]) we have

Recalling that [8] we therefore have

(20)

where as . We ask for a lattice with min-imal probability of error for a given ratio .Poltyrev showed that reliable transmission is possible if

and that the probability of error may be ex-ponentially bounded, just as is done for the constrained AWGNchannel [17], by the random coding bound and expurgatedbound. He showed that the random coding error exponent forthe unconstrained AWGN channel is given by

(21)

for , and the expurgated exponent is given by

(22)


for . The Poltyrev exponent is given by

.(23)

More specifically, there exist lattices for which the probabilityof error satisfies the following exponential bound:

We note that this bound is exponentially tight in the spherepacking region, i.e., for . Note also that

vanishes at meaning that asymptoticreliable communication is possible as long as (andnot below it), so that or has the sig-nificance of capacity in this scenario. In fact, we may think of

as the equivalent of “rate.” As in the other problems,the (asymptotic) performance of general nonlattice codes is notsuperior to lattice codes (with respect to (w.r.t.) the bounds) [32].

In [14], Poltyrev’s unconstrained coding notion was linkedto that of coding for the power-constrained AWGN channel (viathe notions of modulo-lattice transformation and Voronoi con-stellations), thereby giving it an operational significance.

Here, we rederive Poltyrev’s error exponent in a simplifiedway and show that the lattice ensemble is good for AWGNcoding in the sense that, for any given ratio , it achievesthe Poltyrev error exponent. In Section III, we introduce the en-semble of lattices used. A nice property that we obtain is thata lattice drawn from the ensemble attains the expurgated boundwith high probability. This is in contrast to a totally random codefor which expurgation is necessary. In a sense, the expurgationis performed here on codebooks rather than codewords. This issimilar to the behavior of random binary linear codes, as op-posed to totally random binary codes [1].

III. A RANDOM ENSEMBLE OF LATTICES

Let , , and be integers such that and let be a(generating) matrix with elements in . Wedo not assume that is necessarily full rank. The generationof an -dimensional lattice by Construction A consists of thefollowing steps.

1) Define the discrete codebook, ,where all the operations are over (that is, modulo- ).Thus, .

2) Map the code into the unit cube by dividing all the com-ponents by . Thus, we define the “lattice constellation”

GRID.3) Replicate over the entire Euclidean space to form

the lattice . It is well known [8] (and ele-mentary to prove) that is indeed a lattice.

Example:Set , , and . The underlying code is given

by the generating matrix so that

We embed the code “as is” in Euclidean space as depicted inFig. 2. We skip here the second step in Construction A as definedearlier. Using this code, we tessellate the whole of resultingin the lattice

This is depicted in Fig. 3. The lower left quadrant correspondsto the CUBE region.

We note that in Step 1 runs through vectors (notnecessarily distinct), being the number of codewords. Let usindex them as , . We assume thatother than that the ordering is arbitrary. We correspondinglyindex the points of so that

The random ensemble of lattices we consider is generated asfollows.

• Take to be prime.• Draw a generator matrix according to a uniform

i.i.d. distribution over , ,; .

• Apply Construction A, i.e., steps 1–3 as previously de-scribed, to obtain the lattice .

The random ensemble is determined by , , and and inthe sequel we refer to it as an lattice ensemble. Thisensemble was considered by Loeliger in [27] and has the fol-lowing important properties.

1) deterministically.2) is distributed uniformly over GRID for

.3) The difference is uniformly distributed over

GRID for all .4) with high probability (see the following dis-

cussion).The last three properties hold since is prime (otherwise,might be restricted to some subgrid).

By construction, the lattices we consider are periodic modulothe unit cube. All the problems we consider may thus be restatedin equivalent terms in the realm of the lattice constellation inCUBE. Let , , designate the radius of a sphere. We call

CUBE a (radius ) mod-sphere. We say thatis a mod-packing if

That is, the mod-spheres do not intersect. Similarly, we say thatis a mod-covering if CUBE . It is easy

to see that the condition ensures that is a packing(covering) iff is a mod-packing (mod-covering).Therefore, we use the two viewpoints interchangeably in theproofs that follow. Fig. 4 demonstrates the relation between themod-packing and the packing in for and .The lower left quadrant is CUBE and the three black points arethe lattice constellation . The four disjoint full quarter circlesform the set .

Note that if is nonsingular then there are code-words in GRID and the volume of a Voronoi region is . Theprobability that is indeed nonsingular goes to one as


Fig. 2. Lattice constellation (scaled up by 11).

Fig. 3. Forming a lattice by Construction A—tessellating space.

. To see this, denote the rows of by , . Con-sider any specific nontrivial linear combinationdetermined by the coefficient vector . For each component

we have . Since the columnsare statistically independent we get . Thus,applying the union bound over all possible linear combinations,


Fig. 4. Modulo packing.

i.e., all coefficient vectors , we have

(24)

We restrict our attention to ensembles such that forsome so that the latter probability goes to zero atleast exponentially ( may also grow with ). In the sequel, weassume that is full rank. If it is not, we treat that as a failure,adding a vanishing term to the probability that a lattice from theensemble is not good.

In our analysis, we will hold (the effective radius of theVoronoi region) (approximately) constant while taking

. Using (5) and (6), this in turn dictates that andgrow with such that

(25)

To be more precise, since has to be prime and is an in-teger, we cannot keep strictly constant. Rather, it suffices

to choose and (as a function of ) such that as definedby (25) satisfies for all

(26)

where (in Section VII, we further restrict ).Note that (25) and (26) imply that if we hold constant, thenmust grow super-exponentially; if grows linearly with ,

then grows polynomially with . It follows from (25) that for, , we have that grows like and that

(27)

That is, goes to infinity “faster” than . We will eventuallyrequire (needed for the covering problem) that grow fasterthan . Note that this implies that

(28)

We thus restrict the growth of as a function of . Note that eventhough has to be prime, (26) may be satisfied for every largeenough . To see that, define to be the real number satisfying(25) for a radius of , i.e., . From (25) and(26), we get that must satisfy

(29)

Since we assume that with it follows that (29)may be satisfied as it is known (Bertrand’s postulate, see, e.g.,


[22]) that there is a prime number between and for anyinteger .

The number of lattice points per unit volume grows super-exponentially with the dimension (see (25)). For more phys-ical applications (power-constrained transmission, quantizationsubject to a fixed distortion per dimension) it is more natural tohave the lattice density grow exponentially with , in order tokeep the coding rate fixed. However, this is undesirable for thesake of analysis as we have seen that in order to be able to castthe two first problems in terms of mod-packing and mod-cov-ering, we need to impose , implying super-exponen-tial growth of . Thus, in order to use a lattice from the en-semble above as a code we would in practice need to appropri-ately scale it.

IV. GOODNESS FOR PACKING

Actually, Loeliger [27] used the lattice ensemble definedabove to prove that there exist lattices that are good for packing.We provide here a proof for completeness and uniformity ofpresentation.

Theorem 1: A lattice drawn from an ensemble,where , , and where , satisfy (25), (26), is goodfor packing, i.e.,

in probability as .

Note that Theorem 1 holds even for .

Proof: Denote the diagonal of an elementary cube ofGRID by

(30)

Note that by (27), as . Let .Consider the mod-sphere . We show that for large enoughdimension, with high probability no other sphere intersects it,i.e.,

(31)

This in turn implies that

is a packing (32)

Consider , a mod-sphere centered at GRID . Itintersects the mod-sphere centered at the origin iff

(see Fig. 4). Let denote the set of grid points that arewithin a (modulo) radius of from the origin

GRID

Therefore, the probability that a mod-sphere centered atintersects the mod-sphere at the origin satisfies

Let stand for the set of grid points such their fine cubic cellsof side are fully contained in a sphere of radius fromthe origin

GRID CUBE

Notice that . We may therefore bound the cardinalityof as follows:

CUBE

(33)

Consider some arbitrary index . Since is uniformlydistributed over the points of GRID , using (33) we get

(34)

(35)

(36)

(37)

Thus, by the union bound we have

(38)

(39)

Now the number of codewords and the effective radius arerelated by

CUBE

so that

is a packing (40)

(41)

Recall that so by (27), as provided that, where . Therefore, as long as we have ,

(32) holds. Recalling the definition of (7), we concludethat the theorem is proved.

We note again that and are required to satisfy (25). Thisleaves us with a wide range of possible values for and . Wecould even take . As we shall see next, however, such achoice would not allow us to prove that the ensemble is good forcovering. We will ultimately find a sequence of pairssuch that the ensemble will be good in all four senses.

V. GOODNESS FOR COVERING

Theorem 2: A lattice drawn from an ensemble,where for some , but grows faster than ,and where , satisfy (25), (26), is good for covering, i.e.,

in probability as .Proof: The proof is a variation on its binary counterpart

as given in [3]; see also [6, Ch. 12]. We divide the proof into


two parts. In the first, we obtain sufficient conditions such thatmost lattices in the ensemble are “almost complete” coverings(in a sense to be specified below). In the second, we imposestronger conditions on the ensemble so that most of the latticesare complete coverings.

Part I: Almost complete coveringConsider the lattice constellation of the ensemble. Let ,be such that . Denote the lattice constellation

obtained from the first rows of by and let ,, denote the lattice constellation obtained from the

first rows of . Consider an arbitrary point CUBE.For an , to be chosen later, let denote the set of gridpoints within a (modulo) distance from

GRID (42)

Recall that is half the diagonal of an elementary cube ofGRID as defined in (30). Let denote the set of gridpoints such that their fine cubic cells intersect a sphere of radius

centered at

GRID CUBE

Notice that . We may therefore bound the cardi-nality of as follows:

CUBE

(43)

Therefore, by the second property of the lattice ensemble (seeSection III), the probability that is covered by a sphere ofradius centered at any specific point of satisfies

(44)

where and recall that is the th point of . Definethe indicator random variable for

ifotherwise.

(45)

Note, that we do not consider since determin-istically. Excluding from our consideration ensures that

is statistically independent of both and . Defineas

(46)

so that is equal to the number of nonzero codewords -covering . Taking the expectation of and using (44) we get

(47)

(48)

which (since ), is well approximated as

(49)

(50)

Note that both and the lower bound are independent of .Using the pairwise independence of the ’s (which implies thatthey are uncorrelated), we have

(51)

(52)

(53)

(54)

(55)

Using (55), by Chebyshev’s inequality, for any

(56)

Denote

(57)

Then from (56) we have

(58)

Note that if , then is also upper-boundedby .

We call a point CUBE remote from a (discrete) set ofpoints if it is not -covered by , i.e.,if belongs to an -sphere centered at any point of .Thus, amounts to “ is remote from .” Let

stand for the set of remote points from . Denote, and let

CUBE

be the fraction of points which are remote from . We have

(59)

(60)

provided that . Thus, from (58) it follows that

(61)

Using Markov’s inequality, we estimate the deviation of thisfraction of points from the mean

(62)

Therefore, using (61)

(63)

Thus, in words, the probability that the fraction of points ofCUBE that are remote from is greater than is smallerthan . By taking (but still keeping ) thisprobability can be made arbitrarily small as . In orderto have both and , it is sufficient to take

and for some . By (48), this inturn will be satisfied if we choose a radius that satisfies

(64)


Thus, we may conclude that with these choices of parameters,for most lattices almost all points are covered by spheres ofradius . We note that up to this point we may take

and . Therefore, we can obtain an “almost complete”-covering of CUBE with .

Part II: Complete coveringWe wish to obtain an ensemble of lattices such that for most

of its members we have all the points of CUBE covered. We nextshow that this may be achieved by having grow at least as fastas . The key to the proof follows from the observation thata fraction of uncovered points is at least squared when goingform to . We also restrict attention to pointsbelonging to GRID . We correspondingly redefine to bethe set of grid points, i.e., GRID , which are remote from

and similarly for . If we obtain an -covering of allthe grid points, then we will have an -covering of all the pointsof CUBE.

We show that by augmenting the generating matrix with anadditional “small” number of rows , we can makethe fraction of uncovered grid points smaller that GRID ,i.e., smaller than the inverse of the number of points in the cubicconstellation GRID . This will imply that all grid points arecovered as desired.

Choose and such that (25) and (26) are satisfied (playing the role of ) with growing faster than . Con-sider the set . Notice that

(65)

so that and therefore is upper-bounded byGRID . Also note that is an indepen-

dent shift of . Thus, conditional on , the event thatis remote from is independent of whether

is remote from , and the probability of this event is . Itfollows that conditioned on , the probability that is remotefrom is , implying that (as in (61))

GRID(66)

Since , we have . ApplyingMarkov’s inequality we have

(67)

Therefore,

(68)

Using Bayes’ rule and (63)

(69)

(70)

Continuing this procedure, we have for , ,that conditional on

(71)

(72)

with probability at least . The intersection of theseevents and the event has probability of at least

, and implies2 in particular

(73)

We would like to choose such that

(74)

as this would imply that since there are points inGRID . It suffices that and

(75)

or faster. Recalling that , we conclude that withprobability at least

(76)

satisfies , i.e., that every GRID is coveredby at least one sphere of radius . We now impose a strictercondition on than that imposed at the end of Part I, toensure that both and the probability in (76) goes to one,as . An adequate choice is

(77)

As , this implies that the probability that there is anypoint GRID that is not -covered, is arbitrarily smallas . Recall that if every point of GRID is covered by asphere of radius this implies that CUBE is completelycovered by spheres of radius . Therefore, the probability of acomplete covering with spheres of radius goes to one wheresatisfies (see (64))

(78)

(79)

(80)

We have from (80)

(81)

(82)

For efficient covering, i.e., , we wish that the left-handside would go to one. It is sufficient that each of the three termson the right-hand side of (82) goes to one. By (27) and (30), wehave as , provided that where ,and thus also

2Note that the recursion a = + 2a implies a = 2 ( + a )� .


For any fixed , we have . Also since by(28) as grows faster than , we have

Thus, we have that as . This completes theproof.

VI. GOODNESS FOR MSE QUANTIZATION

As was shown in [42], specifically in (27) therein, a latticethat is good for covering is necessarily good for quantization.More specifically.

Proposition 1: For any lattice

where is the normalized second moment of an -di-mensional sphere.

Combining this proposition and Theorem 2 we get the fol-lowing.

Theorem 3: A lattice drawn from an ensemble,where for some , but grows faster than ,and , satisfy (25), (26), is good for quantization, i.e.,

in probability as .

VII. GOODNESS FOR AWGN CHANNEL CODING

In this section, we show that this ensemble achieves thePoltyrev random coding error exponent for any noise. In partic-ular, we present a simple derivation for the Poltyrev exponent.We note that Loeliger [27] showed that the ensemble definedin Theorem 1 indeed achieves capacity, but did not point outthat the ensemble also achieves the Poltyrev exponent. In thissection, we need to restrict , requiring it to satisfy

(83)

where and were defined in Section II-D. We havethe following theorem.

Theorem 4: A lattice drawn from an ensemble,where for , and where , satisfy (25), (26) with

given by (83) is good for coding for the AWGN channel,i.e.,

with probability tending to one as . Here as, but may depend on .

Note that using the definition of (19) and of (17)we have

(84)

Thus, holding fixed means that the variance of the noisedecreases inversely with . We also note that in this theoremwe may take .

Proof of Theorem 4: For a particular value of wewish to show that with high probability (w.r.t. the ensemble oflattices)

(85)

We prove the theorem through a number of lemmas. Define anauxiliary “truncated” Gaussian noise so that its density is

ifotherwise

(86)

where

(87)

is the probability of truncation. Consider the modulo additivenoise channel

(88)

Since is proportional to for any CUBE itfollows that for the noise the decoding regions of arethe Voronoi regions, just as for the original noise . Thus, theprobability of error when transmitting a point of over thischannel is indeed

(89)

Denote also the random coding and expurgated exponents of thischannel by and , respectively.

Lemma 1: For almost all lattices in the considered ensemble,. The proof is given in Ap-

pendix I.

We now bound the error probability of the original AWGNchannel for Euclidean decoding with this noise by the errorprobability corresponding to the - channel. For thoselattices satisfying Lemma 1 we have

CUBE CUBE (90)

(91)

(92)

(93)


So it is sufficient to show that decays exponen-tially in with an exponent greater than . But forsmall enough we have

Therefore, we would like to have

This in turn is satisfied by (83). This completes the proof of thetheorem.

VIII. SIMULTANEOUS GOODNESS

Verifying that there exists a nonempty intersection of the ad-missible sets of Theorems 1–4, the result now followsby the union bound.

Theorem 5: Let be given. Let

(94)

where is given by (23). Let be drawn from anensemble, where for some but grows

faster than , and where , satisfy

(95)

for some such that . Thenis good in all four senses with probability tending to one as

, i.e.,

and

in probability as .

To carry the analogies farther, we may note the following. Letdenote the surface of an -dimensional sphere of radius one

and let be a random direction uniformly distributed over .For a given lattice, define the radius of the basic Voronoi regionin direction as the intersection of a ray emanating from theorigin in direction

(96)

Thus, is a random variable taking values in the interval. The above results yield that for the defined lattice

ensemble as

in probability

and

in probability

Fig. 1 depicts the various radii relevant to the different problems.

IX. SUMMARY AND EXTENSIONS

We considered the problems of packing and covering withEuclidean spheres, MSE quantization, and AWGN channelcoding. Using random coding techniques we demonstratedthat there exist lattices that are good for these four problemssimultaneously. We saw that for the channel coding problemas well as for the packing problem we had a lot of freedom inthe choice of the behavior of , the cardinality of the primefield used, and , the dimension of the underlying linear code,as a function of . In fact, for these problems it is sufficient totake , meaning that a good lattice may be generated bychoosing a single point. It seems that the same is true for thequantization problem (even though we derived the result forquantization as a corollary of that for covering). The coveringproblem, on the other hand, appears to be the most demanding.In order to obtain a good lattice for this problem we were forcedto take growing faster than . It remains to be studiedwhether this condition is indeed necessary or may be relaxed.

For practical applications, it is interesting to study the per-formance associated with constructions having grid resolution(the alphabet size of the underlying linear code) fixed. It isclear that this precludes achieving good performance for thepacking and covering problems. On the other hand, for thechannel coding and quantization problems it is known thatone can approach the optimal performance bounds arbitrarilyclosely (as the dimension goes to infinity), provided that wetake large enough but constant.

As discussed in Section II, one way to use a lattice for trans-mission over a power-constrained AWGN channel is to use acodebook which is the intersection of the lattice with a sphere ofradius . For the considered ensemble of lattices, we sawthat for a constant , the corresponding noise variance de-creases inversely with . Thus, in such a scheme one wouldappropriately scale the lattice obtained from the ensemble ac-cording to the variance of the channel noise. The results are alsodirectly applicable to the lattice coding scheme of [14] that al-lows to achieve the AWGN capacity using lattice encoding anddecoding. Such a scheme incorporates a pair of nested lattices,the Voronoi region of the coarse lattice serving as a shaping re-gion, the fine lattice serving as a channel code. It turns out thatit is important that the coarse lattice be also good for channelcoding. Thus, a lattice obtained from the ensemble proposed inthis work may serve as the coarse (shaping) region. The gener-ation of the fine lattice code is described in [14]. Applicationswhere the necessity for lattices which are simultaneously goodfor channel coding and MSE quantization also arise are in lat-tice-based schemes in various multiterminal problems. Amongthem are the nested code approach for the Costa problem andthe Wyner–Ziv problem, see [44].

These quadratic/Euclidean-norm/Gaussian problems are nat-ural in . However, it may be readily verified that Theorems


1 and 2 equally hold for spheres corresponding to any norm.Specifically, consider the case of an th power norm

(97)

From the result for covering we get as a corollary, as done in theEuclidean case in Section V, the existence of good quantizersfor the single-letter distortion measure .Likewise, the results for channel coding can be extendedto more general noise distributions including the exponentialfamily. By a union bound argument, this multinorm/multimetricoptimality implies an even stronger notion of simultaneousgoodness, namely, that there exist lattices which are not onlygood for the four criteria of sphere arrangements, but also forany finite collection of distortion measures and/or additivenoise channels from the family above.

APPENDIX IPROOF OF LEMMA 1

The difference between any two codewords in the consideredensemble, i.e., , is uniformly distributed over GRID .Therefore, it has the same distribution as that of an ensemble ob-tained by drawing each codeword according to a uniform distri-bution over GRID . Thus, the ensemble has the same randomcoding and expurgated exponent as the totally random one (see[17], [38]). Denote the corresponding random coding exponentof (88) by , where the bar denotes averagingover the ensemble. Similarly, denote the expurgated error expo-nent of (88) corresponding to a uniform distribution over GRIDby and let

(98)Earlier we concluded in Theorem 1 that for almost all latticesfrom the considered ensemble, the minimum distance betweenlattice points satisfies the Minkowski bound. This means thatthe expurgated exponent is achieved for almost all lattices inthe ensemble. Thus, the following applies to this subset (theaveraging is only over this subset).

(99)

In Appendix II we show that approaches, as, the exponent of a uniform distribution over CUBE,

which we denote by . That is

(100)

The claim for the expurgated exponent may be similarly proved.In Appendix III we show that

(101)

and

(102)

for the corresponding values of . Thus the lemma isproved.

APPENDIX IIPROOF OF (100): SENSITIVITY OF ERROR EXPONENT

W.R.T. AN INPUT ON GRID

Consider the random coding error exponent corresponding tothe random code restricted to GRID . It is given by

(103)

where

(104)

Compare this to the random coding exponent corresponding toa uniform input, which is given by

(105)

where we get (106) and (107) at the bottom of the page. Wenext show that for any and

(108)

For any

(109)

Since

(110)

we have

(111)

(106)

(107)


Taking into account (84) and that , , andwe have

(112)

Thus, by the conditions of Theorem 4, i.e., since , wehave that (see derivation of (27)). Therefore,we have that

(113)

Consequently, we obtain (108). Substituting (108) into (107), itfollows that

(114)

This proves the lemma.

APPENDIX IIICALCULATION OF EXPONENTS

The following derivation (as well as Appendix B) is a specialcase of a calculation carried out in [14, the Appendix]. We in-clude it for completeness.

For any modulo additive noise channel over the basic interval

(115)

the random coding exponent is given by (see, e.g., [13])

(116)

(117)

(118)

(119)

(120)

where (length of unit interval) and wheredenotes Rényi entropy, defined for any ,

and

(121)

is the normalized density of the lattice points per dimension, i.e.,the normalized number of lattice points per unit volume.

Thus, the random coding exponent of the -modulo addi-tive noise channel is given by

(122)

We further have

(123)

(124)

(125)

Taking we get

(126)

We therefore have

(127)For a Gaussian random variable , the Rényientropy is

(128)

(129)

The maximization in (127) is carried out explicitly in Ap-pendix III-A to yield

(130)

This yields (101) for in the random coding region, i.e.,corresponding to the first line of (23).

We next treat the expurgated exponent. Define the followinggeneralized Bhattacharyya distances, for :

(131)

and

(132)

For a modulo additive noise channel the expurgated exponentis achieved by a uniform input [17], giving (133) at the top ofthe following page. Similarly to (126) we have

(134)

Thus,

(135)


(133)

In Appendix C-II, the maximization of (135) is explicitly carriedout to yield

(136)

for .

A. Maximization of (127)

For Gaussian we have

(137)

(138)

(139)

We therefore have

(140)

Taking the derivative of with respect to we get

(141)

Thus, an extremum occurs when

(142)

or, equivalently, when

(143)

It is easy to verify that this extremum is indeed a maximum.Substituting (142) and (143) into (140) we get

(144)

From the definition of (121) we get

(145)

Substituting (145) for in (144) we get

(146)

(147)

(148)

B. Maximization of (135)

We use the following property of Gaussian distributions:

(149)

Let . Then the preceding relation holds since

(150)

(151)

(152)

(153)

Using (150) we obtain

(154)

(155)

(156)

(157)

(158)

Thus, it is left to show that

(159)

for . Differentiating the left-hand side of (159) weget

(160)


(161)

Equating to zero we obtain

(162)

or equivalently

(163)

Taking into account that , we get

(164)

Finally, note that from (163) we have that indeed corre-sponds to a rate satisfying

(165)

Finally, the straight-line part of is obtained bycombining the results for the random coding exponent and theexpurgated exponent.

REFERENCES

[1] A. Barg and G. D. Forney Jr, “Random codes: Minimum distances anderror exponents,” IEEE Trans. Inf. Theory, vol. 48, no. 9, pp. 2568–2573,Sep. 2002.

[2] T. Berger, Rate Distortion Theory: A Mathematical Basis for Data Com-pression. Englewood Cliffs, NJ: Prentice-Hall, 1971.

[3] V. M. Blinovskii, “A lower bound on the number of words of a linearcode in an arbitrary sphere with given radius in ” (in Russian), Probl.Pered. Inform. (Probl. Inf. Transm.), vol. 23, no. 2, pp. 50–53, 1987.

[4] E. L. Blokh and V. V. Zyablov, Linear Concatenated Codes (in Rus-sian). Nauka, U.S.S.R.: Moscow, 1982.

[5] G. J. Butler, “Simultaneous packing and covering in Euclidean space,”Proc. London Mat. Soc., vol. 25, pp. 721–735, 1972.

[6] G. Cohen, I. Honkala, S. Litsyn, and A. Lobstein, CoveringCodes. Amsterdam, The Netherlands: North-Holland, 1997.

[7] G. D. Cohen, “A nonconstructive upper bound on covering radius,” IEEETrans. Inf. Theory, vol. IT-29, no. 3, pp. 352–353, May 1983.

[8] J. H. Conway and N. J. A. Sloane, Sphere Packings, Lattices andGroups. New York: Springer-Verlag, 1988.

[9] R. de Buda, “Some optimal codes have structure,” IEEE J. Sel. AreasCommun., vol. 7, no. 6, pp. 893–899, Aug. 1989.

[10] P. Delsarte and P. M. Piret, “Do most binary linear codes achieve theGoblick bound on the covering radius?,” IEEE Trans. Inf. Theory, vol.IT-32, no. 6, pp. 826–828, Nov. 1986.

[11] R. L. Dobrushin, “Asymptotic optimality of group and systematic codesfor some channels,” Theor. Probab. Appl., vol. 8, pp. 52–66, 1963.

[12] P. Elias, “Coding for noisy channels,” in IRE Conv. Rec., 1955, pp.37–46. Also appears in Key Papers in the Development of InformationTheory, D. Slepian, Ed. New York: IEEE Press, 1974, pp. 102-111..

[13] U. Erez and R. Zamir, “Error exponents of modulo additive noise chan-nels with side information at the transmitter,” IEEE Trans. Inf. Theory,vol. 47, no. 1, pp. 210–218, Jan. 2001.

[14] , “Achieving log(1+SNR) on the AWGN channel with latticeencoding and decoding,” IEEE Trans. Inf. Theory, vol. 50, no. 10, pp.2293–2314, Oct. 2004.

[15] G. D. Forney, “On the duality of coding and quantizing. Coding andquantization,” in DIMACS Series on Discrete Mathematics and Theo-retical Computer Science. Providence, RI: Amer. Math. Soc., 1993,pp. 1–14.

[16] R. G. Gallager, Low Density Parity Check Codes. Cambridge, MA:MIT Press, 1963.

[17] , Information Theory and Reliable Communication. New York:Wiley, 1968.

[18] A. Gersho, “Asymptotically optimal block quantization,” IEEE Trans.Inf. Theory, vol. IT-25, no. 4, pp. 373–380, Jul. 1979.

[19] E. N. Gilbert, “A comparison of signalling alphabets,” Bell Syst. Tech.J., vol. 31, pp. 504–522, 1952.

[20] T. J. Goblick, “Coding for a discrete information source with a distortionmeasure,” Ph.D. dissertation, MIT, Dept. Elec. Eng., Cambridge, MA,1962.

[21] R. M. Gray and D. L. Neuhoff, “Quantization,” IEEE Trans. Inf. Theory,vol. 44, no. 6, pp. 2325–2383, Oct. 1998.

[22] G. H. Hardy and E. M. Wright, An Introduction to the Theory of Num-bers, 5th ed. Oxford, U.K.: Oxford Univ. Press, 1979.

[23] G. A. Kabatiansky and V. I. Levenshtein, “Bounds for packings on asphere and in space” (in Russian), Probl. Pered. Inform., vol. 14, no. 1,pp. 3–25, 1978. English translation in Probl. Inf. Transm., vol. 14, no. 1,pp. 1–17, 1978.

[24] T. Linder and R. M. Gray, “Mismatch in high-rate entropy-constrainedvector quantization,” IEEE Trans. Inf. Theory, vol. 49, no. 5, pp.1204–1217, May 2003.

[25] T. Linder, C. Schlegel, and K. Zeger, “Corrected proof of de Buda’stheorem,” IEEE Trans. Inf. Theory, vol. 39, no. 5, pp. 1735–1737, Sep.1993.

[26] T. Linder and R. Zamir, “On the asymptotic tightness of the Shannonlower bound,” IEEE Trans. Inf. Theory, vol. 40, no. 6, pp. 2026–2031,Nov. 1994.

[27] H. A. Loeliger, “Averaging bounds for lattices and linear codes,” IEEETrans. Inf. Theory, vol. 43, no. 7, pp. 1767–1773, Nov. 1997.

[28] F. J. MacWilliams and N. J. A. Sloane, The Theory of Error CorrectingCodes. Amsterdam, The Netherlands: Elsevier, 1977.

[29] H. Minkowski, “Dichteste gitterförmige Lagerung kongruenter Körper,”Nachr. Ges. Wiss. Göttingen, pp. 311–355, 1904.

[30] J. N. Pierce, “Limit distribution of the minimum distance of randomlinear codes,” IEEE Trans. Inf. Theory, vol. IT-13, no. 4, pp. 595–599,Oct. 1967.

[31] Handbook of Coding Theory, V. S. Pless, W. C. Huffman, and R. A.Brualdi, Eds., North-Holland, Amsterdam, The Netherlands, 1998.

[32] G. Poltyrev, “On coding without restrictions for the AWGN channel,”IEEE Trans. Inf. Theory, vol. 40, no. 2, pp. 409–417, Mar. 1994.

[33] C. A. Rogers, “A note on coverings,” Mathematica, vol. 4, pp. 1–6, 1957.[34] , “Lattice coverings of space,” Mathematica, vol. 6, pp. 33–39,

1959.[35] , Packing and Covering. Cambridge, U.K.: Cambridge Univ.

Press, 1964.[36] C. E. Shannon, “A mathematical theory of communication,” Bell Syst.

Tech. J., vol. 27, pp. 379–423, Jul. 1948.[37] , “Probability of error for optimal codes in a Gaussian channel,”

Bell Syst. Tech. J, vol. 38, pp. 611–656, May 1959.[38] N. Shulman and M. Feder, “Random coding techniques for nonrandom

codes,” IEEE Trans. Inf. Theory, vol. 45, no. 9, pp. 2101–2104, Sep.2002.

[39] R. Urbanke and B. Rimoldi, “Lattice codes can achieve capacity on theAWGN channel,” IEEE Trans. Inf. Theory, vol. 44, no. 1, pp. 273–278,Jan. 1998.

[40] R. R. Varshamov, “Estimate of the number of signals in error correctingcodes” (in Russian), Dokl. Akad. Nauk SSSR, vol. 117, no. 5, pp.739–741, 1957.

[41] P. L. Zador, “Topics in the Asymptotic Quantization of ContinuousRandom Variables,” Bell Labs., Tech. Memo, 1966.

[42] R. Zamir and M. Feder, “On lattice quantization noise,” IEEE Trans. Inf.Theory, vol. 42, no. 4, pp. 1152–1159, Jul. 1996.

[43] , “On universal quantization by randomized uniform/lattice quan-tizer,” IEEE Trans. Inf. Theory, vol. 38, no. 2, pp. 428–436, Mar. 1992.

[44] R. Zamir, S. Shamai (Shitz), and U. Erez, “Nested linear/lattice codesfor structured multiterminal binning,” IEEE Trans. Inf. Theory, vol. 48,no. 1, pp. 1250–1276, Jun. 2002.

[45] J. Ziv, “On universal quantization,” IEEE Trans. Inf. Theory, vol. IT-31,no. 2, pp. 344–347, May 1985.

lattices which are good for (almost) everything

Documents