discrete mathematics

Discrete Mathematicsincomplete working draft

do not distribute without permission from the authors

P. Gouveia P. Mateus J. Rasga C. Sernadas

Instituto Superior TecnicoDepartamento de Matematica

2011

Contents

Preface 7

1 Modular congruences 91.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.2 Divisibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.2.1 Divisors, quotients and remainders . . . . . . . . . . . . . 111.2.2 Euclid’s algorithm . . . . . . . . . . . . . . . . . . . . . . 131.2.3 Prime numbers . . . . . . . . . . . . . . . . . . . . . . . . 20

1.3 Modular arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . 241.3.1 Congruence modulo n . . . . . . . . . . . . . . . . . . . . 241.3.2 The rings Zn . . . . . . . . . . . . . . . . . . . . . . . . . 301.3.3 The Chinese Remainder Theorem . . . . . . . . . . . . . . 37

1.4 RSA revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2 Pseudo-random numbers 512.1 Motivation: traffic simulation . . . . . . . . . . . . . . . . . . . . 522.2 Linear congruential generators . . . . . . . . . . . . . . . . . . . . 542.3 Blum-Blum-Schub generators . . . . . . . . . . . . . . . . . . . . 592.4 Traffic simulation revisited . . . . . . . . . . . . . . . . . . . . . . 592.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

3 Polynomials 653.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.1.1 Digital circuit equivalence . . . . . . . . . . . . . . . . . . 653.1.2 Inverse kinematics of a robot . . . . . . . . . . . . . . . . . 66

3.2 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683.2.1 Rings of polynomials . . . . . . . . . . . . . . . . . . . . . 683.2.2 Monomial orderings . . . . . . . . . . . . . . . . . . . . . . 733.2.3 Division of terms and polynomials . . . . . . . . . . . . . . 773.2.4 Reduction modulo a set of polynomials . . . . . . . . . . . 84

3.3 Grobner bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 903.3.1 Ring ideals . . . . . . . . . . . . . . . . . . . . . . . . . . 91

3

4 CONTENTS

3.3.2 Buchberger criterion . . . . . . . . . . . . . . . . . . . . . 95

3.3.3 Buchberger algorithm . . . . . . . . . . . . . . . . . . . . . 101

3.3.4 Properties of Grobner basis . . . . . . . . . . . . . . . . . 111

3.4 Motivating examples revisited . . . . . . . . . . . . . . . . . . . . 118

3.4.1 Equivalence of digital circuits . . . . . . . . . . . . . . . . 118

3.4.2 Inverse kinematics . . . . . . . . . . . . . . . . . . . . . . 122

3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

4 Euler-Maclaurin formula 129

4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

4.2 Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

4.3 Main results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

4.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

4.4.1 Gaussian elimination . . . . . . . . . . . . . . . . . . . . . 146

4.4.2 Insertion sort . . . . . . . . . . . . . . . . . . . . . . . . . 149

4.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

5 Discrete Fourier transform 159

5.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

5.2 Discrete Fourier transform . . . . . . . . . . . . . . . . . . . . . . 161

5.2.1 Complex roots of unity . . . . . . . . . . . . . . . . . . . . 161

5.2.2 Discrete Fourier transform . . . . . . . . . . . . . . . . . . 163

5.3 Fast Fourier transform . . . . . . . . . . . . . . . . . . . . . . . . 167

5.4 Polynomial multiplication revisited . . . . . . . . . . . . . . . . . 171

5.4.1 Coefficient and point-value representations . . . . . . . . . 171

5.4.2 Polynomial multiplication and FFT . . . . . . . . . . . . . 174

5.5 Image processing . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

5.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

6 Generating functions 179

6.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

6.1.1 Search by hashing . . . . . . . . . . . . . . . . . . . . . . . 179

6.1.2 Euclid’s algorithm . . . . . . . . . . . . . . . . . . . . . . 182

6.2 Generating functions . . . . . . . . . . . . . . . . . . . . . . . . . 183

6.3 Motivating examples revisited . . . . . . . . . . . . . . . . . . . . 194

6.3.1 Search by hashing . . . . . . . . . . . . . . . . . . . . . . . 194

6.3.2 Euclid’s algorithm . . . . . . . . . . . . . . . . . . . . . . 205

6.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

References 214

Subject Index 219

CONTENTS 5

Table of Symbols 223

6 CONTENTS

Preface

The material presented herein constitutes a self-contained text on several topicsof discrete mathematics for undergraduate students. An outline of this text is asfollows.

In Chapter 1 we address modular congruences and their applications. Amotivating example from public key cryptography is presented in Section 1.1.In Section 1.2, we introduce several concepts and results related to divisibility.Section 1.3 concentrates on modular arithmetic including basic properties of therings Zn. Then, in Section 1.4, we revisit the motivating example.

In Chapter 2 we discuss the generation of pseudo-random numbers. Randomnumbers are useful in several different fields ranging from simulation and samplingto cryptography. In Section 2.1 we present a motivating example related to trafficsimulation. In section 2.2 we introduce linear congruential generators. Blum-Blum-Shub generators are presented in in Section 2.3. The traffic simulationexample is revisited in Section 2.4.

Chapter 3 presents several key concepts and results related to polynomials. InSection 3.1 we first discuss a motivating example illustrating how polynomials canbe used to verify equivalence of digital circuits. Then we illustrate the relevanceof polynomials in robotics. In Section 3.2 we introduce the notion of polynomialover a field as well as the sum and product of polynomials. We then introducedivision of polynomials and several related results. Grobner bases and theirproperties are presented in Section 3.3. In Section 3.4 we revisit our motivatingexamples and show how to use Grobner bases for finding solutions of systems ofnonlinear polynomial equations.

In Chapter 4, we introduce several techniques to compute summations. InSection 4.1 we present a motivating example in Bioinformatics. In Section 4.2we introduce summation expressions and some of their relevant properties. TheEuler-Maclaurin formula is presented in Section 4.3. In Section 4.4, we illustratethe relevance of summations to the analysis of the Gauss elimination techniqueand the insertion sort algorithm.

In Chapter 5 we present the discrete Fourier transform The discrete Fouriertransform is widely used in many fields, ranging from image processing to efficientmultiplication of polynomials and large integers. In section 5.1 we discuss amotivating example that illustrates the use of the discrete Fourier transform

7

8 CONTENTS

for efficient polynomial multiplication. In Section 5.2 we introduce the discreteFourier transform and in Section 5.3 we present an efficient method for computingthe discrete Fourier transform, the fast Fourier transform. Then, in Section 5.4,we revisit polynomial multiplication based on the discrete Fourier transform.Image processing using the fast Fourier transform is discussed in Section 5.5.

In Chapter 6 we introduce generating functions. In Section 6.1 we presentmotivating examples of the use of generating functions in algorithm analysis andrecurrence relation solving. Generating functions are introduced in Section 6.2,together with their sum, product, derivative and integral. In section 6.3 we revisitthe motivating examples.

1.Modular congruences

2.Pseudo-randomnumbers

3.Polynomials

5.Discrete Fourier transform

4.Euler-MacLaurin formula

6.Generating functions

Figure 1: Chapter dependencies

At the end of each section several exercises are proposed. Chapter dependen-cies are described in Figure 1.

Chapter 1

Modular congruences

In this chapter we address modular congruences and their applications. Thechapter is organized as follows. We start in Section 1.1 with a motivating examplefrom public key cryptography. Then, in Section 1.2, we introduce several conceptsand results related to divisibility. Section 1.3 concentrates on modular arithmeticincluding basic properties of the rings Zn. In Section 1.4 we revisit the example.In Section 1.5 we propose some exercises.

1.1 Motivation

Consider the problem of sending a secret message through a public channel be-tween two parties, say from Alice to Bob, that have not previously agreed upona key. The most-well known solution to this problem consists in using a publickey protocol to exchange the message. In public key cryptography, each partyhas a pair of keys: a public key and a corresponding private key. The publickey, as the name suggests, is made public whereas the private key is kept secret.Messages are encrypted with the public key and can only be decrypted with thecorresponding private key. Moreover, it should be hard to get the private keyfrom the public one, even in the presence of an encrypted message.

Let K be the set of public keys and R a set of private keys, and X, Y besets. Encryption is described as a family of maps u = uk : X → Y k∈K anddecryption as a family of maps v = vr : Y → Xr∈R such that:

1. for each public key k ∈ K there is a unique private key rk ∈ R such thatvrk uk = id;

2. uk and rk must be computed efficiently (in polynomial-time);

3. it should be hard to invert uk, when rk is not known.

To send a message x to Bob, Alice first uses Bob’s public key k to obtain theciphered text uk(x). When Bob receives uk(x) over the public channel, he useshis private key rk to obtain the original message vrk

(uk(x)) = x.

9

10 CHAPTER 1. MODULAR CONGRUENCES

Due to Property 3, it is hard for a third party, say Eve, that eavesdrops thechannel and knows uk(x) and k to obtain x.

Example 1.1.1 The RSA cryptosystem is due to R. Rivest, A. Shamir and L.Adelman [26]. It can be characterized as in Figure 1.1, where mod(n, m) is theremainder of the (integer) division of n by m.

Let n, p, q, a and b be natural numbers such that

n = p× q where p and q are prime numbers

mod(a× b, (p− 1)× (q − 1)) = 1

Message space: X = Y = 0, 1, 2, . . . , n− 1

Public key: (n, a)

u(n,a)(x) = mod(xa, n)

Private key: (n, b)

v(n,b)(y) = mod(yb, n)

Figure 1.1: RSA cryptosystem

For Alice to send a message to Bob, the latter chooses two prime numbers pand q and computes n = p× q and (p− 1)× (q − 1). Then, he chooses a and bsuch that mod(a× b, (p− 1)× (q − 1)) = 1 and publishes (n, a).

For the RSA cryptosystem to work we need to show that a, b, u and v can beefficiently computed and that the equality

vrk uk = idX

indeed holds. The proof uses several notions and results that are presented inthis chapter.

As we shall see, the security of RSA relies on the conjecture that it is notpossible to factorize integers efficiently (in polynomial-time). This conjecturemay well not be true. Indeed, it has been shown by Shor [28] that quantumcomputers factorize in polynomial time. Therefore, the RSA must be abandonedif and when quantum computers become available.

1.2 Divisibility

In this section we introduce the notions regarding divisibility.

1.2. DIVISIBILITY 11

1.2.1 Divisors, quotients and remainders

We start by recalling divisor of an integer number as well as remainder andquotient of integer division.

Definition 1.2.1 Let m, n ∈ Z. We say that m divides n, and write

m|n

whenever there is k ∈ Z such that n = k ×m. When m divides n, we also saythat m is a divisor of n and that n is a multiple of m.

Observe that each integer m divides 0 and that 1 divides any integer n.

The function dividesQ in Figure 1.2 returns True if m divides n and False

otherwise. It uses the Mathematica predicate IntegerQ.

dividesQ=Function[m,n,IntegerQ[n/m]]

Figure 1.2: Divisor test in Mathematica

The remainder of the division of an integer n by an integer m play an impor-tant role in the sequel. We first consider the case where m is positive.

Definition 1.2.2 Let m, n ∈ Z with m > 0. We say that q ∈ Z is the quotientand r ∈ Z is the remainder of the integer division of n by m whenever

n = q ×m + r and 0 ≤ r < m

For simplicity we often only refer to the quotient and remainder of the divisionof n by m. The following result establishes that they are unique.

Proposition 1.2.3 For each n, m ∈ Z with m > 0 there are unique integers qand r such that n = q ×m + r and 0 ≤ r < m.

Proof: Let S = n − k × m : k ∈ Z. The set S ∩ N0 is not empty since,for instance, n + |n| × m ∈ S. Let r be the least element of S ∩ N0. Then,r = n− q×m for some q ∈ Z, and therefore n = q×m + r with r ≥ 0. If r ≥ m,considering r′ = n − (q + 1) ×m and recalling that m > 0, it holds that r 6= r′

and 0 ≤ r′ = r − m < r, contradicting the fact that r is the least element inS ∩N0. Hence, r < m.


We now prove the unicity. Assume there are integers q, q′, r and r′ such thatn = q ×m + r and n = q′ ×m + r′ with 0 ≤ r, r′ < m. Then,

m× (q − q′) = r′ − r

Assuming without loss of generality that q′ ≤ q and recalling that m > 0, we canconclude that either r′ − r = 0 or r′ − r ≥ m. But, since 0 ≤ r, r′ < m, it holdsthat r′ − r < m. Hence, r′ − r = 0 = m× (q − q′). Therefore, r = r′. Moreover,given that m > 0, also q − q′ = 0, that is, q = q′. QED

Let n = q ×m + r be the equality in Proposition 1.2.3. Observe that

q =⌊ n

m

⌋

,

that is, q is the Floor of nm

. Recall that the map Floor associates with each xthe integer ⌊x⌋, that is, the largest integer k such that k ≤ x. We denote theremainder of the division by

mod(n, m).

Clearly,

mod(n, m) = n−⌊ n

m

⌋

×m

for n, m ∈ Z, m > 0.We now present some useful facts about remainders.

Proposition 1.2.4 Let n, n′, m, k ∈ Z with m, k > 0.

1. If 0 ≤ n < m then mod(n, m) = n.

2. mod(n, m) = mod(n′, m) if and only if n− n′ is a multiple of m.

3. If k divides m then mod(mod(n, m), k) = mod(n, k).

Proof:1. Since n = 0×m + n and 0 ≤ n < m, by Proposition 1.2.3, mod(n, m) = n.

2. Assume that mod(n, m) = mod(n′, m). Since

n =⌊

nm

⌋×m + mod(n, m) and n′ =

⌊n′

m

⌋×m + mod(n′, m)

we conclude that

n− n′ =⌊

nm

⌋×m−

⌊n′

m

⌋×m.

Hence, n− n′ = (⌊

nm

⌋−⌊

n′

m

⌋)×m and therefore n− n′ is a multiple of m.

Conversely, assume that m divides n − n′. Hence, n − n′ = q ×m for someq ∈ Z and therefore


n = q ×m + n′ = q ×m +⌊

n′

m

⌋×m + mod(n′, m)

that is, n = (q +⌊

n′

m

⌋) × m + mod(n′, m). Since 0 ≤ mod(n′, m) < m, by

Proposition 1.2.3, mod(n′, m) = mod(n, m).

3. We have that n − mod(n, m) =⌊

nm

⌋×m. As a consequence, since k divides

m, we also have

n−mod(n, m) =⌊

nm

⌋× q × k

for some q ∈ Z. Hence, k divides n − mod(n, m) and, by 2, we conclude thatmod(mod(n, m), k) = mod(n, k). QED

We can extend Definition 1.2.2 to the case where m < 0 and requiring that0 ≤ r < |m|. Clearly, Propositions 1.2.3 and 1.2.4 also extend to this case(Exercise 3 in Section 1.5).

1.2.2 Euclid’s algorithm

We now present the Euclid’s algorithm for computing the greatest common divisorof two integers as well as an extension of the algorithm.

Definition 1.2.5 Let m, n ∈ Z not simultaneously equal to 0. The greatest com-mon divisor of m and n, gcd(m, n), is the greatest integer that divides both mand n.

The case m = n = 0 is excluded because every integer divides 0 and thereforethere is no greatest integer that divides 0.

Clearly, gcd(m, n) = gcd(n, m) and gcd(m, n) is always a positive integer.The following properties of the greatest common divisor are useful in the sequel.

Proposition 1.2.6 Let m, n ∈ Z not simultaneously equal to 0.

1. gcd(0, m) = |m| for m 6= 0.

2. gcd(m, n) = gcd(−m, n) = gcd(m,−n) = gcd(−m,−n).

3. gcd(n, m) = gcd(mod(n, m), m).

Proof:1. It is easy to conclude that |m| is the largest element of the set k ∈ Z : k|mfor m 6= 0. Noting that k|0 for all k ∈ Z, we conclude that gcd(0, m) = |m| holds.

2. Since m = k × d if and only if −m = (−k)× d, for every d, k ∈ Z, we havethat

k ∈ Z : k|m = k ∈ Z : k|(−m)


Therefore, gcd(m, n) = gcd(−m, n). The other equalities also follow easily.

3. Assume n = q × m + mod(n, m) and let d|m, that is, m = k × d for somek ∈ Z. If d|mod(n, m), then

n = q × k × d + k′ × d = (q × k + k′)× d

for some k′ ∈ Z, that is, d|n. Conversely, if d|n, then k′′×d = q×k×d+mod(n, m)for some k′′ ∈ Z and therefore d|mod(n, m). We then conclude that

k ∈ Z : k|m and k|n = k ∈ Z : k|mod(n, m) and k|mTherefore, gcd(n, m) = gcd(mod(n, m), m). QED

The second statement of Proposition 1.2.6 shows that for computing the great-est common divisor we can concentrate only on nonnegative integers. The firstand third statements of Proposition 1.2.6 play a crucial role in the Euclid’s al-gorithm for computing the greatest common divisor. Clearly, one way of findinggcd(m, n), when m and n are both different from 0, consists of listing all the di-visors of n and all the divisors of m picking then the largest element included inboth lists. However, this is not efficient, even if we only list the positive divisors.The Euclid’s algorithm uses the results above to compute the greatest commondivisor of two nonnegative integers in a more efficient way.

The Euclid’s algorithm, or Euclidean algorithm, dates from around 300 BC,being included in the 7th book of Euclid’s Elements. It can be recursively asdescribed in Figure 1.3.

For m, n nonnegative integers not both equal to 0:

euclid(m, n) =

n if m = 0

euclid(mod(n, m), m) otherwise

Figure 1.3: Euclid’s algorithm

Example 1.2.7 Let us illustrate the Euclid’s algorithm by computing euclid(24, 18)

euclid(24, 18) = euclid( 18︸︷︷︸

mod(18,24)

, 24)

= euclid( 6︸︷︷︸

mod(24,18)

, 18)

= euclid( 0︸︷︷︸

mod(18,6)

, 6)

= 6


where it is clear the recursive nature of the algorithm. To compute euclid(24, 18)we can also make just a simple table including the recursion steps:

m n mod(n, m)

24 18 18

18 24 6

6 18 0

0 6 −

The construction ends whenever we get 0 in column “m”. The gcd(m, n) is thevalue in the last line of column“n”, since euclid(0, x) = x = gcd(0, x).

We now discuss the soundness of the algorithm.

Proposition 1.2.8 The Euclid’s algorithm is sound, that is, the equality

euclid(m, n) = gcd(m, n)

holds for nonnegative integers m and n.

Proof: Recall that mod(y, x) is an integer between 0 and x− 1. So, by applyingthe recursion step

euclid(x, y) = euclid(mod(y, x), x),

the first argument strictly decreases. As a consequence, the recursion step cannot be applied more than x times, and, at the end, the first argument necessarilybecomes 0. That is, when computing euclid(m, n), we get a finite number ofrecursive calls

euclid(x0, y0) = euclid(x1, y1) = . . . = euclid(xp, yp) (1.1)

where x ≥ p ≥ 0, (x0, y0) = (m, n), xp = 0 and for 1 ≤ i ≤ p

xi = mod(yi−1, xi−1) and yi = xi−1.

Therefore, by 1 of Proposition 1.2.6 we have

euclid(xp, yp) = euclid(0, yp) = yp = gcd(0, yp) = gcd(xp, yp)


and we conclude thateuclid(x0, y0) = gcd(xp, yp). (1.2)

On the other hand, using 3 of Proposition 1.2.6 and the fact that gcd(a, b) = gcd(b, a),we have

gcd(xi−1, yi−1) = gcd(xi, yi)

for all 1 ≤ i ≤ p, hencegcd(x0, y0) = gcd(xp, yp). (1.3)

From (1.2) and (1.3) we finally conclude that

euclid(m, n) = euclid(x0, y0) = gcd(x0, y0) = gcd(m, n).

QED

Example 1.2.9 From Example 1.2.7 we know that euclid(24, 18) = 6. We canthen conclude that gcd(24, 18) = 6.

The recursive function euclid in Figure 1.4 implements the Euclid’s algorithmin the obvious way (see Figure 1.3).

euclid=Function[m,n,If[m==0,

n,

euclid[Mod[n,m],m]]];

Figure 1.4: Euclid’s algorithm in Mathematica

The time complexity of Euclid’s algorithm is discussed in Section 6.3.2 ofChapter 6.

Extended Euclid’s algorithm

We start by stating an important property of the greatest common divisor.

Proposition 1.2.10 Let m, n ∈ Z not both equal to 0. Then gcd(n, m) is thesmallest positive number of the form a×m + b× n for a, b ∈ Z.

Proof: Let us first consider the case n = 0. Then, a×m + b× n becomes a×m.The smallest positive integer of this form is |m|, that is, gcd(n, m), recalling 1 ofProposition 1.2.6. The case m = 0 is clearly similar.


Assume now that m and n are not both equal to 0, and let S be the set of allpositive integers of the form a×m + b× n for integers a and b. Since m and n arenot both equal to 0, S is a nonempty set of positive integers and therefore it hasa least element x = a×m + b× n for some a and b. Let q and r be the quotientand remainder of the division of m by x, respectively. Then, m = q × x + r with0 ≤ r < x and therefore

r = m− q × (a×m + b× n) = (1− q × a)×m + (q × b)× n

If r > 0 then r ∈ S. But, since r < x, this contradict the fact that x is the leastelement of S. Hence, r = 0 and therefore x divides m. Reasoning in a similarway with respect to x and n we conclude that x divides n. Thus, x is a commondivisor of m and n. Let y ∈ Z be another common divisor of m and n. Then,

x = a×m + b× n = a× k × y + b× k′ × y = (a× k + b× k′)× y

for some k, k′ ∈ Z, and therefore y ≤ x. Hence, x = gcd(n, m). QED

It is possible to modify Euclid’s algorithm in order to obtain values a and bsuch that

gcd(m, n) = a×m + b× n.

This algorithm is named the extended Euclid’s algorithm and is recursively definedin Figure 1.5.

For m, n nonnegative integers not both equal to 0:

exteuclid(m, n) =

(0, 1) if m = 0

(b′ − a′ ×⌊

nm

⌋, a′) otherwise

where (a′, b′) = exteuclid(mod(n, m), m)

Figure 1.5: Extended Euclid’s algorithm

Let us illustrate the extended Euclid’s algorithm.

Example 1.2.11 Consider the case of exteuclid(24, 18). We can use a tablesimilar to the one presented in Example 1.2.9:


m n mod(n, m)⌊

nm

⌋a b

24 18 18 0 1 −1(= 1− (−1)×

⌊1824

⌋)

18 24 6 1 −1 1(= 0− 1×

⌊2418

⌋)

6 18 0 3 1 0(= 1− 0×

⌊186

⌋)

0 6 − − 0 1

We start by filling in the first three columns of each line, top down. Thiscorresponds to the table in Example 1.2.9. Once we get 0 in column “m”, wecan fill in the last line of columns “a” and “b” with 0 and 1, respectively, sinceexteuclid(m, n) = (0, 1). Afterwards, we fill in the other lines of these columns,bottom up, following the second equality of the algorithm, as indicated. Hence,

exteuclid(24, 18) = (1,−1)

Observe that, indeed, gcd(24, 18) = 6 = 1× 24 + (−1)× 18.

We can now compute exteuclid(15, 100), for instance, in a similar way:

m n mod(n, m)⌊

nm

⌋a b

15 100 10 6 7 −1

10 15 5 1 −1 1

5 10 0 2 1 0

0 5 − − 0 1

We conclude that exteuclid(15, 100) = (7,−1). Note again that gcd(15, 100) =5 = 7× 15 + (−1)× 100.

Next we discuss the soundness of the extended Euclid’s algorithm.


Proposition 1.2.12 The extended Euclid’s algorithm is sound, that is,

if exteuclid(m, n) = (a, b) then gcd(m, n) = a×m + b× n

holds for m, n nonnegative integers.

Proof: To compute the value of exteuclid(m, n), with m 6= 0, we start by comput-ing exteuclid(mod(n, m), m), and go on repeating this step until we are requiredto compute exteuclid(0, k) for some k. Hence, we are required to compute

exteuclid(xi, yi)

for 1 ≤ i ≤ p and p ≥ 0, where (x0, y0) = (m, n), xp = 0 and

(xi, yi) = (mod(yi−1, xi−1), xi−1)

for each 1 ≤ i ≤ p. Note that there are indeed finitely many of these pairs (xi, yi),that is, after a finite number of steps we get xp = 0 for some p, because thesepairs are exactly as in (1.1) above. In fact, as remarked therein,

yp = gcd(m, n) (1.4)

When we get xp = 0 then we get the pair (ap, bp) = (0, 1) corresponding toexteuclid(0, yp), that is, we conclude that

exteuclid(xp, yp) = (ap, bp) = (0, 1) (1.5)

Following the algorithm, we can then compute

exteuclid(xp−1, yp−1) = (ap−1, bp−1) = (bp − ap ×⌊

yp−1

xp−1

⌋

, ap)

and so on until exteuclid(x0, y0). More precisely, we have

exteuclid(xi−1, yi−1) = (ai−1, bi−1) = (bi − ai ×⌊

yi−1

xi−1

⌋

, ai)

for 1 ≤ i ≤ p, where (ai, bi) = exteuclid(xi, yi), and therefore

exteuclid(m, n) = exteuclid(x0, y0) = (a0, b0) (1.6)

Moreover, observe that, on one hand

u×mod(y, x) + v × x = u× (y −⌊

y

x

⌋× x) + v × x

= u× y − u×⌊

y

x

⌋× x + v × x

= (v − u×⌊

y

x

⌋)× x + u× y

for integers u, v, x, y with x > 0, and therefore, for 1 ≤ i ≤ p, we have


ai × xi + bi × yi = (bi − ai

⌊yi−1

xi−1

⌋

)× xi−1 + ai × yi−1

= ai−1 × xi−1 + bi−1 × yi−1

As a consequence,

ap × xp + bp × yp = a0 × x0 + b0 × y0

On the other hand, recalling (1.4) and (1.5)

gcd(m, n) = yp = 0× 0 + 1× yp = ap × xp + bp × yp

and thereforegcd(m, n) = a0 × x0 + b0 × y0

Recalling (1.6) we can finally conclude that, indeed, exteuclid(m, n) = (a, b) suchthat gcd(m, n) = a×m + b× n. QED

The recursive function exteuclid in Figure 1.6 implements the extendedEuclid’s algorithm following in Mathematica.

exteuclid=Function[m,n,If[m==0,

0,1,exteuclid[Mod[n,m],m][[2]]-exteuclid[Mod[n,m],m][[1]]Floor[n/m],

exteuclid[Mod[n,m],m][[1]]]];

Figure 1.6: Extended Euclid’s algorithm in Mathematica

1.2.3 Prime numbers

Herein we introduce prime numbers and the Fundamental Theorem of Arithmetic.We also refer to coprime numbers.

Definition 1.2.13 An integer number p is said to be prime whenever it isgreater than 1 and for any positive integer n if n divides p then n = 1 or n = p.

In Mathematica the predicate PrimeQ tests whether a number is prime.

Example 1.2.14 The first prime number is 2 and the following prime numbersless than 10 are 3, 5 and 7. Observe that 2 is the only even prime number.


The following proposition presents a result known as Euclid’s lemma. Thisresult is used in the proof of the Fundamental Theorem of Arithmetic below.

Proposition 1.2.15 Let p be a prime number and let m, n ∈ N0. If p dividesm× n then p divides m or p divides n.

Proof: Assume that p divides m× n but p does not divide m. We prove that-comO p divides n. Since p does not divide m then gcd(p, m) = 1. By Proposition1.2.10, there are a, b ∈ Z such that 1 = a× p + b×m. Multiplying both sides byn, we get

n = a× p× n + b×m× n

and therefore, given that p divides m× n,

n = a× p× n + b× k × p = (a× b + b× k)× p

for some k ∈ Z, that is, p divides n. QED

We now present the Fundamental Theorem of Arithmetic. This theoremasserts that each integer greater than 1 can be written in a unique way as aproduct of prime numbers. This is a very important and useful result since, as aconsequence, many problems in number theory can then be reduced to problemsabout prime numbers only.

Theorem 1.2.16 Every integer number n > 1 can be written as a product ofprime numbers, that is

n =

k∏

i=1

piei

where pi is prime and ei is a positive integer, for every 1 ≤ i ≤ k, and pi 6= pj

for all 1 ≤ i, j ≤ k, i 6= j. This factorization into prime numbers is unique apartfrom permutations of the factors.

Proof: We first prove that such a product exists and then that it is unique.

(1) Existence of factorization. Assume, by contradiction, that there are integersgreater than 1 that cannot be written as a product of primes as in the statementof the Theorem. Let n be the smallest of such integers. Thus, n is not a primenumber and therefore n = a× b for some integers a and b such that a, b > 1 anda, b < n. Since a, b < n, both a and b can be written as a product of primes. As aconsequence, n can also be written as a product of primes. This contradicts theassumption and allows us conclude that every integer greater than 1 can indeedbe written a product of primes.

(2) Uniqueness. Assume now that there are integers greater than 1 that havetwo distinct factorizations in prime numbers, and let n be the smallest of suchintegers. Then n can be written as


n =

s∏

i=1

pi and n =

t∏

i=1

qi

where pi and qj are prime numbers for 1 ≤ i ≤ s and 1 ≤ j ≤ t, and pi ≤ pi+1 andqi ≤ qi+1 for each 1 ≤ i < s and 1 ≤ j < t. Moreover, assuming without loss ofgenerality that s ≤ t, there is 1 ≤ i ≤ s such that pi 6= qi. Clearly, s, t > 1. Then,since p1 divides n, by Proposition 1.2.15, p1 divides q1 or p1 divides q2 × . . .× qt.On one hand, if p1 divides q1, then p1 = q1 since they are both prime. Thus,

n′ =

s∏

i=2

pi and n′ =

t∏

i=2

qi

that is, n′ < n has two distinct prime factorizations. But this contradicts theassumption of n being the smaller integer greater that 1 satisfying this property.On the other hand, since q2 × . . .× qt < n then q2 × . . .× qt has a unique primefactorization. Hence, if p1 divides q2 × . . .× qt then q2 × . . .× qt = k × p1 andtherefore p1 = qj for some 1 ≤ j < t. Removing p1 from the first factorization ofn and qj from the second, we again end up with 1 < n′ < n with two distinctprime factorizations thus contradicting once more the assumption regarding n.As a consequence, we conclude that every integer n > 1 has a unique prime fac-torization. QED

Example 1.2.17 The factorizations into primes of 15, 90 and 2205, for instance,are as follows:

• 15 = 31 × 51

• 90 = 21 × 32 × 51

• 2205 = 32 × 51 × 72

As we have already remarked in Section 1.1, factorization of integers intoprime numbers is computationally hard and, at present date, no polynomial-time(classical) algorithm is known.

Another important result about prime numbers is stated in the Theorem ofEuclid presented in the 9th book of Euclid’s Elements (Exercise 6 in Section 1.5).

We now present the notion of coprime numbers.

Definition 1.2.18 Two integers m and n are coprime, or relatively prime, when-ever gcd(m, n) = 1.

For instance, 18 and 35 are coprime, but 35 and 40 are not coprime sincegcd(35, 40) = 5. When m and n are coprime we also say that m is coprime to n.

We now prove some simple but useful facts about coprime numbers.


Proposition 1.2.19 Let m, n ∈ Z.

1. If m, n are prime numbers and n 6= m then m and n are coprime.

2. If m, n > 1 then m and n are coprime if and only if their factorizations intoprime numbers do not have any prime in common.

3. If n is prime then all the positive integers less than n are coprime to n.

Proof:1. If m is prime then the only positive divisors of m are 1 and m. Similarly, theonly positive divisors of n are 1 and n. Since n 6= m, gcd(m, n) = 1.

2. Assume m and n are coprime. Let m =∏k

i=1 piei and n =

∏k′

i=1 p′ie′i be the

(unique) factorizations of m and n. If pi = p′j for some 1 ≤ i ≤ k and 1 ≤ j ≤ k′

then pi both divides m and n. Since pi > 1, gcd(m, n) 6= 1 and therefore m andn are not coprime, contradicting the assumption.

Conversely, assume that the factorizations of n and m into prime numbersdo not have any prime in common. Hence, if r > 1 divides n then, using Theo-rem 1.2.16, we can conclude that the factorization of r only includes primes alsopresent in the factorization of n. Similarly, when r > 1 divides m, the factoriza-tion of r only includes primes also in the factorization of m. Hence, there is nor > 1 that both divides n and m. As a consequence, gcd(m, n) = 1.

3. Suppose that there is 0 < m < n such that gcd(m, n) > 1. Since gcd(m, n)divides n and n is prime, gcd(m, n) = n. But this cannot be the case, sincegcd(m, n) divides m and therefore gcd(m, n) ≤ m < n. We then conclude that if0 < m < n then, necessarily, gcd(m, n) = 1. QED

The Euler’s phi function associates with each positive integer n the numberof positive integers less than or equal to n that are coprime to n.

Definition 1.2.20 The Euler’s phi function, or just phi function, is the mapφ : N→ N such that φ(n) is the number of positive integers less than or equal ton that are coprime to n.

The Euler’s phi function is also known as the totient function.

Example 1.2.21 Let φ be the Euler’s phi function. Then,

• φ(1) = φ(2) = 1

• φ(3) = φ(4) = φ(6) = 2

• φ(5) = 4

The Euler’s phi function has several interesting properties.


Proposition 1.2.22 Let φ be the Euler’s phi function.

1. If p is prime and k is a positive integer then φ(pk) = pk − pk−1.

2. If n = p×q where p and q are primes then φ(n) = φ(p)φ(q) = (p−1)(q−1).

Proof:1. An integer r > 0 is coprime to pk if and only if r is not a multiple of p (Exercise4 in Section 1.5). Hence, φ(pk) is the number of elements in C = 1, 2, . . . , pkthat are not multiples of p. Observe that the set of multiples of p in C is C ′ =p, 2p, 3p . . . , zp where zp is necessarily pk and therefore z = pk−1. Thus, thatthere are exactly pk−1 multiples of p in C. Finally, φ(pk) is the number of elementsin C\C ′ that is pk − pk−1.

2. An integer r > 0 is coprime to n if and only if r is not a multiple of p andr is not a multiple of q (Exercise 5 in Section 1.5. Hence, φ(n) is the num-ber of such integers in C = 1, 2, . . . , n. The set of multiples of p in C isC ′ = p, 2p, 3p . . . , zp ⊆ C where zp is necessarily n. Since n = p× q, by Theo-rem 1.2.16, z = q and we conclude that there are q multiples of p in C. Similarly,we can conclude that the set of multiples of q in C is C ′′ = q, 2q, 3q . . . , pq ⊆ Cand therefore there are p multiples of q in C. It is an easy exercise to prove thatthere is only one element in C that it is both multiple of p and multiple of q:the integer p× q. Hence, φ(nk) is the number of elements in C\(C ′ ∪C ′′) that isn− q − p + 1 = p× q − q − p + 1 = (p− 1)(q − 1). QED

In Mathematica the function EulerPhi implements the Euler’s phi function.

1.3 Modular arithmetic

This section concentrates on modular arithmetic, that is, where the arithmeticoperations are defined modulo n. Modular arithmetic was first introduced by theGerman mathematician C. Gauss in 1801.

1.3.1 Congruence modulo n

Herein we introduce the congruence modulo n relation, for a positive integer n,and some of its properties.

Definition 1.3.1 Given a positive integer n, the congruence modulo n relationis the binary relation =n on Z such that

a=n b

whenever mod(a, n) = mod(b, n). Whenever a =n b we say that a and b arecongruent modulo n.

1.3. MODULAR ARITHMETIC 25

Let us present some examples.

Example 1.3.2 For instance,

• 5 =5 15 since mod(5, 5) = mod(15, 5) = 0;

• 13 =5 28 since mod(13, 5) = mod(28, 5) = 3;

• 11 6=5 32 since mod(11, 5) = 1 6= 2 = mod(32, 5).

It is straightforward to conclude that a =n mod(a, n) and that a =n b if andonly if a− b is a multiple of n (Exercise 9 in Section 1.5).

Given integers a and b and a positive integer n, the function congrMod inFigure 1.7 returns True if a and b are congruent modulo n and False otherwise.It uses the Mathematica function Mod to compute the integer remainder of thedivision of two integers.

congrMod=Function[a,b,n,Mod[a,n]==Mod[b,n]];

Figure 1.7: Congruence modulo n in Mathematica

The relation =n is an equivalence relation, that is, it is reflexive, symmetricand transitive (Exercise 8 in Section 1.5). The next result relates the congruencesmodulo m and n with the congruence modulo mn.

Proposition 1.3.3 Let a, b, m, n ∈ Z with m, n > 0. If m and n are coprime,then a =mn b if and only if a =m b and a =n b.

Proof:

(→) Assume a =mn b. Using 2 of Proposition 1.2.4, a− b = kmn for some k ∈ Z

and therefore both a =m b and a =n b.

(←) Assume that a =m b and a =n b. The result is straightforward if m or n isequal to 1. Using the reflexivity of =n, the result is also immediate if a = b.

(1) Let m, n > 1 and a > b. Then,

a− b = k′ ×r′∏

i=1

(p′i)e′i = k′′ ×

r′′∏

i=1

(p′′i )e′′i


for some k′, k′′ ∈ Z, where∏r′

i=1(p′i)

e′i and∏r′′

i=1(p′′i )

e′′i are the factorizations ofm and n into prime numbers, respectively. Since m and n are coprime, by 2 ofProposition 1.2.19, p′i 6= p′′j for all 1 ≤ i ≤ r′ and 1 ≤ j ≤ r′′. We have that

a− b = k′ ×r′∏

i=1

(p′i)e′i > 1

and so the factorization of a−b into prime numbers is unique. Thus, we concludethat

a− b = k ×r′∏

i=1

(p′i)e′i ×

r′′∏

i=1

(p′′i )e′′i

for some k ∈ Z. As a consequence, a− b = k ×m× n and therefore mn dividesa− b. Thus, a =mn b.

(2) Let m, n > 1 and a < b. We can conclude in a similar way that mn dividesb− a and therefore b =mn a. By the symmetry of =n we then get a =mn b. QED

The result stated in Proposition 1.3.3 easily extends to the case where twointegers are congruent modulo a product of several pairwise coprime positiveintegers (Exercise 10 in Section 1.5).

A congruence relation on a given set equipped with some operations is anequivalence relation on that set that preserves these operations. The relation=n is a congruence relation with respect to the usual operations of sum andmultiplication in Z. Recall that =n is an equivalence relation.

Proposition 1.3.4 The equivalence relation =n is a congruence with respect tothe usual operations of sum (+) and multiplication (×). That is:

(i) a + b =n a′ + b′ whenever a =n a′ and b =n b′;

(ii) a× b =n a′ × b′ whenever a =n a′ and b =n b′;

for any a, b, n ∈ Z and with n > 0.

Proof: Let a =n a′ and b =n b′. Hence, a− a′ = k1 × n and b− b′ = k2 × n forsome k1, k2 ∈ Z. Then, on one hand,

(a + b)− (a′ + b′) = (a− a′) + (b− b′) = k1 × n + k2 × n = (k1 + k2)× n

and therefore a + b =n a′ + b′, thus establishing (i). On the other hand, withrespect to (ii), we have

(a×b)−(a′×b′) = ((a′+k1×n)×(b′+k2×n))−(a′×b′) = (k1k2n+k1b′+k2a

′)×n


that is, a× b =n a′ × b′. QED

Note that if a =n b holds then the congruences a + c =n b + c and a× c =n b× calso hold.

Since =n is an equivalence relation, we can consider the equivalence class

[a]n = x ∈ Z : x =n ainduced by =n for each a ∈ Z. The equivalence class [a]n consists of all theintegers that have the same remainder as a when divided by n. The set of allthis equivalence classes is

Z/ =n

the quotient set of Z by =n. This set is often denoted by Zn. It is easy to concludethat Zn is a finite set with precisely n elements, the n distinct equivalence classes

[0]n, [1]n, . . ., [n− 1]n

The class [0]n consists of all the integers that have remainder 0 when divided byn, that is, all the integers of the form kn for some k ∈ Z. The class [1]n consists ofall the integers that have remainder 1 when divided by n, that is, all the integersof the form kn + 1 for some k ∈ Z. In the general case, given 0 ≤ r ≤ n− 1, theclass [r]n consists of all the integers that have remainder r when divided by n,that is, all the integers of the form kn + r for some k ∈ Z.

Observe that Z1 is a singular set. The only element of this set is the class [0]1and [0]1 = Z.

Given that =n is a congruence relation with respect to the usual sum andmultiplication of integer numbers, the binary operations +n and ×n on Zn.

[a]n +n [b]n = [a + b]n and [a]n ×n [b]n = [a× b]n

are well defined, that is, the classes [a]n and [b]n of two integers uniquely determinethe classes [a]n +n [b]n and [a]n×n [b]n. We can also consider the unary operation−n on Zn.

−n[a]n = [n− a]n

The set Zn equipped with this operations has several important algebraic prop-erties that are studied in Section 1.3.2.

When no confusion arises we can refer to Zn as the set

0, ..., n− 1for a simplified notation.

We end this section with some results involving modular congruences that willbe useful later on. We begin with the following theorem, known as the Euler’stheorem, that we do not prove. The interested reader is referred to [15].


Theorem 1.3.5 Let a and n be coprime positive integers. Then aφ(n) =n 1.

When an integer a is coprime to n then there always exists an integer b suchthat a× b =n 1.

Proposition 1.3.6 Let a, n ∈ Z with n > 0. There is b ∈ Z such that a× b =n 1if and only if a is coprime to n.

Proof:(→) If n = 1 then gcd(a, n) = 1 and therefore a and n are coprime. Assume thenthat n > 1 and that a × b =n 1 for some b ∈ Z. Hence, mod(a × b, n) = 1. By3 of Proposition 1.2.6 it holds the equality gcd(a× b, n) = gcd(mod(a× b, n), n),and therefore

gcd(a× b, n) = gcd(1, n) = 1

Since gcd(a, n) also divides a× b, if gcd(a, n) > 1 then gcd(a× b, n) cannot be 1,contradicting the above conclusion. Hence, since gcd(a, n) is a positive integer,it holds that gcd(a, n) = 1, and therefore a and n are coprime.

(←) Assume that a is coprime to n. If a ≥ 0 and exteuclid(a, n) = (b, b′) then

a× b + n× b′ = gcd(a, n) = 1.

Hence, 1−a× b = n× b′, that is, 1−a× b is a multiple of n. By 2 of Proposition1.2.4 we conclude that

mod(1, n) = mod(a× b, n)

and therefore a× b =n 1 holds. Let us consider a < 0. Observing that −a > 0and that gcd(a, n) = gcd(−a, n), we can reason as above and conclude that ifexteuclid((−a), n) = (b, b′) then (−a) × b =n 1. Since (−a) × b = a × (−b), itholds that a× (−b) =n 1. QED

Multiplicative orders modulo n and the Carmichael function will be usefullater on in chapter 2.

Proposition 1.3.7 Let a ∈ Z be coprime to n ∈ N. Then there is k ∈ N suchthat ak =n 1.

Proof: If a is coprime to n then, by Proposition 1.3.6, there is b ∈ Z such thata×b =n 1. We have that aj × bi =n aj−i for all i, j ∈ N0 such that j ≥ i (Exercise12 in Section 1.5).

Since mod(ak, n) ∈ 0, . . . , n− 1 for all k ∈ N0, there are k1, k2 ∈ N0 suchthat ak1 =n ak2 and k1 6= k2. Then

ak1 × bk2 =n ak2 × bk2 .


Hence, on one hand ak2 × bk2 =n 1 and, on the other hand, ak1 × bk2 =n ak1−k2

assuming that k1 > k2. As a consequence ak1−k2 =n 1. QED

Observe that if a is not coprime to n then the existence of k ∈ N such thatak =n 1 is not ensured. As an example note that mod(2k, 4) is either 2 or 0 forall k ∈ N (Exercise 11 in Section 1.5).

Definition 1.3.8 Let a ∈ Z be coprime to n ∈ N. The multiplicative order of amodulo n is the least k ∈ N such that ak =n 1. The integer a is a primitive elementmodulo n if there is no other b ∈ Z coprime to n such that the multiplicative orderof b modulo n is greater than the multiplicative order of a modulo n.

When there is no ambiguity we may omit the word multiplicative. Note thatif a and b are primitive elements modulo n then their orders modulo n are equal.

We now prove that the order of any coprime to n divides φ(n).

Proposition 1.3.9 Let a ∈ Z be coprime to n ∈ N. The order of a modulo ndivides φ(n).

Proof: The Euler’s theorem ensures that aφ(n) =n 1. Hence, the order of amodulo n is always less than or equal to φ(n). Assume, by contradiction, thatthe order of a modulo n is some k ≤ φ(n) that does not divide φ(n). Then ak =n 1and 0 < mod(φ(n), k) < k. Since

aφ(n) = ak⌊φ(n)k

⌋+mod(φ(n),k) = (ak)⌊φ(n)

k⌋ × amod(φ(n),k)

then1 =n amod(φ(n),k)

But the above congruence contradicts the assumption that k is the order modulon of a. We can then conclude that k divides φ(n). QED

Given that ak = mod(a, n)k for all k ∈ N0, the order of any a > n coprime ton is always less than or equal to the order of any a′ < n coprime to n. Hence,there is always a primitive number modulo n less than n when n > 1.

Example 1.3.10 Let us consider n = 5. The order of 4 modulo 5 is 2 since4 =5 4 and 42 =5 1, but 4 is not a primitive element modulo 5 since the order of3 modulo 5 is 4. In fact, 3 =5 3, 32 =5 9, 33 =5 2 and 34 =5 1. Noting that theorder of 2 modulo 5 is also 4, we conclude that 2 and 3 are primitive elementsmodulo 5.

The Carmichael function associates any positive integer n with the order ofthe primitive elements modulo n.


Definition 1.3.11 The Carmichael function λ : N→ N is such that λ(n) is theorder modulo n of the primitive elements modulo n for all for all n ∈ N.

Example 1.3.12 Recall Example 1.3.10. We have that λ(5) = 4.

In Mathematica the order of a modulo n and λ(n) can be computed using thefunctions MultiplicativeOrder and CarmichaelLambda, respectively.

The following notion of quadratic residue modulo n is also useful later on inChapter 2.

Definition 1.3.13 Let n be an integer greater than 2. The integer a is aquadratic residue modulo n if a is coprime to n and there is an integer x suchthat x2 =n a.

Example 1.3.14 Recall Example 1.3.10. Let us consider n = 5. Since

12 =5= 42 =5 1 and 22 =5= 32 =5 4

we can conclude that 1 and 4 are quadratic residue modulo 5, but, for instance,2 and 3 are not quadratic residue modulo 5.

1.3.2 The rings Zn

In this section we endow the sets Zn with some algebraic structure. For simplicity,we will consider Zn = 0, 1, ..., n− 1. Then the operations +n, ×n and −n onZn presented at the end of Section 1.3 become

a +n b = mod(a + b, n) a×n b = mod(a× b, n) −na = mod(n− a, n)

for each 0 ≤ a, b ≤ n − 1. In particular, −na is equal to n − a when a 6= 0and it is 0 otherwise. Clearly, a +n b =n a + b =n mod(a, n) + mod(b, n) anda×n b =n a× b =n mod(a, n)×mod(b, n). The operations +n, −n and ×n canbe implemented in Mathematica using the function Mod.

The set Zn equipped with the operations +n,×n, −n, and the element 0constitutes a ring as we shall see below. The notion of ring is useful herein andalso, later on, in Chapter 3.

Definition 1.3.15 A ring is a tuple A = (A, +, 0,−,×) where

• A is a set

• + : A2 → A and × : A2 → A are binary operations on A

• − : A→ A is a unary operation on A


• 0 ∈ A;

such that, for every a, a1, a2, a3 ∈ A

• a1 + (a2 + a3) = (a1 + a2) + a3 (associativity of +)

• a1 + a2 = a2 + a1 (commutativity of +)

• a + 0 = a (additive identity)

• a + (−a) = 0 (−a additive inverse of a)

• a1 × (a2 × a3) = (a1 × a2)× a3 (associativity of ×)

• a1 × (a2 + a3) = (a1 × a2) + (a1 × a3) (left distributivity of × over +)

• (a1 + a2)× a3 = (a1 × a3) + (a2 × a3) (right distributivity of × over +)

The set A is the carrier set of A.

It is straightforward to conclude that −(−a) = a for every element a of aring. Moreover, additive inverses are unique (Exercise 13 in Section 1.5).

Definition 1.3.16 Let A = (A, +, 0,−,×) be a ring. Then A is a commutativering if for every a1, a2 ∈ A

• a1 × a2 = a2 × a1 (commutativity of ×)

The ring A is unitary if there exists an element 1 ∈ A such that

• a× 1 = 1× a = a (multiplicative identity)

for every a ∈ A. When A is a unitary ring we can refer to the inverse with respectto ×. Then, b ∈ A is a multiplicative inverse of a ∈ A if a× b = b× a = 1.

Clearly, multiplicative inverses are unique (Exercise 13 in Section 1.5). Weuse a−1 to denote the multiplicative inverse a, whenever it exists.

A unitary commutative ring where every nonzero element has a multiplicativeinverse has a special name. Such algebraic structure is a field.

Definition 1.3.17 A field is a unitary commutative ring A = (A, +, 0,−,×)where the multiplicative unity is distinct from 0 and every a ∈ A\0 has amultiplicative inverse.

It is easy to conclude that (R, +, 0,−,×) is a field, whereas (Z, +, 0,−,×) isa unitary commutative ring but it is not a field (Exercise 15 in Section 1.5). Inthe following example we show that endowing Zn with the operation +n and ×n

we obtain a unitary commutative ring.


Example 1.3.18 Let n be a positive integer. Then

(Zn, +n, 0,−n,×n)

is a unitary commutative ring. If n > 1, 1 is the multiplicative identity. Whenn = 1 the identity is 0, the unique element of Z1. Let a, b, c ∈ Zn.

• Associativity of +n

Let us prove that a+n (b+n c) = (a+n b)+n c, that is, let us prove that thethe equality mod(a + mod(b + c, n), n) = mod(mod(a + b, n) + c, n) holds.Since a + b =n mod(a + b, n) and b + c =n mod(b + c, n) then

a + (b + c) =n a + mod(b + c, n)

and

(a + b) + c =n mod(a + b, n) + c

Using the associativity of + and the transitivity of =n, it is straightforwardto conclude that mod(a + mod(b + c, n), n) = mod(mod(a + b, n) + c, n)holds.

• Commutativity of +n

We have a +n b = mod(a + b, n) = mod(b + a, n) = b +n a, using thecommutativity of +

• 0 additive identity

We have a +n 0 = mod(a + 0, n) = mod(a, n) = a, using the fact that 0 isthe additive identity in the ring of integers and Proposition 1.2.4.

• Left distributivity of ×n over +n

Let us prove the equality a×n (b +n c) = (a×n b) +n (a×n c), that is, thatmod(a×mod(b + c, n), n) = mod(mod(a× b, n) + mod(a× c, n), n) holds.The congruence b + c =n mod(b + c, n) holds as well as the congruencesa× b =n mod(a× b, n) and a× c =n mod(a× c, n). Hence

a× (b + c) =n a×mod(b + c, n)

and

(a× b) + (a× c) =n mod(a× b, n) + mod(a× c, n)

From the distributivity of × over + and the transitivity of =n it easilyfollows mod(a×mod(b + c, n), n) = mod(mod(a× b, n) + mod(a× c, n), n).


• 1 multiplicative identity, for n > 1

We have a ×n 1 = mod(a × 1, n) = mod(a, n) = a, using the fact that 1is the multiplicative identity in the ring of integers and Proposition 1.2.4.Clearly, the case 1×n a = a is similar.

The proofs of the associativity and commutativity of×n are similar to the ones for+n. Right distributivity follows from the left distributivity and the commutativityof +n and ×n. Proving that −na is the additive inverse of a is also easy and it isleft as an exercise to the reader.

For simplicity, when considering the ring (Zn, +n, 0,−n,×n) we often just referto the ring Zn. A element a ∈ Zn with multiplicative inverse is also said to be aunit of Zn and the corresponding inverse is denoted by a−n1, or simply a−1. Thereference to n can be omitted if no ambiguity arises. Similarly, also the referenceto n can be omitted in the additive inverse −na.

Example 1.3.19 Recall that Z5 = 0, 1, 2, 3, 4. In the ring (Z5, +5, 0,−5,×5):

• −52, the additive inverse of 2, is 3


• 1 has multiplicative inverse and 1−51 is 1 since 1×5 1 = 1

• 2 has multiplicative inverse and 2−51 is 3 since 2×5 3 = mod(2× 3, 5) = 1


Clearly, 3 also has multiplicative inverse and 3−51 is 2. Only 0 has no multiplica-tive inverse. Hence, (Z5, +5, 0,−5,×5) is also a field.

Example 1.3.20 Recall that Z6 = 0, 1, 2, 3, 4, 5. In the ring (Z6, +6, 0,−6,×6):



• 1 has multiplicative inverse and 1−61 is 1 since 1×6 1 = 1


Only 1 and 5 have multiplicative inverses. Hence, the ring (Z6, +6, 0,−6×6) isnot a field.

The elements of Zn that have multiplicative inverse are precisely the elementsof Zn that are coprime to n. These is a consequence of Proposition 1.3.6.


Proposition 1.3.21 Let n be a positive integer. Then a ∈ Zn has multiplicativeinverse if and only if a is coprime to n.

Proof:(→) By Proposition 1.3.6.

(←) Let a ∈ Zn be coprime to n. By Proposition 1.3.6. there is b ∈ Z such thata × b =n 1. Note that 0 ≤ b ≤ n− 1 may not hold. But 0 ≤ mod(b, n) ≤ n− 1holds. Given that b =n mod(b, n), then a × b =n a × mod(b, n) and thereforea×mod(b, n) =n 1. Thus, mod(a×mod(b, n), n) = mod(1, n), that is,

a×n mod(b, n) = 1

since 1 ≤ n. The multiplicative inverse of a is then mod(b, n). QED

The proofs of Propositions 1.3.6 and 1.3.21 suggest an algorithm for com-puting inverses in Zn using the extended Euclid’s algorithm. If a ∈ Zn hasmultiplicative inverse in Zn and exteuclid(a, n) = (c, d) then mod(c, n) is thatmultiplicative inverse (Exercise 17a in Section 1.5).

Example 1.3.22 Consider the ring Z61. Since 61 is prime all nonzero elementsof Z61 have multiplicative inverse. Let us compute the multiplicative inverse of16, for instance. Since

exteuclid(16, 61) = (−19, 5)

the multiplicative inverse of 16 in Z61 is mod(−19, 61) = 42. We can also get themultiplicative inverse of 16 in Z61 looking at −19 as the additive inverse of 19 inZ61, that is, 42 (= 61− 19).

A simple corollary of Proposition 1.3.21 states that if n is prime all the ele-ments of Zn apart from 0 have multiplicative inverse, that is, if n is prime thenZn is a field.

Corollary 1.3.23 Let n be a prime number. Then Zn is a field.

Proof: If a ∈ Zn\0 then 0 < a < n. By 3 of Proposition 1.2.19 and Proposi-tion 1.3.21 a has multiplicative inverse in Zn. QED

In some situations it is useful to consider an extension of the notion of multi-plicative inverse in Zn. Given a positive integer n and a, b ∈ Z, we say that b isa multiplicative inverse of a modulo n whenever

a× b =n 1.


By Proposition 1.3.6, such integer b exists if and only if a and n are coprimeand from its proof it follows that the extended Euclid’s algorithm computes amultiplicative inverse of a modulo n.

As expected, multiplicative inverses modulo n are not unique: if the integerb is a multiplicative inverse of a modulo n then the integer c is a multiplicativeinverse of a modulo n if and only if c =n b. Furthermore, if a is coprime to nthen the multiplicative inverse of mod(a, n) in Zn is a multiplicative inverse of amodulo n (Exercise 18 in Section 1.5). As an example let us consider a = 20 andn = 9. Since mod(20, 9) = 2 and 2−91 = 5 then 5 is a multiplicative inverse of 20modulo 9.

We can use a−n1 to denote a multiplicative inverse of a modulo n.

To end this section we introduce the notions of ring product and ring homo-morphism. A ring product is a binary operation that takes to rings and returnsthe product of the rings.

Definition 1.3.24 Let A′ = (A′, +′, 0′,−′,×′) and A′′ = (A′′, +′′, 0′′,−′′,×′′) betwo rings. The product of A′ and A′′, denoted by A′ ×A′′, is the ring

(A, +, 0,−,×)

where

• A = A′ ×A′′, that is, A is the Cartesian product of the carrier sets of eachring;

• (a′, a′′) + (b′, b′′) = (a′ +′ b′, a′′ +′′ b′′);

• 0 = (0′, 0′′);

• −(a′, a′′) = (−′a′,−′′a′′);

• (a′, a′′)× (b′, b′′) = (a′ ×′ b′, a′′ ×′′ b′′);

for all a′ ∈ A′ and a′′ ∈ A′′.

The product of rings is well defined since A′ ×A′′ is indeed a ring (Exercise20 in Section 1.5)

Example 1.3.25 Consider the rings (Z2, +2, 0,−2,×2) and (Z3, +3, 0,−3,×3).Their product is the ring (Z2 × Z3, +, (0, 0),−,×) where

• Z2 × Z3 = (0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2);

• (a, b) + (a′, b′) = (a +2 a′, b +3 b′);

• −(a, b) = (−2a,−3b);


• (a, b)× (a′, b′) = (a×2 a′, b×3 b′).

It is important to relate rings to each other. Such relationship is called ringhomomorphism. It is a map between the carrier sets that preserve the operations.

Definition 1.3.26 Let A = (A, +, 0,−,×) and A′ = (A′, +′, 0′,−′,×′) be tworings. An homomorphism from A to A′, denote by

h : A → A′

is a map h : A→ A′ such that

• h(a + b) = h(a) +′ h(b)

• h(0) = 0′

• h(−a) = −′h(a)

• h(a× b) = h(a)×′ h(b)

for every a, b ∈ A. An homomorphism h : A → A′ is an isomorphism wheneverh is a bijection.

Example 1.3.27 Consider the rings (Z2, +2, 0,−2,×2) and (Z3, +3, 0,−3,×3)and their product (Z2 × Z3, +, (0, 0),−,×). The map

h : Z2 × Z3 → Z3

such thath((a, b)) = b

is an homomorphism from (Z2×Z3, +, (0, 0),−,×) to (Z3, +3, 0,−3,×3). In fact

• h((a, b) + (a′, b′)) = h((a +2 a′, b +3 b′)) = b +3 b′ = h((a, b)) +′3 h((a′, b′))

• h((0, 0) = 0

• h(−(a, b)) = h((−2a,−3b)) = −3b = −3h((a, b))

• h((a, b)× (a′, b′)) = h((a×2 a′, b×3 b′)) = b×3 b′ = h((a, b))×′3 h((a′, b′))

for every (a, b), (a, b′) ∈ Z2 × Z3. This homomorphism is not an isomorphismsince h is not injective, hence not bijective.

Example 1.3.28 Consider the ring (Z, +, 0,−,×). The map h : Z → Z suchthat h(z) = −z is not an homomorphism from (Z, +, 0,−,×) to (Z, +, 0,−,×).In fact, h(a× b) = −(a× b) 6= (−a)× (−b) = h(a)× h(b).

In Exercise 22 in Section 1.5, the reader is asked to present an homomorphismfrom the ring (Z, +, 0,−,×) to the ring (R, +, 0,−,×).


1.3.3 The Chinese Remainder Theorem

Herein we present the Chinese Remainder Theorem. This theorem is nearly 2000years old and was established by Chinese scholars. In particular, this result isuseful for solving some systems of linear congruences.

Let r be a positive integer and let n1, . . . , nr be positive integers pairwisecoprime. Consider the map

h : Zn1×...×nr→ Zn1 × . . .× Znr

(1.7)

such that

h(x) = (mod(x, n1), . . . , mod(x, nr)) for each x ∈ Zn1×...×nr

This map is a ring homomorphism between the rings Zn1×...×nrand Zn1×. . .×Znr

(Exercise 26 in Section 1.5). We now state the Chinese Remainder Theorem inProposition 1.3.29.

Proposition 1.3.29 The map (1.7) is an isomorphism.

Proof: As we have stated above, h is a ring homomorphism. Hence, we onlyhave to prove that h is injective and surjective.

(1) h is surjective. We have to prove that given (x1, . . . , xr) ∈ Zn1 × . . .× Znr

there exists x ∈ Zn1×...×nrsuch that h(x) = (x1, . . . , xr).

Let

N = n1 × . . .× nr and Ni =N

ni

for all 1 ≤ i ≤ r

For each 1 ≤ i ≤ r, Ni is the product of all those nj with j 6= i. Hence, sinceni is coprime to all such nj , by 2 of Proposition 1.2.19 and Theorem 1.2.16, wecan conclude that Ni is coprime to ni. Therefore, by Proposition 1.3.6, Ni has amultiplicative inverse modulo ni.

Let us then consider

x = mod

(r∑

i=1

(xi ×N−ni

1

i ×Ni), N

)

∈ Zn1×...×nr

where N−ni

1

i is a multiplicative inverse of Ni modulo ni. One one hand,

xi ×N−ni

1

i ×Ni =nixi for every xi ∈ Zni

(1.8)

and, on the other hand,

xi ×N−ni

1

i ×Ni =nj0 for every xi ∈ Zni

and 1 ≤ j ≤ r, j 6= i (1.9)


given that nj divides Ni for each j 6= i.

We have to prove thatxj = mod(x, nj)

for each 1 ≤ j ≤ r. By 3 of Proposition 1.2.4,

x =nj

r∑

i=1

(xi ×N−ni

1

i ×Ni).

Given that mod(xi×N−ni

1

i ×Ni, nj) =njxi×N

−ni1

i ×Ni and using the congruenceproperties of the modular equality, we can conclude that

x =nj

r∑

i=1

mod(xi ×N−ni

1

i ×Ni, nj).

By (1.9), in the above summation∑r

i=1 mod(xi ×N−ni

1

i ×Ni, nj) only the term

mod(xj ×N−ni

1

j ×Nj , nj) is not necessarily equal to 0. Thus,

x =njmod(xj ×N

−ni1

j ×Nj , nj).

Finally, using (1.8) and Proposition 1.2.4, we conclude that

x =njxj

that is, xj = mod(x, nj) as intended. Hence, h is surjective.

(2) h is injective. Given positive integers n1, . . . , nr pairwise coprime, we havethat a =n1×...×nr

b if and only if a =nib for all 1 ≤ i ≤ r (Exercise 10 in Section

1.5).Let x, y ∈ ZN such that h(x) = h(y), where N = n1 × . . .× nr. Then

mod(x, ni) = mod(y, ni)

for all 1 ≤ i ≤ r and therefore, using the above result,

mod(x, N) = mod(y, N).

Hence, since x, y ∈ ZN , we have x = mod(x, N) = mod(y, N) = y . QED

Note that the injectivity of (1.7) can also be proved observing that the setsZn1×...×nr

and Zn1 × . . .× Znrare finite and have the same number of elements.

Therefore, h is injective because every surjective map between finite sets withsame cardinality is necessarily injective. Hence, we can proof the Chinese Remain-der Theorem without using the result stated in Exercise 10. Moreover, we canuse the Chinese Remainder Theorem to prove this result. Let us briefly see how.


Consider positive integers n1, . . . , nr pairwise coprime, and N = n1 × . . .× nr.Let x, y ∈ Z such that

x =niy

for all 1 ≤ i ≤ r. Hence, by 3 of Proposition 1.2.4,

x′ =niy′

for all 1 ≤ i ≤ r where x′ = mod(x, N) ∈ ZN and y′ = mod(y, N) ∈ ZN . Then,using the Chinese Remainder Theorem, h(x′) = h(y′), that is, since h is injective,x′ = y′. Therefore,

x =N y.

Conversely, if x =N y then, using 2 of Proposition 1.2.4, x− y is a multiple of Nand therefore, for all 1 ≤ i ≤ r, x− y is also a multiple of ni, that is, x =ni

y.

The following corollary shows how to use the Chinese Remainder Theoremfor solving some systems of linear congruences.

Corollary 1.3.30 Consider the system of r > 1 congruences

x =n1 k1

. . .

x =nrkr

where k1, . . . , kr ∈ Z and n1, . . . , nr are positive integers pairwise coprime. Thesystem has a unique solution modulo N = n1 × . . .× nr, since

s = mod

(r∑

i=1

(ki ×N−ni

1

i ×Ni), N

)

where Ni = Nni

for each 1 ≤ i ≤ r, is a solution of the system, and x ∈ Z is asolution of the system if and only if x =N s.

Proof: Let S be the given system and consider the following system S ′

x =n1 k′1

. . .

x =nrk′

r

where k′i = mod(ki, ni) for all 1 ≤ i ≤ r. Since, k′

i =niki for all 1 ≤ i ≤ r, by

the congruence properties it is easy to conclude that given x ∈ Z, x is a solutionof S if and only if x is a solution of S ′. Hence, from now on we only refer to thesystem S ′.


Given (k′1, . . . , k

′r) ∈ Zn1 × . . .× Znr

, by the Chinese Remainder Theorem thereexists s′ ∈ ZN such that

h(s′) = (mod(s′, n1), . . . , mod(s′, nr)) = (k′1, . . . , k

′r)

that is, there exists a s′ ∈ ZN such that

s′ =nik′

i for all 1 ≤ i ≤ r (1.10)

Recalling the proof of the Chinese Remainder Theorem, we know that the mod-ular equations (1.10) hold for

s′ = mod

(r∑

i=1

(k′i ×N

−ni1

i ×Ni), N

)

.

We now prove that given x ∈ Z then x =N s′ if and only if x is a solution ofS ′.

(→) If x =N s′ then, by 2 of Proposition 1.2.4, x− s′ is a multiple of N , thusa multiple of ni for all 1 ≤ i ≤ r, and therefore x =ni

s′ for all 1 ≤ i ≤ r. Hence,using (1.10), x =ni

k′i for all 1 ≤ i ≤ r, that is, x is a solution of S ′.

(←) If x is a solution of S ′, then x =nik′

i for all 1 ≤ i ≤ r. Consideringx′ = mod(x, N), we have x =N x′ and, using again Proposition 1.2.4, x′ =ni

x forall 1 ≤ i ≤ r. Therefore, x′ =ni

k′i for all 1 ≤ i ≤ r. Given that x′ ∈ ZN , from the

above unicity of s′ we conclude that s′ = x′. Hence, x =N s′.

We finally prove that given x ∈ Z then x =N s if and only if x is a solutionof S ′. To prove this equivalence, we only have to prove that s =N s′.

Reasoning as in the proof of the Chinese Remainder Theorem, we can concludethat for each 1 ≤ j ≤ r

s =njmod

(r∑

i=1

(ki ×N−ni

1

i ×Ni), N

)

=njkj

and

s′ =njmod

(r∑

i=1

(k′i ×N

−ni1

i ×Ni), N

)

=njk′

j

Hence, s =njkj and s′ =nj

k′j for all 1 ≤ j ≤ r. Given that k′

j =njkj for all

1 ≤ i ≤ r, we conclude that s =njs′ for all 1 ≤ i ≤ r. Then, using the result

stated in the Exercise 10 in Section 1.5, s =N s′ as intended. QED

Example 1.3.31 Consider the following system of congruences (or modular equa-tions)


13x + 1 =7 4−4x− 2 =9 −5

We want to find all the integer solutions of this system using the Chinese Re-mainder Theorem (Corollary 1.3.30).

(i) We first transform the given system into an equivalent one where eachcongruence ax + b =k c is replaced by a congruence of the form x =k c′.

Let us first consider the congruence 13x + 1 =7 4. Since −1 =7 −1, byProposition 1.3.4,

13x + 1− 1 =7 4− 1

that is13x =7 3

Given that 13 =7 6 then 13x =7 6x, using the congruence properties of =7.Hence,

6x =7 3

Since 6 is coprime to 7, by Proposition 1.3.21, it has multiplicative inversein Z7. We can use its inverse, 6−71, to obtain an equivalent congruence inthe intended form, taking again into account the congruence properties of=7:

6−71 × 6x =7 6−71 × 3

that is1× x =7 6−71 × 3

and, given that 6−71 = 6 since 6× 6 = 36 =7 1, we finally obtain

x =7 18

Considering now the second congruence and reasoning in a similar way wehave

−4x− 2 =9 −5 ⇔ 4x− 2 + 2 =9 −5 + 2

⇔ −4x =9 −3

⇔ 4x =9 3

⇔ 4−91 × 4x =9 4−91 × 3

⇔ x =9 7× 3

⇔ x =9 21

Recall that inverses in Zn can be computed using the extended Euclid’salgorithm.

Hence, we now have the system


x =7 18x =9 21

Although we could already use Corollary 1.3.30 to find the solutions, we canfurther simplify this system observing that 18 =7 4 and 21 =9 3. Hence, weget the equivalent system

x =7 4x =9 3

(ii) Since 7 and 9 are coprime we can use Corollary 1.3.30. Assuming n1 = 7and n2 = 9, then

k1 = 4 k2 = 3 N = 7× 9 = 63 N1 = 637

= 9 N2 = 639

= 7

Hence, given that N−711 = 4 and N−91

2 = 4 we have

s = mod(4× 9× 4 + 3× 7× 4, 63) = mod(144 + 84, 63) = 39

(iii) Finally, the solutions of the system are the integers x such that x =63 39.

This technique for solving systems of congruences can be extended to caseswhere the ni’s are not necessarily pairwise coprime. We refer the interested readerto, for instance, [15].

Given a positive integer n, a list w = n1, . . . , nr of positive integers and a in Zn,the function isoCRT in Figure 1.8 returns the image of a by the isomorphism ofthe Chinese Remainder Theorem. If n 6= n1× . . .×nr or the ni’s are not pairwisecoprime, isoCRT returns an error message. It first checks if elements of w arepairwise coprime using the Mathematica function GCD. Then it computes eachcomponent of the image of a.

1.4 RSA revisited

At the light of the results presented in this chapter we show that the RSA cryp-tosystem, described in Section 1.1 is sound, that is, we prove that when Bobdecrypts the ciphered text sent by Alice he obtains the original message.

Recall the RSA cryptosystem in Figure 1.9 where φ(n) = (p− 1)(q − 1) (seeProposition 1.2.22).

We now prove that v(n,b) u(n,a) = idZn. The proof applies Euler’s theorem

and the Chinese Remainder Theorem (Proposition 1.3.29).

1.4. RSA REVISITED 43

isoCRT=Function[n,w,a,Module[i,j,coprime,If[Apply[Times,w]!=n,

Print["Erro"],

coprime=True;

i=1;

While[i<Length[w]&&coprime,

j=i+1;

While[j<=Length[w]&&coprime,

coprime=(GCD[w[[i]],w[[j]]]==1);

j=j+1];

i=i+1];

If[!coprime,

Print["Error"],

Table[Mod[a,w[[k]]],k,1,Length[w]]]]]];

Figure 1.8: Chinese remainder theorem in Mathematica

Let n, p, q, a and b be natural numbers such that

n = p× q where p and q are prime numbers

mod(a× b, φ(n)) = 1

Message space: Zn = 0, 1, 2, . . . , n− 1

Public key: (n, a)

u(n,a)(x) = mod(xa, n)

Private key: (n, b)

v(n,b)(y) = mod(yb, n)

Figure 1.9: RSA cryptosystem


Proposition 1.4.1 Consider the RSA cryptosystem with public key (n, a) andprivate key (n, b). Then v(n,b) u(n,a) = idZn

.

Proof: Recall that n = p × q where p and q are distinct prime numbers andtherefore, by 1 of Proposition 1.2.19, p and q are coprime. Let h be the mappresented in (1.7) at Section 1.3.3.

(1) We first prove that h(v(n,b)(u(n,a)(x))) = (mod(xab, p), mod(xab, q)) for eachx ∈ Zn. Since xa =n mod(xa, n) it is straightforward to conclude that

xab =n (mod(xa, n))b

using the congruence properties of =n (Exercise 9 in Section 1.5). Using the factthat u(n,a)(x) = mod(xa, n), we have

v(n,b)(u(n,a)(x)) = mod((u(n,a)(x))b, n) = mod(xab, n).

Therefore, by 3 of Proposition 1.2.4, we obtain

mod(v(n,b)(u(n,a)(x))), p) = mod(mod(xab, n), p) = mod(xab, p).

Similarly, by replacing p with q, we get

mod(v(n,b)(u(n,a)(x))), q) = mod(xab, q).

Then, we conclude that

h(v(n,b)(u(n,a)(x)))) = (mod(xab, p), mod(xab, q)). (1.11)

(2) We now prove that h(x) = (mod(xab, p), mod(xab, q)) for each x ∈ Zn. We

start by showing that mod(x, p) = mod(xab, p). If p divides x then p also dividesxab and so

mod(x, p) = 0 = mod(xab, p).

If p does not divide x, since p is prime, x and p are coprime. Hence, by theEuler’s theorem (Theorem 1.3.5), xφ(p) =p 1. Using 1 of Proposition 1.2.22, weconclude that

xp−1 =p 1. (1.12)

By definition of RSA cryptosystem (see Figure 1.9) we have that mod(ab, φ(n)) =1, and therefore

ab = kφ(n) + 1

for some k ∈ Z. From (1.12), using the congruence properties of =p, we get

(xp−1)k(q−1) =p 1

and thenxxk(p−1)(q−1) =p x.

1.4. RSA REVISITED 45

Using 2 of Proposition 1.2.22, we conclude that

xxk(p−1)(q−1) = xxkφ(n) = xab

and therefore x =p xab, that is,

mod(x, p) = mod(xab, p).

The proof of the equality mod(xab, q) = mod(x, q) is similar. Hence,

h(x) = (mod(xab, p), mod(xab, q))

for each x ∈ Zn.

(3) From (1) and (2), we conclude that

h(v(n,b)(u(n,a)(x))) = h(x)

and thereforev(n,b)(u(n,a)(x)) = x

since, by Proposition 1.3.29, h is injective. QED

We now present an example that illustrates the RSA encryption and decryp-tion of messages.

Example 1.4.2 Consider the RSA cryptosystem and, just for illustration pur-poses, let us assume that Bob has chosen the primes

p = 13 and q = 7.

In practice, to ensure that n can not be easily factorized, p and q should bevery large primes (with a few hundreds of digits). Given p and q, Bob can nowcompute n and φ(n):

n = 13× 7 = 91 and φ(91) = 12× 6 = 72

To choose the exponents a and b, Bob first picks up an element a in Z72 thathas multiplicative inverse, that is, a coprime to 72. Let us consider a = 5. Then,the extended Euclid’s algorithm can be used to compute its inverse b:

x y mod(y, x)⌊

y

x

⌋c d

5 72 2 14 29 −2

2 5 1 2 −2 1

1 2 0 2 1 0

0 1 − − 0 1


Given that exteuclid(a, φ(n)) = exteuclid(5, 72) = (29,−2), then b = 29. Hence,Bob’s public key is (91, 5) and his private key is (91, 29).

Assume that Alice wants to send the message 2 ∈ Z91 to Bob. Then she usesBob public key and the corresponding encryption rule and obtains the encryptedmessage

u(91,5)(2) = mod(25, 91) = 32

that she sends through the channel.When Bob receives the encrypted message 32 he decrypts it using the decryp-

tion rule associated with his private key, that is,

v(91,29)(32) = mod(3229, 91)

= mod(32× (322)14, 91)

= mod(32× 102414, 91)

= mod(32× 2314, 91)

= mod(32× (232)7, 91)

= mod(32× 5297, 91)

= mod(32× 747, 91)

= mod(32× 74× (742)3, 91)

= mod(32× 74× 54763, 91)

= mod(32× 74× 163, 91)

= mod(32× 74× 16× 162, 91)

= mod(32× 74× 16× 74, 91)

= mod(32× 16× 16, 91)

= mod(32× 74, 91)

= mod(2, 91) = 2

As expected, Bob gets the original message 2. Note that we repeatedly computesquares and reduced it to elements in Z91.

1.5 Exercises

1. Using the Euclid’s algorithm compute

(a) gcd(32, 63)

1.5. EXERCISES 47

(b) gcd(81, 22)

(c) gcd(105, 72)

2. Apply the extended Euclid’s algorithm to

(a) 32 and 63

(b) 81 and 22

(c) 105 and 72

3. Definition 1.2.2 can be extended to the case where m < 0 by requiring that0 ≤ r < |m|. Prove the extensions of Propositions 1.2.3 and 1.2.4 to thiscase.

4. Let p be a prime number and k a positive integer. Prove that a positiveinteger r is coprime to pk if and only if r is not a multiple of p.

5. Let p and q be two prime numbers. Prove that a positive integer r is coprimeto p× q if and only if r is not a multiple of p and r is not a multiple of q.

6. Prove the Euclid’s Theorem: there are infinitely many prime numbers.Hint: assume that all primes are smaller than n and find a contradictionwith the prime factorization of n! + 1

7. Prove that if n = p× q where p and q are coprime then φ(n) = φ(p)φ(q).

8. Let n be a positive integer. Prove that the relation =n is an equivalencerelation, that is, it fulfills the following properties, where a, b, c ∈ Z:

(i) a =n a (reflexivity);

(ii) if a =n b then b =n a (symmetry);

(iii) if a =n b and b =n c then a =n c (transitivity).

9. Let a, b, n ∈ Z with n > 0. Prove that

(a) a =n mod(a, n).

(b) a =n b if and only if a− b is a multiple of n.

(c) if a =n b then ak =n bk, for all nonnegative integer k.

10. Let n1, . . . , nr be positive integers pairwise coprime. Prove that a =n1×...×nrb

if and only if a =nib for all 1 ≤ i ≤ r.

11. Prove that mod(2k, 4) is either 2 or 0 for all k ∈ N.

12. Let a, b, n ∈ Z with n > 0. Prove that if b is a multiplicative inverse of athen aj × bi =n aj−i for all i, j ∈ N0 such that j ≥ i.


13. Let (A, +, 0,−,×) be a ring and a, b, c ∈ A. Prove that

(a) if a + b = 0 then b = −a (additive inverses are unique).

(b) c + a = c + b if and only if a = b.

(c) −(−a) = a.

(d) if a × b = 1 and a × c = 1 then b = c (multiplicative inverses areunique).

(e) (a−1)−1 = a.

(f) 0× a = a× 0 = 0.

(g) (−a)× b = −(a× b).

(h) if there is y ∈ A such that y + x = x + y = 0 for all x ∈ A then y = 0.

14. Prove that there exists a ring (A, +, 0,−,×) and a, b, c ∈ A such that a 6= 0,a× b = a× c and b 6= c.

15. Prove that

(a) (Z, +, 0,−,×) is a unitary commutative ring but it is not a field.

(b) (R, +, 0,−,×) is a field.

16. Let (A, +, 0,−,×) be a field. Prove that if a 6= 0 and a × b = a × c thenb = c for all a, b, c ∈ A.

17. Let n be a positive integer and let a ∈ Zn have multiplicative inverse in Zn.

(a) Prove that if exteuclid(a, n) = (c, d) then mod(c, n) is the multiplica-tive inverse of a in Zn.

(b) Compute the multiplicative inverses of 3 and 5 in Z13.

(c) Compute the multiplicative inverses of 18 and 22 in Z35.

(d) Compute the multiplicative inverses of 32 and 45 in Z63.

18. Let a, b, c, n ∈ Z such that n > 0, a is coprime to n and b is a multiplicativeinverse of a modulo n. Prove that

(a) c is a multiplicative inverse of a modulo n if and only if c =n b.

(b) mod(a, n)−n1 is a multiplicative inverse of a modulo n.

19. Let n be a positive integer and I = a ∈ Zn : a is coprime to n. Provethat f(a) = a−n1 is a well defined map from I to I. Prove that f is abijection and use this fact to conclude that

∏

a∈I a2 =n 1.

20. Let A′ and A′′ be two rings. Prove that A′ ×A′′ is a ring.

1.5. EXERCISES 49

21. Prove that h : Z→ Zn such that h(a) = mod(a, n) is a ring homomorphism.

22. Present a ring homomorphism from (Z, +, 0,−,×) to (R, +, 0,−,×).

23. Let h : A → A′ be a ring homomorphism. Prove that

(a) h(an) = h(a)n for every a in the carrier set and n ∈ N.

(b) 〈a : h(a) = 0′, +, 0,−,×〉 is a ring.

24. Let h : A → A′ be a ring isomomorphism. Prove that

(a) A is unitary if and only if A′ is unitary.

(b) A has a multiplicative inverse in A if and only if h(a) has a multi-plicative inverse in A′.

25. Consider the properties of ring isomorphisms stated in Exercise 24. Dothese properties also hold for ring homomorphisms?

26. Prove that the map (1.7) is a ring homomorphism.

27. Show that proving the surjectivity of the map (1.7) amounts to prove thatgiven any (x1, . . . , xr) ∈ Zn1 × . . .× Znr

the system of congruences

x =n1 x1

. . .

x =nrxr

has a solution in Zn1×...×nr.

28. Find all the integer solutions of the following systems of congruences

(a)

3x− 2 =7 4

13x =9 −2

(b)

2x + 4 =9 −1

12x− 2 =5 6

(c)

5x + 10 =9 −15x− 4 =7 64x− 2 =5 6

(d)

3x + 1 =7 104x− 2 =9 −3x + 3 =4 1


29. Consider the cryptographic system RSA with prime numbers p = 3 andq = 11, public key (33, 3) and private key (33, 7). Explain how it worksencrypting the message 2 and decrypting the resulting message.

30. Consider the cryptographic system RSA with prime number p = 13, andpublic key (143, 7) and let 9 be an encrypted message. Compute the cor-responding private key and decrypt 9 to obtain the original message x.Confirm that u(143,7(x) = 9.

31. Consider the cryptographic system RSA with prime numbers p = 7 andq = 11. Choose an appropriate public key and a corresponding privatekey, and explain how it works encrypting the message 2 and decrypting theresulting message.

32. Consider the cryptographic system RSA with prime numbers p = 7 andq = 13. Choose an appropriate public key and a corresponding privatekey, and explain how it works encrypting the message 3 and decrypting theresulting message.

33. The RSA cryptographic system requires a fast modular exponentiation.Develop in Mathematica an efficient algorithm for modular exponentiationusing the binary representation of the exponent.

Hint: assuming that b is the exponent and that its binary representationis bk, bk−1, . . . , b1, b0, start by assigning to the result variable the valueabk and then do a cycle from k − 1 until 0 such that in the i-iteration,if bk−i = 1 then square the result variable, multiply it by a and reduceit modulo n, otherwise square the result variable and reduce it modulo n.This algorithm is known as Repeated Squaring Algorithm.

34. Assume that the RSA cryptographic system is being used with Zn as themessage space and (n, a) as the public key. Show that if a and the primefactors of n are known then it is possible to obtain b. (This explains thereason why factoring is considered to be the Achilles’ heel of the RSA.)

35. Assume that the RSA cryptographic system is being used with Zn as themessage space and (n, a) as the public key. Show that it is feasible foran attacker to know the private key (n, a) corresponding to the public key(n, a) used, if he knows φ(n).

36. Prove that if there exists an algorithm that in polynomial time, given u(x),returns the last bit of x, then the private key can be found in polynomialtime, that is, u can be inverted in polynomial time.

Chapter 2

Pseudo-random numbers

In this chapter we discuss the generation of pseudo-random numbers. Randomnumbers are useful in several different fields such as simulation, sampling andcryptography. In simulation and sampling, random numbers are used to createrepresentative real world scenarios. In cryptography, they are used, for instance,to generate strings of bits and to random generate keys in a given key space.

Physical methods based in entropy theory can be used to generate sequencesof numbers that can be considered close to truly random number sequences. Butthey can be expensive and slow for many applications. For many purposes, it isenough to use some suitable number sequence generating algorithms, the pseudo-random number generators. In this case, the number sequences are completelydetermined by the initial value (the seed), but a careful choice of the appropriatealgorithms often yields useful number sequences for many applications.

There are several features of pseudo-random number generators that can bemeasured. Suitable statistical pattern detection tests can then be used. Provinga pseudo-random number generator secure is more difficult, but it is of utmostrelevance in cryptography. A pseudo-random number generator should be fastand secure. But it is not easy to get both at the same time. Linear congruentialgenerators [17], for instance, are fast and therefore useful in simulation appli-cations, but they are not secure enough for cryptographic applications. Othersgenerators, like, for example, Blum Blum Shub generators [4] are slow but theirsecurity properties makes them suitable for cryptographic applications.

In Section 2.1 we present a motivating example related to traffic simulation.In section 2.2 we introduce linear congruential generators. Blum-Blum-Shubgenerators are presented in in Section 2.3. In Section 2.4 we revisit the trafficsimulation example. In Section 2.5 we propose some exercises.

51

52 CHAPTER 2. PSEUDO-RANDOM NUMBERS

2.1 Motivation: traffic simulation

Many complex systems can be studied using computer simulation. Using suitablemodels we can study their behaviour and predict their evolution. Computersimulation is then an important tool in many different fields such as Physics,Chemistry, Biology, Engineering, Economics and even Sociology.

In computer simulation we can use continuous or discrete models, depend-ing on the particular application. Continuous models usually use differentialequations that describe the evolution of relevant continuous variables. Discretemodels can be used when the systems we want to study can be described byevents and their consequences. Simulation can also be classified as deterministicor stochastic. In the later case, the variables follow random laws.

Herein, we present a traffic problem simulation example. The problem can bedescribed as follows.

Vehicles randomly come in to a given toll road. After arriving to the tollbooth (with only one toll gate), they stay in the queue until the paymentis done, and then they leave the road. The goal is to study the evolutionof the number of vehicles in the toll queue, in terms of some given randomlaws of the intervals between arrivals, arrival of a vehicle to the toll gateand departure after payment.

This is an example of discrete event simulation. The system is representedby a sequence of events and the occurrence of an event corresponds to a changeof state in the system. There is a list of simulation events listing the pendingevents, that is, the events that will have to simulated. This list is also known asthe pending event set.

According to this technique, the first step is the identification of the kindsof relevant events in the system being considered. In this case there are thefollowing three kinds of events (named according to the usual designations inqueue simulation): arr (arrival), ess (end of self-service) and dep (departure).The first one corresponds to the arrival of a vehicle to the toll road, the secondone to the arrival of a vehicle to the toll gate, and the third one to the departureof a vehicle after the payment. Each event is characterized by several attributes.The relevant ones herein are the following: time (time of the event occurrence)and kind (the kind of the event). In more complex simulations more attributesmay be considered. For example, if we want to study not only the time spentat the queue but also the time spent since the vehicle enters the road until itreaches the toll queue, it is also necessary to indicate for each event the vehiclewith which it is associated.

The Mathematica package in Figure 2.1 offers an intuitive and user-friendlycollection of services that includes the creation of an event and the access to theirdifferent attributes.

2.1. MOTIVATION: TRAFFIC SIMULATION 53

BeginPackage["trafficSim`des`eventsP`"]

eventsP::usage = "Operations on events."

evt::usage = "evt[t,k]: the event on time t of kind k."

time::usage = "time[e] returns the time of event e."

kind::usage = "kind[e] returns the kind of event e."

Begin["`Private`"]

evt = Function[t,k,t,k]time = Function[e,e[[1]]]

kind = Function[e,e[[2]]]

End[]

EndPackage[]

Figure 2.1: Mathematica package for events

The second step is the definition of the random laws followed by the events.With respect to arrivals we assume herein that the interval of time betweenconsecutive arrivals is a random variable following a exponential distribution withaverage value ba (between arrivals). With respect to self-service, we consider thatthe time that a vehicle takes to cross the road is a random variable following anexponential distribution with average value ss (self-service). Finally, with respectto payments, it is assumed that the time that a vehicle takes to pay (since thebeginning of the payment to its departure) is a random variable following anexponential distribution with average value st (service time).

Recall that a random variable following an exponential distribution with av-erage value m has distribution function F (t) = 1− e

tm . For example, for m = 2

and t = 6, the value of 1 − etm is approximately 0.95. This value can be inter-

preted as follows: the probability that the observed value is less than or equal to3 times the average value is a little bit more than 95%. That is, the probabilitythat the observed value is greater than 3 times the average value is less than 5%.

The third step consists in defining procedures for simulating the observationof the random variables of the system. The Mathematica package in Figure 2.2includes a collection of services providing the procedure exprandom.

Observe that in order to define the function exprandom we use the Mathematicafunction Random that generates a pseudo-random number in the interval [0, 1]following an uniform distribution. Recall that all random variable distributions


BeginPackage["trafficSim`des`randomnumbersP`"]

randomnumbersP::usage = "Exponential random numbers."

exprandom::usage = "exprandom[m] returns an observation of

the exponential random variable with mean value m."

Begin["`Private`"]

exprandom = Function[m, -(m*Log[Random[]])]

End[]

EndPackage[]

Figure 2.2: Mathematica package for random numbers

can be obtained from the uniform random variable. So it is of utmost importanceto know how to obtain uniform pseudo-random generators in order to get othertypes of generators.

2.2 Linear congruential generators

In this section we present linear congruential generators. We introduce linearcongruential sequences and the conditions its parameters should meet in order toensure maximum length period.

Definition 2.2.1 A linear congruential sequence is a sequence snn∈N0 suchthat

sn+1 = mod(asn + c, m)

for all n ∈ N0, where m, s0, a, c ∈ N0 and s0, a, c < m.

A linear congruential sequence snn∈N0 is a sequence of elements of Zm forsome positive integer m. The parameters s0, a, c and m are the seed, the multi-plier, the increment and the modulus of the linear congruential sequence, respec-tively.

Example 2.2.2 Let us look at several consecutives terms of some linear congru-ential sequences snn∈N0. All examples start with the term s1

(i) s0 = 2, a = 1, c = 2 and m = 11

4, 6, 8, 10, 1, 3, 5, 7, 9, 0, 2, 4, 6, 8, 10, 1, 3, 5, 7, 9, 0, 2, 4, 6, 8, 10, 1, . . .

2.2. LINEAR CONGRUENTIAL GENERATORS 55

(ii) s0 = 2, a = 7, c = 5 and m = 9

1, 3, 8, 7, 0, 5, 4, 6, 2, 1, 3, 8, 7, 0, 5, 4, 6, 2, 1, 3, 8, 7, 0, 5, 4, 6, 2, 1, . . .

(iii) s0 = 2, a = 2, c = 2 and m = 11 then we have

6, 3, 8, 7, 5, 1, 4, 10, 0, 2, 6, 3, 8, 7, 5, 1, 4, 10, 0, 2, 6, 3, 8, 7, 5, 1, 4, . . .

(iv) s0 = 2, a = 3, c = 2 and m = 12 then we have

8, 2, 8, 2, 8, 2, 8, 2, 8, 2, 8, 2, 8, 2, 8, 2, 8, 2, 8, 2 . . .

(v) s0 = 2, a = 5, c = 2 and m = 25 then we have

12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, . . .

Using the word “random” in a rather informal way, can say that some sequenceslook more random than others. Note that the seed may or may not occur againin the sequence as a term of index greater than 0. Moreover, it may be the casethat not all the elements of Zm occur in the sequence. The last sequence is evenconstant for n ≥ 1, since mod(5× 12 + 2, 25) = mod(62, 25) = 12.

The following proposition states that the seed indeed occurs more than oncein a linear congruential sequence whenever the multiplier and the modulus arecoprime.

Proposition 2.2.3 Let snn∈N0 be a linear congruential sequence such that itsmultiplier a and its modulus m are coprime. Then there is k ∈ N such thatsk = s0.

Proof: First of all note that for i, j > 0 if si = sj then si−1 = sj−1. In fact,recalling Definition 2.2.1, we have asi−1 + c =m asj−1 + c, that is, asi−1 =m asj−1.Since a and m are coprimes, a has an inverse a−1 in Zm. Hence, multiplying theleft and the right hand sides by a−1 and noting that mod(si−1, m) = si−1 andmod(sj−1, m) = sj−1, we get si−1 = sj−1.

Assume that k ∈ N is the least index such that sk = si for some 0 ≤ i < k.Such a k always exists because every term of s is in Zm and this set is finite. Itcan not be the case that i ≥ 1 since then si−1 = sk−1 and therefore k would notbe the least index satisfying the conditions above. Hence, i = 0 and we concludethat sk = si. QED

In any linear congruential sequence snn∈N0 there is a finite sequence ofnumbers that is repeated an infinite number of times. In fact, since the set Zm isfinite and each term uniquely determines the next one, there are i, k ∈ N0 suchthat si+k = si and, once si+k = si, the terms following si+k are exactly the samefollowing si.


The period length of a linear congruential sequence snn∈N0 is the least k ∈ N

such that there is i ∈ N0 such that si+k = si. Clearly, the period is always lessthan or equal to the modulus m of s and therefore we can always determine theperiod of s after computing m + 1 terms at the most. The maximum periodlentght of a linear congruential sequence is the modulus m of that sequence.

Example 2.2.4 Recall the linear congruential sequences presented in Example2.2.2. The period length of the sequence presented in (i) is 11. It is the maximumperiod lenght. Similarly, the sequence presented in (ii) also has maximum periodlength (9 in this case). The sequence presented in (iii) has a period of length 10(note that 9 does not occur in the sequence). Finally, the period length of thesequence presented in (iv) is 2 and the period length of the sequence presentedin (v) is 1.

The choice of the multiplier m determines an upper bound for the period of alinear congruential sequence. The others parameters have to be carefully chosenin order to ensure that the period is as long as possible.

The following proposition, known as the Maximum Period Length Theorem,establishes several conditions on the parameters that ensure maximum periodlength.

Proposition 2.2.5 Let snn∈N0 be linear congruential sequence with multipliera, increment c and modulus m. The sequence s has period length m if and onlyif all the following conditions hold

(i) c and m are coprime;

(ii) a− 1 is a multiple of every prime divisor of m;

(iii) a− 1 is a multiple of 4 whenever m is a multiple of 4;

where a, c and m are the multiplier, the increment and the modulus of s respec-tively.

Note that the seed is not relevant to ensure maximum period length. For theproof of Proposition 2.2.5 we refer the reader to [17], for instance.

Example 2.2.6 Recall again the linear congruential sequences presented in Ex-ample 2.2.2. The sequence presented in (i) satisfies the conditions of Proposition2.2.5. Observe that a− 1 = 0 and 0 a multiple of any integer. The sequencepresented in (ii) also satisfies those conditions. Note that m is not a multiple of 4and therefore a−1 is not required to be a multiple of 4. In the sequence presentedin (iii), although c and m are coprime, a− 1 = 1 and 1 is not a multiple of 11(note that 11 the only prime number that divides m in this case) and thereforethe sequence does not verify the conditions of Proposition 2.2.5.

2.2. LINEAR CONGRUENTIAL GENERATORS 57

It is worth noticing that a long period is not the only requirement for agood choice of the parameters of a linear congruential sequence. The sequence(i) in Example 2.2.2, for instance, has maximum period length but, informallyspeaking, it is not much “random” since even and odd numbers alternate. Thedifference of two consecutive terms is almost always equal to 2 because a = 1 andtherefore in most cases sn+1 − sn = c = 2.

The Mathematica function MCL in Figure 2.3, computes the terms of linearcongruential sequences, assuming that suitable integer values have already beingassigned to the variables a , s0 , c and m.

MCL=Function[,s=Mod[a*s+c,m]];

Figure 2.3: Linear congruential sequence in Mathematica

When computation time is an issue we often consider linear congruential se-quences snn∈N0 with c = 0. However, in this case, a period of length m is nolonger advisable since

sn+1 = mod(asn, m)

and therefore once si = 0 for some i ∈ N0 then sj = 0 for all j > i.

Another relevant fact in this case is that given a common divisor d ∈ N of si

and m we have

si+1 = mod(asi, m) = asi −⌊asi

m

⌋

m = d(

ksi −⌊asi

m

⌋

k′)

for some integers 0 < k, k′ ≤ m, that is, si+1 a multiple of si. Moreover, we canalso conclude that sj is a multiple of d for all j ≥ i. It is then advisable to ensurethat m and sn are coprime for all n ∈ N0. In this case the period length is atmost the number of positive integers coprime to m, that is, the period length isat most φ(m), where φ is the Euler function (see Definition 1.2.20).

The following proposition establishes some conditions that ensure the maxi-mum possible period length when the increment is 0. For the proof we refer thereader to [17]. Recall from Definition 1.3.11 that λ(m) is the order (modulo m)of the primitive elements modulo m.

Proposition 2.2.7 Let snn∈N0 be a linear congruential sequence with multi-plier a, modulus m and increment 0. If s0 and m are coprime and a is a primitiveelement modulo m then the period length of s is λ(m). Moreover, λ(m) is themaximum possible period length of any linear congruential sequence with modu-lus m and increment 0.


Example 2.2.8 Let us consider a linear congruential sequence snn∈N0 suchthat m = 5, s0 = 4 and a = 2. Recall from Example 1.3.10 that 2 is a primitiveelement modulo 5. Since the seed and the modulus are coprime, the sequences satisfies the conditions of Proposition 2.2.7 and therefore it has period λ(5).Recall from Example 1.3.12 that λ(5) = 4.

As we have already remarked in Section 2.2 with respect to the generation ofpseudo-random numbers for simulation purposes, it is of utmost importance tohave uniform pseudo-random generators in order to get other types of generators,capitalizing on the fact that every distribution can be generated from the uniformdistribution in the interval [0, 1]. Linear congruential sequences allow us to obtainsuch sequences of pseudo-random numbers in [0, 1] as follows [17]: given a linearcongruential sequence (sn)n∈N0 with modulus m we just consider the sequenceu = (un)n∈N0 where

un =sn

m

for each n ∈ N0.

In cryptographic applications we often have to randomly generate stringsof bits. We now describe how linear congruential sequences can be use as bitgenerators. First, we define bit generator functions.

Definition 2.2.9 Let j, k ∈ N such that k > j. A (j, k)-bit generator is a func-tiom

f : Zj2 → Zk

2

that can be computed in polynomial time with respect to j. For each r ∈ Zj2, the

string f(r) is the bit string generated by r.

In practice, given j, k is obtained as a polynomial function of j. Let s = (sn)n∈N0

be a linear congruential sequence with modulus m. We can the define a (j, k)-linear congruential generator as follows: given j = 1 + ⌊log2 m⌋ and j < k < m,then

f : Zj2 → Zk

2

is such that f(s0) = (z1, z2, . . . , zk) where

zi = mod(si, 2)

for each 1 ≤ i ≤ k. Note that in f(s0) we are assuming the binary representationof the seed of s that, by hypothesis, has a length less than or equal to j.

2.3. BLUM-BLUM-SCHUB GENERATORS 59

2.3 Blum-Blum-Schub generators

2.4 Traffic simulation revisited

Having discussed the problem pseudo-random numbers generation we proceedherein the example of traffic simulation.

According to the discrete event simulation technique we are using, the centralpart of the simulator is the pending event list. Consider the traffic simulationassuming that there are vehicles in the road, vehicles in the toll queue, and avehicle is paying in the toll gate. How can the simulation proceed from thissituation? Note that the simulation is mainly a loop where in each step theoccurrence of an event is simulated. The next event to be simulated can be thearrival of a vehicle to the road (arr), the arrival of a vehicle already in the road tothe toll booth, that is, its inclusion in the payment queue (ess), or the conclusionof the payment of the vehicle that is paying (dep). In general, the next event tobe simulated is the next event in the pending event list, where by the next eventin the pending occurrence list we mean the event in the list with the least timevalue.

Another important issue is the way the pending event list is fed with newevents. The answer to this question is not so simple. First, note that the list hasto contain the events that we already know that have to be simulated in the future,and that have not already been simulated because their time has not come yet.For example, each time an arrival is simulated, the arrival of the vehicle to the tollbooth, that is, the event evt[ct+exprandom[ss],"ess"], can be immediatelyplaced in the pending event list, as well as, the arrival of the next vehicle to theroad, that is, the event evt[ct+exprandom[ba],"arr"], where ct is the variableof the simulator holding the current time. Each time an arrival to the toll boothis simulated, it should be checked if the toll gate is empty or not. If it is notempty then the toll queue should be incremented with another vehicle. Otherwisethe vehicle should begin the payment phase and we can then generate the eventfor the end of the payment, that is, the event evt[ct+exprandom[st],"dep"].Finally, each time a vehicle leaves the toll road after paying, that is, each timea departure event occurs, the toll queue should be inspected. If it is empty thenthe toll gate should be set to not occupied. Otherwise the first element of the tollqueue should be removed from the queue and the event for the conclusion of itspayment can already be placed in the pending event list, since it will occur forcertain, that is, the event evt[ct+exprandom[st],"dep"] can already be placein the list.

The pending event list (schedule) is, therefore, a collection of events togetherwith some operations. A Mathematica package for the scheduling is presented inFigure 2.4). Note that the events are kept ordered in the list according to itstime attribute. Hence, the next function simply returns the first element of the


BeginPackage["trafficSim`des`schedulesP`"]

Needs["trafficSim`des`eventsP`"]

schedulesP::usage="Operations on schedules."

empty::usage="The empty schedule."

next::usage="next[s] returns the next event of schedule s."

add::usage="add[e,s] inserts event e into schedule s."

delete::usage="delete[s] removes the next event from

schedule s."

Begin["`Private`"]

empty=next=Function[s, s[[1]]]

add=Function[e,s,If[s==empty,

e,If[time[e]<time[s[[1]]],

Prepend[s,e],

Prepend[ add[e,Rest[s]],s[[1]]]]]]

delete=Function[s,Rest[s]]

End[]

EndPackage[]

Figure 2.4: Mathematica package for scheduling

list. The function add is more complicated since it inserts the received event inthe right place in the list.

The simulator is presented in Figure 2.5. The simulator is no more than aloop. In each step of the loop the next event in the pending event list is simulated.In that simulation, the list and the state variables (length of the toll queue, stateof the toll gate, etc) are changed accordingly to the event being simulated, andthe variables under simulation (namely, the length of the toll queue) are observed.

The local variables of the function sim are the following: busy (it indicateswhether the toll gate is occupied or not), ce (the current event being simulated),ct (the time of the event currently being simulated), nss (number of clientsin self-service, that is, the number of vehicles in the toll road), nwc number ofwaiting clients, that is, the number of vehicles in the toll queue), sch (schedule),tnc (total number of clients, that is, the total number of vehicles in the toll),trace (it records the simulated events and the values of some variables), ck (thekind of the event currently under simulation).

2.4. TRAFFIC SIMULATION REVISITED 61

BeginPackage["trafficSim`des`simulationP`"]

Needs["acs`des`eventsP`"]

Needs["acs`des`schedulesP`"]

Needs["acs`des`randomnumbersP`"]

simulationP::usage = "Discrete event simulation."

sim::usage = "sim[ba,ss,st,ht] runs the simulation with:

average time between arrivals ba, average time of self-

service ss, average service time st, and halting time ht."

Begin["`Private`"]

sim=Function[ba,ss,st,ht,Module[busy,ce,ct,nss,nwc,sch,tnc,trace,ck,

simArr=Function[,sch=add[evt[ct+exprandom[ba],"arr"],sch];

sch=add[evt[ct+exprandom[ss],"ess"],sch];

tnc=tnc+1; nss=nss+1];

simEss=Function[,nss=nss-1;

If[busy==1,

nwc=nwc+1,

sch=add[evt[ct+exprandom[st],"dep"],sch];

busy=1]];

simDep=Function[,If[nwc==0,

busy=0,

nwc=nwc-1;

sch=add[evt[ct+exprandom[st],"dep"],sch]]];

busy=0;

ce=evt[exprandom[ba],"arr"]; ct=time[ce]; ck=kind[ce];

nss=0; nwc=0; sch=empty; tnc=0; trace=;While[ct<=ht,

Switch[ck,"arr",simArr[],"ess",simEss[],"dep",simDep[]];

trace=Append[trace,ct,nwc];ce=next[sch]; ct=time[ce]; ck=kind[ce];

sch=delete[sch]];

ListPlot[trace,PlotJoined->True]]]

End[]

EndPackage[]

Figure 2.5: Mathematica package for the simulation


The output of the simulation is a graphic displaying the evolution of the lengthof the toll queue, that is the value of the variable nwc. Figure 2.6 depicts thegraphic corresponding to a simulation assuming that ba, the average interval oftime between consecutive arrivals, is 1, that ss, the average time a vehicle takesto cross a road, is 50, and that st, the average time a vehicle takes to pay, is0.9. Figure 2.6 depicts the graphic corresponding to a similar simulation but

200 400 600 800 1000

5

10

15

20

25

Figure 2.6: Evolution of the toll queue length assuming ba=1, ss=50 and st=0.9

assuming that st is 2. In both cases, the starting time is 0 and the halting timeis 1000

200 400 600 800 1000

100

200

300

400

Figure 2.7: Evolution of the toll queue length assuming ba=1, ss=50 and st=2

2.5 Exercises

1. Let s = snn∈N0 be a linear congruential sequence with parameters s0, a,c and m. Prove that if a = 1 then sn = mod(s0 + nc, m).

2.5. EXERCISES 63

2. Let s = snn∈N0 be a linear congruential sequence with parameters s0,a 6= 1, c and m. Prove that for every nonnegative integer k

sn+k = mod

(

aksn +ak − 1

a− 1c, m

)

.

3. Let s = snn∈N0 be a linear congruential sequence with parameters s0, a,c and m and assume that m = 2e some positive integer e.

a) Prove that if c is odd and mod(a, 4) = 1 then s has maximum lengthperiod.

b) Prove that implication in a) is an equivalence whenever e ≥ 2.

4. Let s = snn∈N0 be a linear congruential sequence with parameters s0, a,c and m, such that m = 10e for some integer e ≥ 2, and that neither 2nor 5 divide c. Prove that s has maximum length period if and only ifmod(a, 20) = 1.

5. Let s = snn∈N0 and s′ = s′nn∈N0 be linear congruential sequences withparameters s0, a, c and m, and s′0, a′, c′ and m′, respectively, such thats0 = s′0, a = a′, c = c′ and m = re and m′ = re−1 for some e, r ∈ N.

a) Prove that s′k = mod(sk, m′) for every nonnegative integer k.

b) Prove that if s has maximum period length then s′ has also maximumperiod length.

6. Let s = snn∈N0 be a linear congruential sequence with maximum periodlength, seed s0 = 0 and modulus m = re with r, e ∈ N. What is the valueof sre−1?

7. Let s = snn∈N0 be a linear congruential sequence with parameters s0, a, cand m. Find all the values of a that satisfy the conditions of the maximumperiod theorem when m = 106−1 (note that 106−1 = 33×7×11×13×37).

8. Define a linear congruential sequence with maximum period length with

(a) m = 162

(b) m = 402

9. Determine all the primitive elements modulo 5 in Z5 and define a linearcongruential sequence with maximum possible period length with modulus5 and increment 0.




12. Define linear congruential sequences with maximum possible period lengthwith increment 0 and

(a) modulus 15.

(b) modulus 162.

(c) modulus 402.

Determine the period length of each sequence. Note that 5 is a primitiveelement modulo 162 and 7 is a primitive element modulo 402.

13. Develop an enriched version of the simulator presented in Subsection 2.4that traces the average number of vehicles during a specific period of timeand the maximum length of the toll queue.

14. Develop an enriched version of the simulator presented in Subsection 2.4that distinguishes between two kinds of vehicles, the light vehicles and theheavy vehicles, such that each kind has a different average time betweenarrivals to the toll road. Make an histogram, by each kind of vehicle, of thenumber of vehicles in the poll queue during the simulation time.

Chapter 3

Polynomials

Polynomials and polynomial equations are widely used in science and engineering.Herein, we present several key concepts and results related to polynomials.

In Section 3.1 we start by motivating that polynomials can be used to verifyequivalence of digital circuits. Then we illustrate the relevance of polynomialsin robotics. In Section 3.2 we introduce the notion of polynomial over a fieldas well as the sum and product of polynomials. We then introduce division ofpolynomials and several related results. Grobner bases and their properties arepresented in Section 3.3. In Section 3.4 we revisit our motivating examples andshow how to use Grobner bases for checking equivalence of digital circuits andfor finding solutions of systems of nonlinear polynomial equations. In Section 3.5we propose some exercises.

3.1 Motivation

In this section, we present two motivating examples using polynomial equations.The first one is related to the validity of propositional formulas and the problemof checking the equivalence of digital circuits. The second one illustrates whysolving systems of nonlinear equations is important in robotics.

3.1.1 Digital circuit equivalence

Digital circuits are hardware components that manipulate binary information[23]. In particular, combinational circuits consist of input variables, output vari-ables and interconnected logic gates. They accept binary signals from the inputsand generate binary signals at the outputs. Outputs are determined by combin-ing the input values using the logic operations associated with the logic gates.Hence, a combinational circuit can be specified by a truth table that lists theoutput values for each combination of input values. If there is only one outputvariable, the combinational circuit computes a Boolean function and can also be

65

66 CHAPTER 3. POLYNOMIALS

represented by a propositional formula. If there are n > 1 output variables, itcomputes n Boolean functions, one for each variable, and can be represented byn propositional formulas.

There are algorithms for designing combinational circuits that compute theBoolean function(s) corresponding to a given truth table. However, the resultingcircuits are not always as simple/efficient as they could be. For instance, theymay have more gates than strictly needed. Hence, we may have to introducemodifications in the design. But, of course, we have to ensure that the originalversion of the circuit, A, is equivalent to the new version, B. That is, the twoproduce the same outputs for the same inputs. If there is only one output,this amounts to say that the propositional formula ϕA induced by circuit A isequivalent to the propositional formula ϕB induced by circuit B. That is, thepropositional formula

ϕA ⇔ ϕB

is valid, or equivalently, its negation is not satisfiable. Therefore, the task ofchecking the equivalence of these circuits corresponds to the task of checkingwhether two propositional formulas are equivalent. For simplification herein weonly illustrate circuit with one output, but the technique can be extended tocircuits with more outputs.

There are several algorithms for checking the satisfiability of propositionalformulas known as SAT-algorithms [16, 21]. In Section 3.4 we will see how poly-nomials can also be used for this purpose.

3.1.2 Inverse kinematics of a robot

Consider the robot arm depicted in Figure 3.1. It represents a robot consistingof three arm links a1, a2 and a3 with fixed length, two joints J1 and J2, an handE and a base A that supports the robot.

J1

J2

E

a3a2

a1

A

Figure 3.1: Sketch of a robot arm

To study the robot movement we assume that the arm links are represented byline segments and the joints and hand by points in the 3-dimensional Euclideanspace. We consider a Cartesian coordinate system 0xyz with origin in J1. A

3.1. MOTIVATION 67

projection on the yz plane of the robot arm is depicted in Figure 3.2 (the x-axispoints toward the observer). A projection on the xy-plane of the arm link a2 isdepicted in Figure 3.3.

(a, b, c)-y

0

z6 θ1

θ2

A

Figure 3.2: Projection on the yz-plane

For simplicity, we assume that the arm link a1 always lies in the z-axis and,moreover, that the three arm links always lie in the same plane. Furthermore, weassume that base A is fixed but the arm a1 might rotate around the z-axis. Thejoint J1 only rotates around the axis that contains J1 and it is perpendicular tothe plane containing the arms. Joint J2 works similarly.

-

?

0 y

x

β

Figure 3.3: Projection of the arm link a2 on the xy-plane

Kinematics is the task of computing the coordinates (a, b, c) of hand E. Thatis, its position in space, given the angle β at the base and the angles θ1 and θ2

of the joints. Clearly, there is only one possible solution. The coordinates can beeasily computed using simple calculations involving trigonometric functions.

Inverse kinematics is the reverse task (see for instance [25, 3]). That is,given the intended coordinates of the hand we want to determine what are thesuitable angles at the base and joints for reaching that position. This task isoften difficult. It can be the case that no solution or more that one solution exist.The task becomes harder if the degrees of freedom of the joints increase and insome cases infinitely many solutions may exist.

Let l1, l2 and l3 be the length of the arm links a1, a2 and a3, respectively.Then, in the coordinate system 0xyz above we have that

a = (l2 cos θ1 + l3 cos θ2) cos β

b = (l2 cos θ1 + l3 cos θ2) sin β

c = l2 sin θ1 + l3 sin θ2


Hence, to determine the angles β, θ1 and θ2 given the coordinates (a, b, c), wehave to solve this system. One possibility consists in converting these equationsinto polynomial equations where the variables are the sines and cosines. We canconsider, for instance, one of the following equivalent systems

a = (l2v2 + l3v3)v1

b = (l2v2 + l3v3)u1

c = l2u2 + l3u3

1 = u21 + v2

1

1 = u22 + v2

2

1 = u23 + v2

3

or

l2v2v1 + l3v3v1 − a = 0

l2v2u1 + l3v3u1 − b = 0

l2u2 + l3u3 − c = 0

u21 + v2

1 − 1 = 0

u22 + v2

2 − 1 = 0

u23 + v2

3 − 1 = 0

The systems have 6 variables, ui and vi for i = 1, 2, 3, where u1 = sin β, v1 = cos β,and ui = sin θi−1 and vi = cos θi−1 for i = 2, 3. The last three equations expressthe well known Pythagorean trigonometric identity that involves the values ofthe sine and cosine of an angle.

Note that taking into account the Pythagorean trigonometric identity wecould also have considered only the three variables u1, u2 and u3. Each vi wouldthen be replaced by 1− u2

i and the three last equations omitted.The Gaussian elimination algorithm (see Section 4.1) solves systems of poly-

nomial equations where each term of the polynomial involves at most one variableand the corresponding exponent is always less or equal than 1 (linear equations).In order to solve systems of nonlinear polynomial equations, as the one we haveabove, other algorithms have be considered. In Section 3.4 we will see howGrobner bases can be used for solving these systems.

3.2 Basic concepts

In this section we first present the notion of polynomial (over a field) as wellas sums and products of polynomials [20, 10]. Polynomials over a field togetherwith the sum and product of polynomials constitute a ring (see Section 1.3 ofChapter 1). Next we introduce the notion of ordered polynomial and then thedivision of polynomials. At the end we refer to polynomial reduction modulo aset of polynomials.

3.2.1 Rings of polynomials

Our goal is to define a particular kind of ring: a ring of polynomials. We haveto introduce the notion of polynomial in n variables with coefficients in a field Cand the associated operations of sum and product. We first introduce the notionof monomial in n variables.

3.2. BASIC CONCEPTS 69

Definition 3.2.1 The set of monomials in the variables x1, . . . , xn, with n ∈ N

and n > 0, is denoted byMx1,...,xn

and is the set of sequencesxα1

1 . . . xαnn

where αi ∈ N0 for 1 ≤ i ≤ n. The degree of the monomial xα11 . . . xαn

n , denotedby

deg(xα11 . . . xαn

n )

is the natural α1 + . . . + αn.

Example 3.2.2

• x21 is a monomial in the variable x1 with degree 2;

• x21x

32 is a monomial in the variables x1, x2 with degree 5.

For simplicity, we can refer just to monomials in x1, . . . , xn. Given a monomialxα1

1 . . . xαnn we often omit xαi

i when αi = 0 and write 1 when α1 = . . . = αn = 0.For instance, we often write x2

1x12x

14 instead of x2

1x12x

03x

14 and 1 instead x0

1x02x

03.

We can also write xi for xαi

i when αi = 1, that is, we can write x21x2x4 instead of

x21x

12x

03x

14. We may even write M for Mx1,...,xn

when no confusion arises.We now introduce the notion of polynomial in the variables x1, . . . , xn over a

field.

Definition 3.2.3 Let C = (C, +, 0,−,×, 1) be a field and let n ∈ N. A polyno-mial in the variables x1, . . . , xn over C is a function

p : Mx1,...,xn→ C

such that p(m) = 0 for all but a finite number of elements of Mx1,...,xn.

Each p(m) ∈ C is a coefficient of p, the coefficient of the monomial m in thepolynomial p.

If p is such that p(m) = 0 for each monomial m then p is the zero polynomial.If p is a nonzero polynomial the degree of p, denoted by deg(p), is the integer

maxdeg(m) : m ∈M and p(m) 6= 0.

The set of all polynomial in the variables x1, . . . , xn over C is denoted by

C[x1, . . . , xn].

Polynomials in C[x1, . . . , xn] are called univariate polynomials when n = 1, andmultivariate polynomials when n > 1. The zero polynomial in C[x1, . . . , xn] isdenoted by 0C[x1,...,xn]. The subscript is omitted when no confusion arises. The


polynomial p in C[x1, . . . , xn] such that p(x01 . . . x0

n) = 1 and p(m) = 0 for all theother monomials in x1, . . . , xn is denoted by 1C[x1,...,xn]. The subscript is againomitted when no confusion arises.

A polynomial p in C[x1, . . . , xn] is often presented as a sum of all monomialsweighted with their nonzero coefficients. That is,

∑

m∈M :p(m)6=0p(m)m.

Note that using this notation the same polynomial can be referred to in differentways. For simplicity, all the conventions introduced above for monomials canalso be used. Moreover, a monomial mi can be omitted when deg(mi) = 0 and acoefficient ci can be omitted whenever it is the multiplicative identity of C, unlessdeg(mi) = 0.

Given p in C[x1, . . . , xn] and m ∈M , we say that p(m)m is a term of p. Asexpected, the coefficient of the term is p(m) and its monomial is m. A monicterm is a term whose coefficient is the multiplicative identity of C and a zeroterm is a term whose coefficient is the additive identity of C. The degree of aterm t, denoted by

deg(t)

is 0 if t is a zero term and is the degree of its monomial otherwise. When noconfusion arises, we can refer just to terms in x1, . . . , xn over C without mentionany particular polynomial in C[x1, . . . , xn].

Example 3.2.4 Consider the polynomial p in Z5[x1, x2] such that

• p(x31x

22) = 2;

• p(x21x

22) = 3;

• p(x01x

02) = 1;

• p(m) = 0 for all the other monomials in x1, x2.

We can present p as2x3

1x22 + 3x2

1x22 + 1

Observe that p has degree 5 and its terms are 2x31x

22, 3x2

1x22 and 1x0

1x02. The

coefficient of 21x31x

22 is 2 and its monomial is x3

1x22.

The Mathematica function degmon in Figure 3.4 receives as input a term q

and positive integer n, and returns the degree of q. Assuming that q is the termcxα1

1 . . . xαmm , the function first creates the list c, xα1

1 , . . . , xαmm . Then it removes

the coefficient c and adds the exponent of each variable. The function degmon

uses the built-in Mathematica function PolynomialMod that, given a polynomial


degmon = Function[q,n, Module[p,w,p=PolynomialMod[q,n];

If[Head[p]===Times, w=Apply[List,p], w=p];If[NumberQ[First[w]],w=Rest[w]];

Apply[Plus,Map[Function[m,

If[Head[m]===Power,m[[2]],1]],w]]]];

Figure 3.4: Degree of a term over Zn

poly and a positive integer n, returns the polynomial that results from poly byreplacing each coefficient c by a new coefficient c′ ∈ Zn such that c′ =n c.

Polynomials in x1, . . . , xn over C can be evaluated. Let

µ : x1, . . . , xn → C

be a map. This map can be extended to monomials considering the map

µ : Mx1,...,xn→ C

such thatµ(xα1

1 . . . xαn

n ) = µ(x1)α1 × . . .× µ(xn)αn

for each monomial xα11 . . . xαn

n in Mx1,...,xn. Then, the µ-evaluation of a polynomial

p in C[x1, . . . , xn] is

evalµ(p) =∑

m∈M :p(m)6=0p(m)× µ(m).

For simplicity, we also write p(c1, . . . , cn) for evalµ(p) when µ(xi) = ci for each1 ≤ i ≤ n.

Example 3.2.5 Let p = 2x21x

22 + x1x

22 + 1 be a polynomial in Z5[x1, x2] and

consider the map µ : x1, x2 → Z5 such that µ(x1) = 3 and µ(x2) = 4. Then,evalµ(p) = 2× 32 × 42 + 3× 42 + 1 = 2.

We now define sum and multiplication of polynomials.

Definition 3.2.6 Let p1 and p2 be polynomials in C[x1, . . . , xn]. The sum of p1

and p2 is the polynomial p1 + p2 in C[x1, . . . , xn] such that

p1 + p2(m) = p1(m) + p2(m)

for each monomial m in x1, . . . , xn.


In the polynomial p1 + p2 the coefficient of each monomial m is the sum (inC) of the coefficients of m in p1 and p2. Clearly,

deg(p1 + p2) ≤ maxdeg(p1), deg(p2).

Example 3.2.7 Let p1 = 3x31x2 +2x2

1x22 +x1x

22 +1 and p2 = 4x3

1x2 +2x1x22 +x3

2

be polynomials in Z5[x1, x2]. Then, p1 + p2 is

2x31x2 + 2x2

1x22 + 3x1x

22 + x3

2 + 1.

The symmetric of a polynomial p is the polynomial −p such that (−p)(m) isthe symmetric of p(m) in the field C.

Let m1 = xα11 . . . xαn

n and m2 = xβ1

1 . . . xβnn be two monomials in x1, . . . , xn.

The product of m1 and m2 is the monomial

xα1+β11 . . . xαn+βn

n

in x1, . . . , xn, denoted by m1 ×m2.Let us now introduce the multiplication of polynomials.

Definition 3.2.8 Let p1 and p2 be polynomials in C[x1, . . . , xn]. The product ofp1 and p2 is the polynomial p1× p2 in C[x1, . . . , xn] such that, for each monomialm in x1, . . . , xn

p1 × p2(m) =

0 if deg(m) > deg(p1) + deg(p2)

c11 × c21 + . . . + c1r × c2r if deg(m) ≤ deg(p1) + deg(p2)

where, letting (m11, m21), . . . , (m1r, m2r) be all the distinct pairs of monomials inx1, . . . , xn whose product is m, cij is the coefficient of mij in pi for each 1 ≤ i ≤ 2and 1 ≤ j ≤ r.

It is easy to conclude that deg(p1 × p2) = deg(p1) + deg(p2).

Example 3.2.9 Let p1 = 3x31x2 +2x2

1x22 +x1x

22 +1 and p2 = 4x3

1x2 +2x1x22 +x3

2

be polynomials in Z5[x1, x2]. Then, p1 × p2 is

2x61x

22 + x3

1x42 + 3x5

1x32 + 2x2

1x52 + 2x2

1x42 + x1x

52 + 4x3

1x2 + 2x1x22 + x3

2.

As usual we may omit the symbol × in a product of monomials or polynomialsand write p1p2 instead of p1 × p2.

It is easy to conclude that the above operations of sum and multiplication ofpolynomials satisfy all the relevant properties of a ring (Exercise 5 in Section 3.5).In the sequel, given a field C and n ∈ N, the ring of polynomials in x1, . . . , xn

over C, denoted byC[x1, . . . , xn]


is the ring (C[x1, . . . , xn], +, 0,−,×). As usual, we will assume that × takesprecedence over +. Ww will also write just p1 − p2 for p1 + (−p2).

Every ring of polynomials is unitary. The multiplicative identity is the poly-nomial 1 (Exercise 6 in Section 3.5).

Note that, in general, a ring of polynomials is not a field. Consider, forinstance, the polynomial 2x1 in R[x1]. It is easy to conclude that there is nopolynomial p in R[x1] such that 2x1 × p = 1.

3.2.2 Monomial orderings

Our next goal is to define division of polynomials. However, in order to definethis operation, we have to introduce first an ordering on monomials. In thissection we just introduce monomial orderings and the induced term orderings,postponing division to the next section.

A total order > in the set of all monomials in x1, . . . , xn is a monomial orderproviding that:

(i) > preserves products of monomials, that is

if m′ > m′′ then m′ ×m > m′′ ×m;

(ii) > is well founded, that is, there is no infinite sequence m1, . . . , mj , . . . ofmonomials such that

mj > mj+1 for all j ∈ N.

For more details on orders, see, for instance, [11]. There are several orders thatfulfill the above conditions. Each of them induces a different division. Herein,we consider the lexicographic order and the graded lexicographic order.

We first introduce the notion of lexicographic order on monomials in thevariables x1, . . . , xn, similar to the usual ordering of words in a dictionary.

Definition 3.2.10 The lexicographic order >lx on monomials in x1, . . . , xn issuch that (xα1

1 . . . xαnn , xβ1

1 . . . xβnn ) ∈>lx, denoted by,

xα11 . . . xαn

n >lx xβ1

1 . . . xβn

n ,

if there is 1 ≤ i ≤ n such that αi > βi and αj = βj for all 1 ≤ j < i.

As usual, given two monomials m and m′ we write m 6>lx m′ to denote thatit is not the case that m >lx m′ and we write m ≥lx m′ to denote that m >lx m′

or m = m′.

Example 3.2.11 Let us consider monomials in x1, x2, x3, x4. Then


• x31x

22x3x

24 >lx x2

1x42x

23x

54 taking i = 1;

• x1x54 6>lx x1x2x

24 since the only exponent of a variable in x1x

54 that is greater

than the corresponding one in x1x2x24 is the exponent of x4 and the expo-

nents of x2 in the two polynomials are not equal.

Note that we assume that in a monomial the powers of the variables alwaysoccur by the order of the variables in x1, . . . , xn. For instance, by Definition 3.2.1,x3

2x21 is not a monomial in x1, x2. This property of monomials has been implicitly

used in the definition of >lx above.As a side comment, we remark that it is not mandatory to impose a particular

order when defining monomials, that is, both x21x

32 and x3

2x21 can be considered

monomials in x1, x2. Then, to introduce a definition of lexicographic order onmonomials as the one above, an ordering of the variables has to be previouslyfixed and the monomials written accordingly. To obtain the order >lx above thevariable ordering is, of course, x1 > . . . > xn. But, different orderings on thevariables can also be considered, leading to different lexicographic orderings ofmonomials.

The order >lx on monomials can be used for defining division of polynomials,since it satisfies the requirements above.

Proposition 3.2.12 The order >lx is a monomial order.

Proof: The order >lx is total and well founded (Exercise 7 in Section 3.5).We now prove that >lx is preserved by the product of monomials. Assumethat mi = xα1i

1 . . . xαnin , for i = 1, 2 are monomials such that m1 >lx m2. Let

m = xβ1

1 . . . xβnn . Then, there is 1 ≤ j ≤ n such that αj1 > αj2 and αi1 = αi2

for all 1 ≤ i ≤ i. Since

m×mi = xα1i+βi

1 . . . xαni+βn

n

for i = 1, 2, and αj1 + βj > αj2 + βj and αi1 + βi = αi2 + βi for all 1 ≤ i ≤ i, wealso have m×m1 >lx m×m2. QED

Next, we introduce the graded lexicographic order. This order also takes intoaccount the degree of monomials.

Definition 3.2.13 The graded lexicographic order on monomials in x1, . . . , xn

is denoted by >glx and defined as follows: given the monomials m and m′

m >glx m′

whenever one of the following conditions holds:

• deg(m) > deg(m′)


• deg(m) = deg(m′) and m >lx m′.

We again write m 6>glx m′ to denote the fact that it is not the case thatm >glx m′.

Example 3.2.14 Let us consider again monomials in x1, x2, x3, x4. Then

• x1x54 >glx x1x2x

24 since deg(x1x

54) > deg(x1x2x

24);

• x21x

32x3 >glx x2

1x22x

24 since

(i) deg(x21x

32x3) = deg(x2

1x22x

24) = 6;

(ii) x21x

32x3 >lx x2

1x22x

24.

Proposition 3.2.15 The order >glx is a monomial order.

Proof: The order >glx is total and well founded (Exercise 7 in Section 3.5). Let usprove that >glx is preserved by the product of monomials. Let mi = xα1i

1 . . . xαnin ,

for i = 1, 2, and consider m = xβ1

1 . . . xβnn . Assume m1 >glx m2. Then, either

(i) deg(m1) > deg(m2) or (ii) deg(m1) = deg(m2) and m1 >lx m2. We have that

m×mi = xα1i+βi

1 . . . xαni+βnn and deg(m×mi) = deg(m) + deg(mi)

for i = 1, 2. In case (i), clearly deg(m1 + m) > deg(m2 + m). In case (ii), we havedeg(m1 + m) = deg(m2 + m) and, by Proposition 3.2.12, m × m1 >lx m × m2.Hence, we conclude that m×m1 >glx m×m2. QED

Observe that since >lx is a total order, given any finite set M of monomialswe can determine the maximum max(M) with respect to >lx. Moreover, since>lx is also well founded we can determine the minimum min(M) of any set M ofmonomials. Similarly with respect to >glx .

Any order > on monomials induces an order on the set of terms of a poly-nomial. Term coefficients are irrelevant and just the monomials are compared.Hence, t > t′ whenever mt > mt′ where mt and mt′ are the monomials of t andt′, respectively. The properties of monomial orders clearly extend to the ordersinduced on the terms of a polynomial.

The Mathematica function monorderQ in Figure 3.6 receives as input a termm1, a term m2 and a positive integer n, and returns a Boolean value. It returnsTrue if m1 >glx m2 and False otherwise. The function monorderQ uses the func-tion degmon already presented in Figure 3.4 and the function index depicted inFigure 3.5.

The function index receives as input a term q and a positive integer n. Itreturns the index of the first variable in the monomial of q that has a nonzero


exponent. Assuming that q is cxα11 . . . xαm

m , the function index first creates thelist c, xα1

1 , . . . , xαmm and then removes c. Afterwards, it creates the list with

the indexes of the variables with a nonzero exponent and then returns its firstelement.

index=Function[q,n,Module[p,w,p=PolynomialMod[q,n];

If[Head[p]===Times,w=Apply[List,p],w=p];If[NumberQ[First[w]],w=Rest[w]];

First[

Map[Function[m,

If[Head[m]===Power,m[[1,2]],m[[2]]]],w]]]];

Figure 3.5: Index of the first variable with a nonzero exponent

The function monorderQ is recursively defined. Given the input terms m1 andm2, it first tests if they are equal returning False if this is the case. Other-wise, it compares their degrees returning True if deg(m1) > deg(m2) and False

if deg(m1) < deg(m2). When deg(m1) = deg(m2), the function returns True if theindex of the first variable in m1 with a nonzero exponent is less than the index ofthe first variable in m2 with a nonzero exponent. If it is greater it returns False.Otherwise, the function decrements by 1 the exponents of these variables andrecursively checks the the resulting terms.

monorderQ=Function[m1,m2,n,If[PolynomialMod[m1,n]===PolynomialMod[m2,n],False,

If[degmon[m1,n]>degmon[m2,n],True,

If[degmon[m1,n]<degmon[m2,n],False,

If[index[m1,n]<index[m2,n],True,

If[index[m1,n]>index[m2,n],False,

monorderQ[m1/xindex[m1,n],m2/xindex[m1,n],

n]]]]]]];

Figure 3.6: Checking whether m1 is greater than m2

In the sequel, when presenting a polynomial in C[x1, . . . , xn] as

s∑

i=1

ti or t1 + . . . + ts


where t1, . . . , ts are the nonzero terms of the polynomial, we will often assumethat this presentation is ordered, that is, ti > ti+1 for all 1 ≤ i < s. Hence, termsoccur according to their ordering (induced by the monomial order > we areconsidering).

Example 3.2.16 The presentation of the polynomial in R[x1, x2, x3]

6x13 + 5x1x2 + 3x1

2x2 + 2x1x22 + 4x1

2x3 + x2x3 + 2x1x2x3 + 6x22x3

is not ordered. Its ordered presentation is

6x13 + 3x1

2x2 + 4x12x3 + 2x1x2

2 + 2x1x2x3 + 6x22x3 + 5x1x2 + x2x3

The next notions are useful in the sequel. Recall that each polynomial has afinite number of nonzero terms and that all the monomial orders we are consid-ering induce a a total order on the terms of a polynomial.

Definition 3.2.17 Given a nonzero polynomial p in C[x1, . . . , xn] the leadingterm of p, denoted by

lt(p)

is the nonzero term t of p such that t > t′ for each nonzero term t′ of p distinctfrom t. The polynomial p is said to be monic if lt(p) is a monic term.

Example 3.2.18 The leading term of the polynomial in R[x1, x2, x3]

6x13 + 3x1

2x2 + 4x12x3 + 2x1x2

2 + 2x1x2x3 + 6x22x3 + 5x1x2 + x2x3

is 6x13. This polynomial is not monic.

The Mathematica function polsort presented in Figure 3.7 receives as inputa polynomial q and a positive integer n, and returns the ordered list of the termsof q. The integer n indicates that we are considering polynomials over Zn. Thefunction first creates the list of the terms of q and then orders this list using thefunction monorderQ (see Figure 3.6).

The Mathematica function lt in Figure 3.8 receives as input a polynomial p ana positive integer n, and returns the leading term of p. The function polsort isused to get the ordered list of the terms of p.

3.2.3 Division of terms and polynomials

In this section we introduce division of terms an division of (ordered) polynomialsin x1, . . . , xn, for some n ∈ N, over a field C. We consider the >glx order onmonomials.


polsort=Function[q,n,Module[p,p=PolynomialMod[q,n];

If[Head[p]===Plus,

Sort[Apply[List,p],

Function[h1,h2,monorderQ[h1,h2,n]]],p]]];

Figure 3.7: Ordered list of the terms of q

lt=Function[p,n,First[polsort[p,n]]];

Figure 3.8: Leading term of a polynomial

Terms

We start with the notion of divisibility of terms.

Definition 3.2.19 Let t1 and t2 be terms in x1, . . . , xn over C where t2 is anonzero term. We say that t1 is divisible by t2, or that t2 divides t1, wheneverthere is a term t in x1, . . . , xn over C such that

t1 = t× t2.

The term t is the quotient of the division of t1 by t2 and it is denoted byt1t2

.

As we will see below, when t1 is divisible by t2 the term t is unique, hence theabove notion of quotient is well defined. We can also say that t1 is a multiple oft2 whenever t1 is divisible by t2.

Example 3.2.20 Let us consider terms in x1, x2 over Z5. Then

• t1 = 3x31x

22 is divisible by t2 = 2x1x

22 since t1 = 4x2

1 × t2;

• t1 = x1x42 is not divisible by t2 = x2

1x2 since the exponent of x1 is 1 in t1and 2 in t2 and therefore there is no term t such that t1 = t× t2.

When a term t1 is divisible by a given nonzero term t2 then deg(t1) ≥ deg(t2).However, if t1 is not divisible by t2 then it may be the case that deg(t1) < deg(t2)but it is also possible that deg(t1) = deg(t2) or even deg(t1) > deg(t2).

Clearly, a zero term is divisible by any term. A nonzero term t1 is divisible bya nonzero term t2 if and only if the exponent of each variable in the t1 monomialis greater or equal to the exponent of that variable in the t2 monomial. Hence,


divisibility of nonzero terms only depends on the monomials of the terms. Recallthat nonzero term coefficients are nonzero elements of a field C and therefore havemultiplicative inverse.

Lemma 3.2.21 Let ti = cixα1i

1 . . . xαnin , for i = 1, 2, be two nonzero terms in

x1, . . . , xn over C.

1. t1 is divisible by t2 if and only if αj1 ≥ αj2 for every 1 ≤ j ≤ n.

2. If t1 is divisible by t2 then t = (c1 × c−12 ) xα11−α12

1 . . . xαn1−αn2n is the only

term in x1, . . . , xn over C such that t1 = t× t2.

Proof: Note that c1, c2 6= 0.

1. Assume t1 = t× t2 with t = c xβ1

1 . . . xβnn . Thus,

c1xα111 . . . xαn1

n = (c× c2) xβ1+α12

1 . . . xβn+αn2n .

As a consequence, αj1 = βj + αj2 for every 1 ≤ j ≤ n. Since, βj ≥ 0 for every1 ≤ j ≤ n, then αj1 ≥ αj2 for every 1 ≤ j ≤ n.

Conversely, we have that αj1 − αj2 ≥ 0 for every 1 ≤ j ≤ n and conclude thatt1 = t× t2 considering

t = (c1 × c−12 ) xα11−α12

1 . . . xαn1−αn2n .

2. Clearly, we have that t1 = t × t2. Assume now that also t1 = t′ × t2 witht′ = c′xγ1

1 . . . xγnn . Then,

t1 = c1xα111 . . . xαn1

n = (c′ × c2) xγ1+α12

1 . . . xγn+αn2n .

Hence, αj1 = γj + αj2, that is, γj = αj1 − αj2 for every 1 ≤ j ≤ n. Moreover,c1 = c′ × c2 and therefore c1 × c−1

2 = c′ × c2 × c−12 , that is, c1 × c−1

2 = c′. QED

The Mathematica function divisibleQ presented in Figure 3.9 checks whethera term is divisible by another term in the ring of polynomials over Zn. It re-ceives as input a term t1, a term t2 and a positive integer n, and returns aBoolean value. It returns True if t1 is divisible by t2 and False otherwise.The function first checks whether t1 = 0, returning True if this is the case.Otherwise, it checks whether t2 = 0 returning False is this is the case. Other-wise, assuming that t1 = c1x

α111 . . . xα1m

m and t2 = c2xα211 . . . xα2m

m , it creates thelist c1/c2, x

α11−α211 , . . . , xα1m−α2m

m and returns True if all the exponents α1i − α2i

are nonnegative and False otherwise.


divisibleQ=Function[t1,t2,n,Module[r1,r2,r1=PolynomialMod[t1,n];

r2=PolynomialMod[t2,n];

If[r1===0,True,

If[r2===0,False,

If[Head[r1/r2]===Times,

Length[Select[Apply[List,r1/r2],

Function[p,If[Head[p]===Power,

p[[2]]<0,False]]]]==0,

If[Head[r1/r2]===Power,

(r1/r2)[[2]]>0,True]]]]]];

Figure 3.9: Checking whether term t1 is divisible by term t2

Polynomials

We now consider division of polynomials. In the sequel, presentations such as∑s

i=1 ti or t1 + . . . + ts we are always assumed to be ordered. The >glx orderon monomials is assumed by default. To simplify the presentation we denote bygdt(p, t) the greatest nonzero term of a polynomial p that is divisible by a termt.

Proposition 3.2.22 Let p and d be nonzero polynomials in C[x1, . . . , xn]. Thenp can be written as

p = q × d + r

where q and r are polynomials in C[x1, . . . , xn], and r = 0 or r is a nonzeropolynomial whose nonzero terms are not divisible by lt(d). Furthermore, q andr are unique.

Proof (sketch): Let p and d be nonzero polynomials in C[x1, . . . , xn]. To get thepolynomials q and r we build a suitable sequence p0, . . . , pk of polynomials wherep0 = p, at least one nonzero term of pi is divisible by lt(d) for each 0 ≤ i < k, andpk = 0 or there are no nonzero terms of pk divisible by lt(d). Then r = pk and qis obtained from d and the polynomials in the sequence. More precisely, considera finite sequence of polynomials

p0, . . . , pk

with k ∈ N0, such that

• p0 = p;


• for each 0 ≤ i < k

lt(d) divides at least one nonzero term of pi and

pi+1 = pi −gdt(pi, lt(d))

lt(d)× d;

• pk = 0 or pk is a nonzero polynomial whose nonzero terms are not divisibleby lt(d).

Then, let

q =

0 if k = 0k−1∑

i=0

gdt(pi, lt(d))

lt(d)if k > 0

and r = pk.

We can prove that indeed p = q× d + r and that q and r are unique in the abovesense. We do not present herein the details of the proof and refer the reader to[10]. QED

The steps described above to get polynomials q and r such that p = q × d + rare the starting point for the division algorithm. The polynomial d is the divisor.Proposition 3.2.22 ensures that such q and r are unique when we assume that r =0 or r is a nonzero polynomial whose nonzero terms are not divisible by lt(d).Then q is said to be the quotient of the division of p by d and r is said to be theremainder of the division of p by d.

If r is a nonzero polynomial then p is not divisible by d. Otherwise, p is divis-ible by d. Clearly, any zero polynomial p is divisible by any nonzero polynomiald, since p = 0× d.

Example 3.2.23 Let us consider the polynomials

p = 3x31x

32 + 5x2

1x32 − 6x3

1x2 + 2x21x

22 − 10x2

1x2 − 4x21

d = x1x22 − 2x1

in R[x1, x2]. We have that p = q × d + r where

q = 3x21x2 + 5x1x2 + 2x1 and r = 0

Hence, p is divisible by d. Following the division algorithm described above, toget the polynomials q and r we proceed as follows (note that the leading term ofany ordered polynomial is always its greatest nonzero term):

i) lt(p) is divisible by lt(d) thus

lt(p)

lt(d), that is 3x2

1x2, is a term of q,


p1 = p− lt(p)

lt(d)× d = 5x2

1x32 + 2x2

1x22 − 10x2

1x2 − 4x21.

ii) lt(p1) is divisible by lt(d) thus

lt(p1)

lt(d), that is 5x1x2, is a term of q,

p2 = p− lt(p1)

lt(d)× d = 2x2

1x22 − 4x2

1.

iii) lt(p2) is divisible by lt(d) thus

lt(p2)

lt(d), that is 2x1, is a term of q,

p3 = p− lt(p2)

lt(d)× d = 0.

The above computations can also be presented in the following way where, forillustration purposes, we have also included the names of the polynomials:

p 3x31x

32 + 5x2

1x32 − 6x3

1x2 + 2x21x

22 − 10x2

1x2 − 4x21 x1x

22 − 2x1 d

3x21x2 + 5x1x2 + 2x1 q−3x3

1x32 + 6x3

1x2

p1 5x21x

32 + 2x2

1x22 − 10x2

1x2 − 4x21

−5x21x

32 + 10x2

1x2

p2 2x21x

22 − 4x2

1

−2x21x

22 + 4x2

1

p3 0

We now present another example.

Example 3.2.24 Let us consider the polynomials

p = 2x41x

32x

23 + 2x3

2x53 + 3x2

1x22x

33 + x2

1x32x3 + 4x2

2x23

d = 4x21x3 + 2

in Z5[x1, x2, x3]. We have that p = q × d + r where

q = 3x21x

32x3 + 2x2

2x23 and r = 2x3

2x53

since using the division algorithm we get


4x21x3 + 2

3x21x

32x3 + 2x2

2x233x4

1x32x

23 + 4x2

1x32x3

2x41x

32x

23 + 2x3

2x53 + 3x2

1x22x

33 + x2

1x32x3 + 4x2

2x23

2x32x

53 + 3x2

1x22x

33 + 4x2

2x23

2x21x

22x

33 + x2

2x23

2x32x

53

Recall that in Z5 the equalities −2 = 3 and −1 = 4 hold and the equality 3+2 = 0also holds.

It is worthwhile noticing that when we divide a nonzero polynomial p by anonzero polynomial d the uniqueness of the quotient and remainder polynomialsdepends on the particular ordering of monomials we are considering. Differentorders may lead to different quotients and remainders, since the leading term ofd depends on the particular monomial order. The following example illustratesthis situation.

Example 3.2.25 Let us consider the polynomial p = 4x21x

22 + 2x3

2 and the poly-nomial d = x2

2 + x1 in R[x1, x2]:

• assuming the order >glx we have p = q × d + r where

q = 4x21 + 2x2,

r = −4x31 − 2x1x2,

since using the division algorithm we get the following

x22 + x1

4x21 + 2x2−4x2

1x22 − 4x3

1

4x21x

22 + 2x3

2

−4x31 + 2x3

2

−2x32 − 2x1x2

−4x31 − 2x1x2.

• assuming the order >lx we get p = q × d + r where

q = 4x1x22 − 4x4

2,

r = 4x62 + 2x3

2,

given that we now have

x1 + x22

4x1x22 − 4x4

2−4x1x22 − 4x1x

42

4x21x

22 + 2x3

2

−4x1x42 + 2x3

2

4x1x42 + 4x6

2

4x62 + 2x3

2.


Note that the ordered form of d is different in both cases, thus the leadingterm is also different.

3.2.4 Reduction modulo a set of polynomials

We now discuss the reduction of a polynomial modulo a set of polynomials sothat we can define certain polynomials as a combination

a1 × d1 + . . . am × dm

of a given finite set of polynomials d1, . . . , dm, where a1, . . . , am are also poly-nomials. We start by considering the one step reduction.

Definition 3.2.26 Let p and d be nonzero polynomials in C[x1, . . . , xn]. Then preduces to p′ modulo d in one step, written

pd−→ p′,

if

p′ = p− t

lt(d)× d

for some nonzero term t of p divisible by lt(d).

Example 3.2.27 Let us consider the polynomials p and d in R[x1, x2] where

p = 3x31x

32 + 5x2

1x32 − 6x3

1x2 + 2x21x

22 − 10x2

1x2 − 4x21

d = x1x22 − 2x1

Since lt(d) divides lt(p) and

p− lt(p)

lt(d)× d = 5x2

1x32 + 2x2

1x22 − 10x2

1x2 − 4x21

we have that

pd−→ 5x2

1x32 + 2x2

1x22 − 10x2

1x2 − 4x21.

However lt(d) also divides the term 5x21x

32 of p and therefore, since

p− 5x21x

32

lt(d)× d = 3x3

1x32 − 6x3

1x2 + 2x21x

22 − 4x2

1

we can also conclude that

pd−→ 3x3

1x32 − 6x3

1x2 + 2x21x

22 − 4x2

1.


Another reduction of p modulo d is also possible given that the term 2x21x

22 is also

divisible by lt(d).Note that the first reduction above is just the polynomial p1 we got in the

first step of the division algorithm in Example 3.2.23. It is easy to see that if wereduce p1 modulo d in one step using the term lt(p1) we get the polynomial p2

therein. Similarly, we get 0 by reducing p2 modulo d.

When reducing p modulo d in one step, we always get rid of the term t divisibleby lt(d) that we choose to compute the reduction. This term is replaced in p bya multiple of the polynomial that results from d by removing lt(d). It is also easyto conclude that the reduction of p modulo d in one step may correspond to astep of the division algorithm presented in Section 3.2.3. The only difference isthat in the division algorithm we always choose the greatest nonzero term thatis divisible by the leading term of d and herein we can choose any term divisibleby the leading term of d.

The following result is useful in the sequel.

Lemma 3.2.28 Let p and d be nonzero polynomials in C[x1, . . . , xn]. Assumethat t is a term of p divisible by lt(d) and that p′ = p − t

lt(d)× d is a nonzero

polynomial. Then, lt(p) >glx lt(p′) whenever t = lt(p) and lt(p) = lt(p′) whenevert 6= lt(p).

Proof: Let p =∑rp

i=1 tpi and d =∑rd

i=1 tdi . Hence,

p′ =

rp∑

i=1

tpi −rd∑

i=1

(t

lt(d)× tdi

)

.

Since p and d are ordered polynomials, lt(p) = tp1, lt(d) = td1. Moreover, given1 < i ≤ rp and 1 < i′ ≤ rd we have that lt(p) >glx tpi and lt(d) >glx tdi′ . In partic-ular, by Proposition 3.2.15,

t

lt(d)× lt(d) >glx

t

lt(d)× tdi

that is,

t >glxt

lt(d)× tdi

for all 1 < i ≤ rd. We consider two cases.

(1) t = lt(p). The terms tp1 and tlt(d)× td1 cancel each other and

p′ =

rp∑

i=2

tpi −rd∑

i=2

(t

lt(d)× tdi

)

.


Since lt(p) >glx tpi for all 1 < i ≤ rp and lt(p) >glxt

lt(d)× tdi for all 1 < i ≤ rd, we

conclude that lt(p) >glx lt(p′).

(2) t 6= lt(p). Then, t = tpj for some j > 1 and, when computing p′, the terms tpjand t

lt(d)× lt(d) cancel each other. Moreover, tpk >glx t for all 1 ≤ k < j and, as a

consequence,

tpk >glxt

lt(d)× tdi

for all 1 < i ≤ rd. Hence, no term tpk, with 1 ≤ k < j, cancels with a termt

lt(d)× tdi , with 1 < i ≤ rd, and therefore tpk is a term of p′ for all 1 ≤ k < j. We

can then conclude that lt(p′) = tp1 = lt(p). QED

Note that Lemma 3.2.28 also holds when polynomials are ordered using theorder >lx instead of >glx. Clearly, in this case we have that

lt(p) >lx lt(p′).

selterm=Function[q,u,n,Module[p,t,w,r,i,p=PolynomialMod[q,n];

t=PolynomialMod[u,n];

If[Head[p]===Plus,w=Apply[List,p],w=p];

r=0;

i=1;

While[i<=Length[w]&&r===0,

If[divisibleQ[w[[i]],t,n],r=w[[i]]];

i=i+1];

r]];

Figure 3.10: Selecting a term of q divisible by u

The Mathematica function redone in Figure 3.11 receives as input two poly-nomials f and g and a positive integer n. It returns as output the polynomial thatresults from a reduction of f modulo g in one step. The integer n indicates thatwe are considering polynomials over Zn. Besides the function lt (see Figure 3.8),it uses the Mathematica function selterm in Figure 3.10.

The function selterm receives as input a polynomial q, a term u and a positiveinteger n. It returns a term of q that is divisible by u, if such a term exists, and0 otherwise. The function first creates the list of the terms of q and then passesthrough the list using the function divisibleQ (see Figure 3.9) to pick the firstterm divisible by u. When there is no such term it returns 0.


To obtain a reduction of f modulo g in one step, the function redone usesselterm and lt to select a term of f divisible by the leading term of g and thencomputes the reduction as expected.

redone=Function[f,g,n,Expand[PolynomialMod[f-

(selterm[f,lt[g,n],n]/lt[g,n])*g,n]]];

Figure 3.11: One step reduction

The notion of reduction can now be introduced capitalizing on the one stepreduction.

Definition 3.2.29 Let p be a nonzero polynomial and D a finite set of nonzeropolynomials in C[x1, . . . , xn]. Then, p reduces to p′ modulo D, written

pD−→ p′,

if there is a sequence

p0, . . . , pm

where m ∈ N0, such that

• p0 = p and pm = p′;

• pid−→ pi+1 for some d ∈ D, for each 0 ≤ i < m.

When m = 0 we say that p reduces to p′ modulo D in zero steps and, otherwise,p reduces to p′ modulo D in several steps. If p cannot be reduced in one stepmodulo an element of D we say that p is irreducible modulo D or that p isD-irreducible. Clearly, if a nonzero polynomial p is irreducible modulo D thenp /∈ D. (Exercise 15 in Section 3.5).

Example 3.2.30 Let us consider the polynomial p and the set of polynomialsD = d1, d2 in Z7[x1, x2] where

p = 6x42 + 3x2

1x2 + 5x1 d1 = 3x22 + 2 d2 = x2

1

We have that

pD−→ 3x2

2 + 5x1

since pd2−→ 6x4

2 + 5x1d1−→ 3x2

2 + 5x1.


When p reduces to p′ modulo a finite set D of polynomials, then the polyno-mial p− p′ can be expressed in terms of the polynomials in D. In particular, if preduces to 0, then we can express p in terms of the polynomials in the set. Theseproperties are useful later on.

Lemma 3.2.31 Let d1, . . . , dk, k ∈ N0, be nonzero polynomials in C[x1, . . . , xn].Assume that p0, . . . , pm, m ∈ N0, are polynomials in C[x1, . . . , xn] such that, foreach 0 ≤ i < m, pi reduces to pi+1 modulo dj for some 0 ≤ j ≤ k. Then,

p0 − pm =

k∑

i=1

aidi

where ai is a polynomial in C[x1, . . . , xn] for each 1 ≤ i ≤ k.

Proof: The proof is by induction on m.

Basis: m = 0. In this case, pm = p0 and therefore

p0 − pm = 0 =

k∑

i=1

0di.

Step: Let p0, . . . , pm+1 be such that, for each 0 ≤ i < m + 1, pi

dj−→ pi+1 for some1 ≤ j ≤ k. By the induction hypothesis,

p0 − pm =k∑

i=1

aidi

where ai is a polynomial in C[x1, . . . , xn] for each 1 ≤ i ≤ k. Since pm

dj−→ pm+1

for some 1 ≤ j ≤ k, then

pm+1 = pm −t

lt(dj)× dj

for some term t of pm. Hence,

p0 − pm+1 = p0 −(

pm −t

lt(dj)× dj

)

= (p0 − pm) +t

lt(dj)× dj =

k∑

i=1

a′idi

where a′i = ai +

t

lt(dj)if i = j and a′

i = ai otherwise, for all 1 ≤ i ≤ k. QED

Proposition 3.2.32 Let p and d1, . . . , dk, k ∈ N0, be a nonzero polynomials in

C[x1, . . . , xn]. If pd1,...,dk−−−−→ p′ then

p− p′ =

k∑

i=1

aidi

where ai is a polynomial in C[x1, . . . , xn] for each 1 ≤ i ≤ k.


Proof: By Definition 3.2.29 and Lemma 3.2.31. QED

The proof of Lemma 3.2.31 sketches a technique for obtaining polynomials ai

such that p − p′ =∑n

i=1 ai × di whenever p reduces to p′ modulo d1, . . . , dn.We present in the sequel an illustrative example.

Example 3.2.33 Consider the polynomials in Z5[x1, x2]

d1 = 2x21x2 + x2 d2 = x3

2 + x1 d3 = 4x31 + 2x1

andp = 4x3

1x2 + 4x41 + x1x

32 + 3x2

1 + 2x1x2.

Assuming D = d1, d2, d3, we have that

pD−→ 0

sincep

d3−→ 4x41 + x1x

32 + 3x2

1d3−→ x1x

32 + x2

1d2−→ 0.

These reductions allow us to express p in terms of the polynomials in D. In fact,assuming that p1 = 4x4

1 + x1x32 + 3x2

1 and p2 = x1x32 + x2

1 we have

• pd3−→ p1 where p1 = p− lt(p)

lt(d3)× d3 = p− x2 × d3;

• p1d3−→ p2 where p2 = p1 −

lt(p1)

lt(d3)× d3 = p1 − x1 × d3

• p2d2−→ 0 where 0 = p2 −

lt(p2)

lt(d2)× g2 = p2 − x1 × d2.

Therefore,p = p1 + x2 × d3

= p2 + x1 × d3 + x2 × d3

= x1 × d2 + x1 × d3 + x2 × d3

,

that isp = x1 × d2 + (x1 + x2)× d3,

orp = 0× d1 + x1 × d2 + (x1 + x2)× d3

when considering all the polynomials in D.

The Mathematica function red in Figure 3.12 receives as input two polynomi-als f and g and a positive integer n. It returns a polynomial irreducible modulog. The function repeatedly uses the function redone (see Figure 3.11) to reducef modulo g in one step until an irreducible polynomial is obtained.


red=Function[f,g,n,FixedPoint[Function[h,redone[h,g,n]],f]];

Figure 3.12: Reduction of polynomial f modulo g

The Mathematica function redmod in Figure 3.13 receives as input a polyno-mial f, a list of polynomials G and a positive integer n. It returns a polynomialG-irreducible that results from the reduction of f modulo G. This function extendsthe function red to a set of polynomials G = g1, . . . , gm. It is the fixed point ofthe function that, given a polynomial h, obtains a polynomial h1 by reducing h

modulo g1, a polynomial h2 by reducing h1 modulo g2 and so on, finally returninghm.

redmod=Function[f,G,n,FixedPoint[

Function[h,Module[i,r,r=h;

i=1;

While[i<=Length[G],

r=red[r,G[[i]],n];

i=i+1];

r]],

f]];

Figure 3.13: Reduction of polynomial f modulo G

3.3 Grobner bases

In this section we introduce the notion of Grobner basis and some related conceptsand properties. Grobner bases were first introduced by B. Buchberger [7, 8] andwere originally proposed as an algorithmic solution to some important problemsin polynomial ring theory and algebraic geometry (for more technical details see[1]). Since then many applications and generalizations have been proposed. Theinterested reader can consult [30, 5, 32, 6, 27].

Grobner bases are often seen as a multivariate generalization of the Euclideanalgorithm for computing the greatest common divisor for univariate polynomials.

3.3. GROBNER BASES 91

They are also seen as a nonlinear generalization of the Gauss elimination algo-rithm for linear systems. In Section 3.4.2 we discuss how to use Grobner basesto solve systems of nonlinear polynomial equations.

3.3.1 Ring ideals

In a nutshell, the idea behind the Grobner basis approach is as follows. Givena (finite) set S of polynomials in some ring of polynomials (that depends on theparticular problem at hand), S can be transformed into a set G of polynomials(the Grobner basis) which is equivalent to S in a sense to be discussed later on.This new set G satisfies some good properties and, as a consequence, many prob-lems and questions that were difficult to handle when considering the arbitraryset S become easier when we work with G. Since the transformation from Sto G can be algorithmically performed, many problems involving finite sets ofpolynomials become algorithmically solvable.

A typical example of this situation is the ideal membership problem. Theideal membership problem consists in checking whether or not some polynomialis a member of the ideal of polynomials generated by a given finite set S of poly-nomials. In general, this problem is not easy to solve. However, if we transformS into a Grobner basis G that generates the same ideal the problem is easilysolved.

There are several concepts that have to be presented before defining Grobnerbases. In this section we introduce ideals, ideals of polynomials and some relevantrelated notions and properties. In Section 3.3.2 we then discuss Grobner bases.

An ideal is a subset of a ring with some interesting properties. Using idealsimportant properties of integer numbers can be generalized to other rings. Forexample, prime ideals and coprime ideals can be defined as a generalization ofprime and coprime numbers and there is even a Chinese remainder theorem in-volving ideals. Herein we do not detail this subject. The interested reader isreferred to [20].

Definition 3.3.1 An ideal I over a commutative ring (A, +, 0,−,×) is a nonemptysubset of A such that for all a1, a2 ∈ A

• if a1, a2 ∈ I then a1 + a2 ∈ I;

• if a1 ∈ A and a2 ∈ I then a1 × a2 ∈ I.

Observe that 0 ∈ I for every ideal I over (A, +, 0,−,×) (Exercise 8 in Section3.5).

We are mainly concerned with ideals of polynomials, that is, ideals over ringsof polynomials (hence over unitary commutative rings).


Example 3.3.2 Consider the ring Z of integer numbers with the usual opera-tions and n ∈ Z. The set of the multiples of n is an ideal over Z since (i) this setis not empty, (ii) the sum of multiples of n is a multiple of n and (iii) the productof an integer number by a multiple of n is again a multiple of n.

Example 3.3.3 Let (A, +, 0,−,×) be a unitary commutative ring. It is easy toconclude that

• 0 and A are ideals over A

• if 1 ∈ I then I = A.

In the second case note that given any a ∈ A we have that a× 1 = a ∈ I.

We say that an ideal I over A is proper whenever I 6= A.

Example 3.3.4 Let (A, +, 0,−,×) be a unitary commutative ring. It is easy toconclude that given g1, . . . , gk ∈ A, with k ∈ N, the set

p ∈ A : p = a1 × g1 + . . . + ak × gk with a1, . . . , ak ∈ A

is an ideal over A. This ideal is usually denoted by (g1, . . . , gk).

Definition 3.3.5 Let I be an ideal over a commutative unitary ring (A, +, 0,−,×).A set G ⊆ I is a set of generators of I if for any p ∈ I there exist g1, . . . , gk ∈ Gand a1, . . . , ak ∈ A, k ∈ N, such that

p = a1 × g1 + . . . + ak × gk.

When G ⊆ I is a set of generators of I, we also say that I is generated by G. Anideal I is finitely generated if it has finite set G of generators. Then, the set G isa basis of I.

Example 3.3.6 The set Ze of even integer numbers is an ideal over Z (Exercise9 in Section 3.5). The set 2 is a set of generators of Ze. Indeed any evennumber d ∈ Z can be written as d = a× 2 for some a ∈ Z. Hence, Ze is finitelygenerated and 2 is a basis of Ze.

Proposition 3.3.7 Let (A, +, 0,−,×) be a unitary commutative ring and con-sider g1, . . . , gk ∈ A, with k ∈ N. Then

(i) g1, . . . , gk is a basis of the ideal (g1, . . . , gk)

(ii) if g1, . . . , gk ⊆ I is a set of generators of an ideal I then I = (g1, . . . , gk).


Proof: Recall the ideal (g1, . . . , gk) from Example 3.3.4. Statement (i) followseasily from Definition 3.3.5. With respect to statement (ii), note that if p ∈ Ithen

p =m∑

j=1

ag′jg′

j

for some g′1, . . . , g

′m ∈ g1, . . . , gk, where ag′

j∈ A for every 1 ≤ j ≤ m. Hence,

p =k∑

i=1

aigi

where ai = agiif gi ∈ g′

1, . . . , g′m and ai = 0 otherwise, for every 1 ≤ i ≤ k. As a

consequence, I ⊆ (g1, . . . , gk). Conversely, let p =∑k

i=1 aigi ∈ (g1, . . . , gk). Sinceg1, . . . , gk ∈ I, then using the definition of ideal, it is easy to conclude that alsop ∈ I, and therefore (g1, . . . , gk) ⊆ I. QED

The following result states that every ideal over a ring of polynomials is gen-erated by some finite set of polynomials.

Proposition 3.3.8 Every ideal I over C[x1, . . . , xn] is finitely generated.

In fact, if I = 0 then I = (0). Otherwise, letting J be the ideal generatedby the leading terms of the polynomials in I, there is a finite set g1, . . . , gr ⊆ Isuch that J = (lt(g1), . . . , lt(gr)), that is, J is finitely generated. Moreover,I = (g1, . . . , gr) and therefore I is finitely generated. For the details of the proofwe refer the reader to [10], for instance.

When G is a basis of an ideal I of polynomials and pG−→ p′, then, using

Proposition 3.2.32, it is easy to conclude that p − p′ ∈ I and that p ∈ I if and

only if p′ ∈ I. In particular, if pG−→ 0, then p ∈ I. These properties are useful

in the sequel.

Proposition 3.3.9 Let G be a basis of an ideal I over C[x1, . . . , xn] and let p bea polynomial in C[x1, . . . , xn].

1. If pG−→ p′ then p− p′ ∈ I and p ∈ I if and only if p′ ∈ I.

2. If 1 ∈ G then pG−→ 0.

Proof:1. The fact that p− p′ ∈ I follows from Proposition 3.2.32 and Definition 3.3.5. Ifp′ ∈ I then, since p− p′ ∈ I and I is an ideal, we also have p = p′ + (p− p′) ∈ I.The converse is similar.


2. If p is 0 the result is immediate. Otherwise, note that lt(1) divides lt(p) andtherefore

p1−→ p− lt(p)

lt(1)× 1.

That is p1−→ p− lt(p). Hence, we have reduced p to a polynomial that results

from p by removing lt(p). It is then easy to conclude that with a suitable numberof reductions in one step modulo 1 we get 0. A rigorous proof of this result usesinduction on the number of terms of p and is left as an exercise. QED

In the proof of Proposition 3.3.9 we use Proposition 3.2.32 to conclude that

p− p′ ∈ I whenever pG−→ p′. However, note that the proof of Proposition 3.2.32

not only ensures that p− p′ ∈ I but also provides a technique d for expressing

p− p′ in terms of the polynomials in the basis G. In particular, if pG−→ 0, then

we can easily express p in terms of the polynomials of the basis of the ideal.

Example 3.3.10 Consider the polynomials

g1 = 2x21x2 + x2 g2 = x3

2 + x1 g3 = 4x31 + 2x1.

in Z5[x1, x2]. The set G = g1, g2, g3 is a basis of the ideal I = (g1, g2, g3) overZ5[x1, x2]. Consider the polynomial

p = 4x31x2 + 4x4

1 + x1x32 + 3x2

1 + 2x1x2

inZ5[x1, x2]. Recall from Example 3.2.33 that

pG−→ 0.

Hence, by Proposition 3.3.9, we conclude that p− 0 ∈ I and therefore p ∈ I.Moreover, recalling again Example 3.2.33, we have that

p = x1 × g2 + (x1 + x2)× g3,

or even

p = 0× d1 + x1 × d2 + (x1 + x2)× d3

if we want to explicitly consider all the polynomials in the basis.

There is an infinite number of possible bases of an ideal I over a ring C[x1, . . . , xn].Among them we can distinguish the Grobner bases.


3.3.2 Buchberger criterion

Given polynomials g1, . . . , gk in C[x1, . . . , xn] and a polynomial p in C[x1, . . . , xn]it can be a hard task to determine whether or not p ∈ (g1, . . . , gk). Recall that wehave to check whether p = a1×g1+. . .+gk×gk for some a1, . . . , ak in C[x1, . . . , xn].As we have already remarked above, this problem is usually known as the idealmembership problem and it has relevant applications in several domains.

Note that if pG−→ 0, where G = g1, . . . , gk, then Proposition 3.3.9 immedi-

ately ensures that p is an element of (g1, . . . , gk). But, if there is no reduction ofp to 0 modulo G nothing can be concluded from that result.

Grobner bases provide an algorithmic solution for the ideal membership prob-lem. First of all, whenever G is a Grobner basis, then p is in the ideal generatedby G if and only if p can be reduced to the zero polynomial using the polynomialsin G.

Hence, given a set G of generators, there is an algorithm to solve the idealmembership problem if there is algorithm to obtain from G a new set G′ ofgenerators such that G′ is a Grobner basis. In this section we introduce thenotion of Grobner basis and in Section 3.3.3 we will present an algorithm toobtain a Grobner basis of an ideal I from any finite set of generators of I.

As already stated, we consider the order >glx on monomials. However, anyother order on monomials will do provided it is suitable for polynomial division(see Section 3.2.3). The choice of a particular monomial order may depend onthe particular application of Grobner bases we are interested in. Moreover, someproperties of Grobner bases depend on the monomial order we are considering.

In the sequel we always consider ideals of polynomials that are generated bysome given finite basis. This is not a restriction since Proposition 3.3.8 ensuresthat every ideal of polynomials is generated by a finite set of polynomials.

Definition 3.3.11 Let I be an ideal over C[x1, . . . , xn] and let G be a basis of I.The set G is a Grobner basis of I whenever for any nonzero p ∈ I there is g ∈ Gsuch that lt(p) is divisible by lt(g).

Note that if 1 is an element of a basis G of I then G is a Grobner basis..Observe also that given any polynomial g in C[x1, . . . , xn] then g is a Grobnerbasis of the ideal (g) (Exercise 15 in Section 3.5).

The following result constitutes an alternative way of characterizing a Grobnerbasis G of an ideal I.

Proposition 3.3.12 Let I be an ideal over C[x1, . . . , xn] and G a basis of I.

Then G is a Grobner basis of I if and only if pG−→ 0 for all p ∈ I.

Proof:(→) Assume that G is a Grobner basis of I and let p ∈ I. If p = 0, then p can


be reduced to 0 modulo G and we are done. Otherwise, by Definition 3.3.11,there is g′ ∈ G such that lt(p) is divisible by lt(g′). The polynomial p can thenbe reduced to

p′ = p− lt(p)

lt(g)× g.

If p′ = 0, we are done. Otherwise, by Lemma 3.2.28, lt(p) >glx lt(p′) and, byProposition 3.3.9, p′ ∈ I. Similarly, we can now reduce p′ modulo G either to 0or to p′′ ∈ I such that lt(p′) >glx lt(p′′). Since >glx is well founded, at some pointwe will have to reduce to 0, and therefore

pG−→ 0.

(←) We prove the result by contraposition. Assume that G is not a Grobnerbasis. Then there is a nonzero polynomial p ∈ I such that lt(p) is not divisibleby lt(g) for all g ∈ G. If p is irreducible modulo G, since p 6= 0, we conclude thatp cannot be reduced to 0 modulo G. If p reduces to p′ in one step modulo anelement of G, Lemma 3.2.28 ensures that

lt(p′) = lt(p).

Hence p′ 6= 0. Again, lt(p′) is not divisible by lt(g) for all g ∈ G and so we canreason in a similar way. Therefore, it is easy to conclude that it is not possibleto reduce p to 0 modulo G. QED

Although not obvious from the Definition 3.3.11, it is the case that every idealover C[x1, . . . , xn] has a Grobner basis. In fact, it can have many Grobner bases.Reduced Grobner bases play an important role since every polynomial ideal hasonly one reduced Grobner basis (see Proposition 3.3.30).

Definition 3.3.13 A Grobner basis G of an ideal I over the ring C[x1, . . . , xn]is said to be a reduced Grobner basis if for each g ∈ G

• lt(g) is monic;

• lt(g) does not divide the nonzero terms of g′ for all g′ ∈ G\g.

Observe that if g is a monic polynomial then g is a reduced Grobner basisof the ideal (g). Note also that if 1 ∈ G and G is basis such that G 6= 1, thenG is Grobner basis, as we have already remarked, but G is not a reduced Grobnerbasis, since lt(1) divides any term.

Checking whether a set of polynomials G is a Grobner basis using Definition3.3.11, or even Proposition 3.3.12, is often a very hard task. Observe that wehave to check all the polynomials of the ideal. We now present the Buchbergercriterion that provides an alternative way of checking whether G is a Grobner


basis. Using the Buchberger criterion we only have to consider a finite numberof polynomials. We first introduce the notion of least common multiple of twoterms.

Definition 3.3.14 Let t1 = k1xα11 . . . xαn

n and t2 = k2xβ1

1 . . . xβnn be nonzero terms

in x1, . . . , xn over C. The least common multiple of t1 and t2, denoted lcm(t1, t2),is the monic term

xγ1

1 . . . xγn

n

where γi = maxαi, βi for each 1 ≤ i ≤ n.

The Mathematica function polLCM in Figure 3.14 computes the least commonmultiple of two terms using the built-in Mathematica function PolynomialLCM

and the auxiliary function coeflt. The function coeflt receives as input apolynomial p and a positive integer n. It returns the coefficient of the leadingterm of p. The function first checks whether p has only a nonzero term in whichcase it returns its coefficient. Otherwise, it uses the function lt (see Figure 3.8)to obtain the leading term of p. The integer n indicates that we are consideringpolynomials over Zn.

coeflt=Function[p,n,Module[q,q=PolynomialMod[p,n];

If[Head[lt[q,n]]===Power,1,

If[NumberQ[q],q,

If[NumberQ[lt[q,n][[1]]],lt[q,n][[1]],1]]]]];

polLCM=Function[t1,t2,PolynomialLCM[t1/coeflt[t1],t2/coeflt[t2]]];

Figure 3.14: Least common multiple of terms t1 and t2

The function polLCM receives as input two terms t1 and t2, and returnslcm(t1, t2). The terms t1 and t2 are first divided by the corresponding leadingterm coefficient. Then the function PolynomialLCM is used to obtain their leastcommon multiple.

We now present Buchberger polynomials also called S-polynomials.

Definition 3.3.15 Let p1 and p2 be nonzero polynomials in C[x1, . . . , xn]. TheBuchberger polynomial of p1 and p2, denoted B(p1, p2), is the polynomial

B(p1, p2) =lcm(lt(p1), lt(p2))

lt(p1)× p1 −

lcm(lt(p1), lt(p2))

lt(p2)× p2


in C[x1, . . . , xn].

Example 3.3.16 Considering the polynomials in R[x1, x2, x3]

p1 = 3x21x

32x3 + 2x2

1x2 + x2x3 + 1

p2 = 5x32x

43 + 3x2

2x23

then

lcm(lt(p1), lt(p2)) = x21x

32x

43

B(p1, p2) =x2

1x32x

43

3x21x

32x3× p1 −

x21x

32x

43

5x32x

43

× p2

=2

3x2

1x2x33 +

1

3x2x

43 +

1

3x3

3 −3

5x2

1x22x

23.

Observe that the leading terms of the polynomials p1 and p2 are canceled whenwe compute their Buchberger polynomial. Moreover, B(p1, p2) = −B(p2, p1),and, therefore, it is easy to conclude that if B(p1, p2) reduces to 0 modulo a setof polynomials so does B(p2, p1) (Exercise 14 in Section 3.5). Note also that ifp1, p2 ∈ I then B(p1, p2) ∈ I, where I is an ideal of polynomials.

The Mathematica function polBuch in Figure 3.15 receives as input two poly-nomials f and g and a positive integer n. It returns the Buchberger polynomial off and g. The Buchberger polynomial is computed as expected using the functionpolLCM (see Figure 3.14).

polBuch=Function[f,g,n,Expand[PolynomialMod[polLCM[lt[f,n],lt[g,n]]/lt[f,n]*f

-polLCM[lt[f,n],lt[g,n]]/lt[g,n]*g,n]]];

Figure 3.15: Buchberger polynomial of f and g

We can now introduce the Buchberger criterion for checking whether a givenbasis of an ideal is a Grobner basis. The following theorem is known as theBuchberger theorem. We only sketch the proof of the theorem. For the completeproof we refer the interested reader to [10].

Theorem 3.3.17 Let G be a basis of an ideal I over the ring C[x1, . . . , xn]. ThenG is Grobner basis if and only if

B(g, g′)G−→ 0

for all distinct polynomials g, g′ ∈ G.


Proof (sketch):(→) Since B(g, g′) ∈ I for all distinct polynomials g, g′ ∈ G, if G is Grobner basis

then Proposition 3.3.12 ensures that B(g, g′)G−→ 0.

(←) Conversely, we have to prove that for each p ∈ I there is g ∈ G such thatlt(g) divides lt(p). Consider p ∈ I and assume that G consists of k polynomialsg1, . . . , gk. Without loss of generality, assume that all the polynomials in G aremonic. Let A be the set of all tuples (a1, . . . , ak) of polynomials in C[x1, . . . , xn]such that p =

∑ki=1 aigi. For each (a1, . . . , ak) ∈ A let M(a1,...,ak) be the set of

the monomials of lt(aigi) for all 1 ≤ i ≤ k and let m(a1,...,ak) = max(M(a1,...,ak)).Note that either m(a1,...,ak) is the monomial of lt(p) or m(a1,...,ak) >glx lt(p). Themonomial of lt(p) may differ from m(a1,...,ak) because the lt(aigi)’s whose monomialis m(a1,...,ak) may cancel each other. Let M be the set of all m(a1,...,ak) with(a1, . . . , ak) ∈ A and let m = min(M).

As remarked above, either m is the monomial of lt(p) or m >glx lt(p). Usingthe hypothesis that B(g, g′) reduces to 0 modulo G for all distinct g, g′ ∈ G, ifm >glx lt(p) then it would be possible to get a′

1, . . . , a′k ∈ C[x1, . . . , xn] such that

p =∑k

i=1 a′igi and m >glx lt(a′

igi) for each 1 ≤ i ≤ k. But this contradicts theassumption m = min(M) and, as a consequence, we can conclude that m is themonomial of lt(p). Then there is 1 ≤ i ≤ k such that lt(gi) divides lt(p) as re-quired. QED

Taking into account Theorem 3.3.17, to check whether a set of polynomialsis a Grobner basis we just have to consider a finite number of polynomials: theBuchberger polynomials of the pairs of distinct elements of G.


g1 = 2x21x2 + x2 g2 = x3

2 + x1 g3 = 4x31 + 2x1

in Z5[x1, x2]. Let us use the Buchberger criterion to check whether G = g1, g2, g3constitutes a Grobner basis of the ideal (g1, g2, g3):

• B(g1, g2) = 4x31 + 3x3

2

and

4x31 + 3x3

2G−→ 0 since 4x3

1 + 3x32

g2−→ 4x31 + 2x1

g3−→ 0;

• B(g1, g3) = 0;

• B(g2, g3) = x41 + 2x1x

32

and

x41 + 2x1x

32

G−→ 0 since x41 + 2x1x

32

g3−→ 2x1x32 + 2x2

1

g2−→ 0.


Given that B(gi, gj)G−→ 0 for all 1 ≤ i, j ≤ 3, i 6= j, by Theorem 3.3.17, G is

a Grobner basis. It is not a reduced Grobner basis since lt(g1) and lt(g3) are notmonic.

Example 3.3.19 Consider the polynomials g1 and g2 in Z5[x1, x2] presented inExample 3.3.18. Let us use again the Buchberger criterion to check whether theset G = g1, g2 constitutes a Grobner basis of the ideal (g1, g2). We have that

B(g1, g2) =x2

1x32

2x21x2

× g1 −x2

1x32

x32

× g2 = 4x31 + 3x3

2

and

4x31 + 3x3

2G−→ 4x3

1 + 2x1 since 4x31 + 3x3

2

g2−→ 4x31 + 2x1.

Note that the nonzero polynomial 4x31 + 2x1 cannot be reduced modulo g1 or

g2 since neither of its terms are divisible by lt(g1) or lt(g2), respectively. Sincewe have not proven that reductions modulo a set of polynomials are unique, in

the sense that p′ = p′′ whenever pG−→ p′ and p

G−→ p′′ and p′, p′′ are irreduciblemodulo G, we cannot use just yet Theorem 3.3.17 to conclude that G is not aGrobner basis. We have to ensure that there is no other possible reductions ofB(g1, g2) modulo G. It is easy to conclude that this is indeed the case, since lt(g1)does not divide any term of 4x3

1 +3x32 and lt(g2) only divides the term 3x3

2. Thus,we can conclude that B(g1, g2) cannot be reduced to a zero polynomial moduloG. By Theorem 3.3.17, G is not a Grobner basis.

The Mathematica function GrobnerQ in Figure 3.16 receives as input a listof polynomials G and returns a Boolean value. It returns True if G is a Grobnerbasis of the ideal generated by the polynomials in G, and returns False otherwise.The predicate uses the functions redmod (see Figure 3.13) and polBuch (seeFigure 3.15) according to Theorem 3.3.17.

GrobnerQ=Function[G,n,Apply[And,

Flatten[Table[

redmod[polBuch[G[[i]],G[[j]],n],G,n]===0,

i,1,Length[G],j,1,Length[G]]]]];

Figure 3.16: Checking whether G is a Grobner basis


3.3.3 Buchberger algorithm

In this section we present the Buchberger algorithm. Given a finite set of poly-nomials G as input, the Buchberger algorithm outputs a finite set of polynomialsthat constitutes a reduced Grobner basis of the ideal generated by the polynomialsin G. A possible implementation of the Buchberger algorithm in Mathematica isalso presented. Unless otherwise stated, we consider the order >glx on monomialsbut the Buchberger algorithm can be used assuming any other order on mono-mials will do provided it is suitable for polynomial division (see Section 3.2.3).

We first address the problem of obtaining a Grobner basis of an ideal I ofpolynomials from any given set of generators of I. Then, we show how to obtaina reduced Grobner basis of I from a given Grobner basis of I.

The first step of the Buchberger algorithm consists of a technique for obtaininga Grobner basis of an ideal I from a given basis of I. It is based on the Buchbergertheorem (Theorem 3.3.17).

In this first step, given any basis G of I, we look for pairs of polynomialsg, g′ ∈ G whose Buchberger polynomial does not reduce to 0 modulo G. If thereare none, G is a Grobner basis. Otherwise, we consider an appropriated nonzero

polynomial p such that B(g, g′)G−→ p and we add it to G, thus obtaining a new

basis G ∪ p of I. Note that B(g, g′) reduces to 0 modulo G ∪ p. We goon adding such new polynomials until the Buchberger polynomial of any pairof elements of the current basis reduces to 0, that is, the current basis satisfiesthe condition of the Buchberger theorem (Theorem 3.3.17). Then, this basis isa Grobner basis of I. The following proposition ensures that this can indeed beachieved in a finite number of steps.

Proposition 3.3.20 Let G be a basis of an ideal I over C[x1, . . . , xn]. Then,there is a finite sequence of sets of polynomials G0 . . . Gm such that

• G0 = G;

• Gi+1 = Gi ∪pi where pi is a nonzero Gi-irreducible polynomial such that

B(g, g′)Gi−→ pi for some g, g′ ∈ Gi, for each 0 ≤ i < m;

• B(g, g′)Gm−→ 0 for every pair of distinct polynomials g, g′ ∈ Gm.

The set Gm is a Grobner basis of I.

Proof (sketch): Assume that for each i ∈ N0 there is a set Gi+1 = Gi ∪ piwhere pi 6= 0 is such that B(g, g′) reduces to pi modulo Gi for some g, g′ ∈ Gi

and pi is Gi-irreducible.Let LT0 = lt(g) : g ∈ G0 and, for each i ∈ N, let LTi = LTi−1 ∪ lt(pi−1).

Let Ji be the ideal generated by LTi for each i ∈ N0. Note that no term t ∈ LTi


divides lt(pi), for each i ∈ N0 and therefore J0 ⊂ J1 ⊂ J2 ⊂ . . ., that is, the in-clusion Ji ⊂ Ji+1 holds for each i ∈ N0. However, one can prove that no suchincreasing chain of ideals exists. Hence, there is some m ∈ N0 such that, for allg, g′ ∈ Gm, if B(g, g′) reduces to p modulo Gm and p is Gm-irreducible then eitherp = 0 or p ∈ Gm. Since p is Gm-irreducible, it can not be the case that p ∈ Gm.Hence, p = 0. Given that each B(g, g′) can always be reduced to a Gm-irreduciblepolynomial we conclude that such a finite sequence G0 . . . Gm of set of polynomialindeed exists.

Since G ⊆ Gm and G is a basis of T , the set Gm is also a basis of I. Theorem3.3.17 ensures that it is a Grobner basis. QED

Proposition 3.3.20 sketches a technique for obtaining a Grobner basis of anideal I from any basis of I. The following example illustrates that constructionof a Grobner basis.


g1 = 2x21x2 + x2 and g2 = x3

2 + x1

in Z5[x1, x2]. We now follow Proposition 3.3.20 to compute a Grobner basis forthe ideal (g1, g2):

1. G0 = g1, g2

2. B(g1, g2) = 4x31 + 3x3

2

and

4x31 + 3x3

2

g2−→ 4x31 + 2x1 and 4x3

1 + 2x1 is irreducible modulo G0

therefore

G1 = g1, g2, g3 with g3 = 4x31 + 2x1

3. B(g1, g2)G1−→ 0, B(g1, g3)

G1−→ 0, B(g2, g3)G1−→ 0 (see Example 3.3.18)

We can then conclude that G1 = g1, g2, g3 is a Grobner basis for (g1, g2).

The function calcBuch in Figure 3.17 constitutes a possible implementationin Mathematica of the first step of the Buchberger algorithm. The functioncalcBuch receives as input a list G of polynomials and a positive integer n. Theinteger n indicates that we consider polynomials over Zn. The function consists ofa loop that works with two lists K and H of polynomials. The list K is initially setto G. At the beginning of each iteration H is empty. Using the functions redmod

and polBuch (see Figures 3.13 and 3.15), the function calcBuch then successivelycomputes the Buchberger polynomials of all pairs of elements of K, storing in H

the nonzero polynomials resulting from their reduction modulo K∪H. At the end


calcBuch=Function[G,n, Module[K,H,g,c,h,K=G;

g=True;

While[g,

c=Length[K];

H=;Do[h[i,j]=redmod[polBuch[K[[i]],K[[j]],n],

Union[K,H],n];

If[h[i,j]=!=0,H=Union[Append[H,h[i,j]]]],

i,1,c-1,j,i,c];K=Union[Join[K,H]];

g=(Length[H]!=0)];

K]];

Figure 3.17: First step of the Buchberger algorithm in Mathematica

of an iteration if H is empty the loop ends and the function returns K. OtherwiseK is updated with the polynomials in H and H is reseted to the empty list.

The next goal of the Buchberger algorithm is to obtain a reduced Grobnerbasis of an ideal I from a given Grobner basis G of I. To this end, there are threemore steps.

First, we remove from G polynomials g such that lt(g) is divisible by lt(g′)for some g′ 6= g in the basis. Then, we make the remaining polynomials monicby multiplying the terms of each polynomial by the inverse of its leading termcoefficient. Finally, we replace each monic polynomial h by h′ where h′ is obtainedfrom h reducing it as much as possible. The next propositions ensure that weindeed end up with a reduced Grobner basis after performing these steps.

Proposition 3.3.22 Let G be a Grobner basis of an ideal I over C[x1, . . . , xn]and let g ∈ G be such that lt(g) is divisible by lt(g′) for some g′ ∈ G\g. Theset G\g is a Grobner basis of I.

Proof: Since lt(g) is divisible by lt(g′), the polynomial g can be reduced modulo

G to h = g − lt(g)lt(g′)× g′. By Proposition 3.3.9, h ∈ I since g ∈ I and therefore, by

Proposition 3.3.12, hG−→ 0. Hence,

hf0−→ h1

f1−→ h2f2−→ . . . hr

fr−→ 0

where f0, f1, . . . , fr ∈ G and hr 6= 0. By Lemma 3.2.28, we can conclude thatlt(g) >glx lt(h) and, moreover, that lt(hi) >glx lt(hi+1) or lt(hi) = lt(hi+1) for all


0 ≤ i < r, assuming h0 = h. As a consequence, lt(g) >glx lt(hi) for all 0 ≤ i ≤ rand therefore no term of hi is divisible by lt(g), for 0 ≤ i ≤ r. Hence, g cannotbe used in the above reductions, that is, g 6= fi for all 0 ≤ i ≤ r. By Proposition3.2.32, the polynomial h can be expressed in terms of the polynomials in G\gand, since,

g = h +lt(g)

lt(g′)× g′

so can g.The set G\g is then also a basis of I. It is also a Grobner basis because

if lt(p) is divisible by lt(g) for some p ∈ I, then it is also divisible by the termlt(g′). QED

By removing from a Grobner basis G the polynomials g such that lt(g) isdivisible by lt(g′) for some g′ 6= g in the basis we can obtain a smaller Grobnerbasis G′.

Proposition 3.3.23 Let G be a Grobner basis of an ideal I over C[x1, . . . , xn]such that 0 /∈ G. For each g ∈ G, let hg = (cg)

−1×g, where cg is the coefficient oflt(g). The set H = hg : g ∈ G is also a Grobner basis of I and every polynomialin H is monic.

Proof: Clearly, hg = (cg)−1×g is a monic polynomial for each g ∈ G. Moreover,

the terms lt(hg) and lt(g) have the same monomial.Since G is a basis of I, and assuming that G consists of k polynomials

g1, . . . , gk, given p ∈ I there are polynomials a1, . . . , ak in C[x1, . . . , xn] such thatp =

∑k

i=1 ai × gi. Then,

p =k∑

i=1

ai × ci × hgi

because cg × hg = g for each g ∈ G. Hence, H is also a basis of I.Since G is a Grobner basis of I, given any p ∈ I there is some g ∈ G such that

lt(g) divides lt(p). Since, the terms lt(hg) and lt(g) have the same monomial,lt(hg) also divides lt(p). Hence, H is also a Grobner basis of I. QED

Proposition 3.3.24 Let G be a Grobner basis of an ideal I over C[x1, . . . , xn]such that for each g ∈ G

• g is monic

• lt(g′) does not divide lt(g) for all g′ ∈ G such that g′ 6= g.

Let g ∈ G and assume that gG\g−→ h where h is (G\g)-irreducible. Then h is

monic and G′ = (G\g)∪h is also a Grobner basis of I. Moreover, lt(g′) doesnot divide any nonzero term of h for all g′ ∈ G\g.


Proof: Assume that G consists of k polynomials g1, . . . , gk, and, without loss ofgenerality, assume that g is g1. If p ∈ I then p can be expressed in terms of the

polynomials in G. Since g(G\g)−→ h, by Proposition 3.2.32, g1 − h =

∑ki=2 a′

igi forsome a′

2, . . . , a′k in C[x1, . . . , xn]. Hence,

g1 = h +k∑

i=2

a′igi

that is, g1 can be expressed in terms of the polynomials in G′. Therefore, p canalso be expressed in terms of the polynomials in G′. We then conclude that G′ isa basis of I.

The fact that lt(g′) does not divide any nonzero term of h for all g′ ∈ G\gfollows from the assumption that h is (G\g)-irreducible.

It remains to prove that h is monic and that G′ is a Grobner basis of I. Ifh = g, then, clearly, that is the case. Assume now that h 6= g. Then,

gf1−→ h1

f2−→ h2f3−→ . . .

fr−→ hr

where f1, f2, . . . , fr ∈ G\g, r ≥ 1 and hr = h. Since lt(g) is not divisible bylt(g′) for all g′ ∈ G\g, some other term t of g is used to reduce g to h1 modulof1. Hence, by Lemma 3.2.28, lt(h1) = lt(g). Similarly, we can conclude that, infact, lt(hi) = lt(g) for all 1 ≤ i ≤ r. Since g is monic, so is hr, that is, h is monic.

Finally, consider any nonzero p ∈ I. Since G is a Grobner basis, lt(p) is divis-ible by lt(g′) for some g′ ∈ G. If lt(p) is divisible by lt(g) then it is also divisibleby lt(h), because lt(h) = lt(g). Therefore, the set G′ is a Grobner basis of I. QED

Note that given a Grobner basis G of I obtained following the steps describedabove, Proposition 3.3.24 ensures that if we replace each polynomial g by g′, whereg′ results from g by successive reductions modulo all the other polynomials untilno more reductions are possible, then we indeed get a reduced Grobner basis ofI.

The Buchberger algorithm can be sketched as in Figure 3.18. The input setG is any basis of an ideal I 6= 0 such that 0 6∈ G. The output set G4 obtainedin the last step is a reduced Grobner basis of I. In the sequel, given a finite set Gof nonzero polynomials, we use Buch(G) to denote the output of the algorithmwhen it receives G as input. Observe that Buch(G) is well defined since reducedGrobner basis are unique (see Proposition 3.3.30). The above propositions ensurethe correction of Buchberger algorithm.

Note that it is easy to obtain a Grobner basis for the ideal 0. The set 0is, in fact, the only basis of this ideal and it is a Grobner basis. Moreover, it isalso easy to conclude that if G is a basis of an ideal I 6= 0 and 0 ∈ G thenG\0 is also a basis of I. Hence, there is no loss of generality by assuming thatthe input set of the Buchberger algorithm does not include 0.


Buchberger algorithm

input: Finite nonempty set G of nonzero polynomials

step 1: Compute a sequence H0 . . .Hm of sets of polynomials such that

• H0 = G;

• Hi+1 = Hi ∪ p where p is a nonzero Hi-irreducible polynomial such

that B(g, g′)Hi−→ p for some distinct g, g′ ∈ Hi, for each 0 ≤ i < m;

and Hm is the first set such that B(g, g′)Hm−→ 0 for any distinct polynomials

g, g′ ∈ Hm.

Let G1 = Hm.

step 2: Assume that G1 consists of k ∈ N polynomials g1, . . . , gk and com-pute a sequence H0 . . .Hk−1 of sets of polynomials such that

• H0 = G1;

• Hi+1 = Hi\gi+1 if lt(gj) divides lt(gi+1) for some i+1 < j ≤ k, andHi+1 = Hi otherwise, for each 0 ≤ i < k − 1

Let G2 = Hk.

step 3: Let G3 = c−1g × g : g ∈ G2 where cg is the coefficient of the

leading term of g for each g ∈ G2.

step 4: Assume that G3 consists of k ∈ N polynomials g1, . . . , gk and com-pute a sequence H0 . . .Hk of sets of polynomials such that

• H0 = G3;

• Hi+1 = (Hi\gi+1) ∪ h where gi+1

Hi\gi+1−−−−→ h and h is (Hi\gi+1)-irreducible, for each 0 ≤ i < k.

Let G4 = Hk.

output: G4

Figure 3.18: Buchberger algorithm


Next we present two illustrative examples.

Example 3.3.25 Consider the set of polynomials G = g1, g2, g3 in x1, x2 overZ2 where

g1 = x1 g2 = x1x2 − x2 g3 = x2 + 1

We use the Buchberger algorithm to compute a reduced Grobner basis of theideal (g1, g2, g3). Let us compute Buch(g1, g2, g3):

step 1:

• H0 = g1, g2, g3

• B(g1, g2) = x2

x2H0−→ 1 and 1 is irreducible modulo H0

H1 = g1, g2, g3, g4 with g4 = 1

Moreover B(g, g′)H1−→ 0 for all polynomials g, g′ ∈ H1 such that g 6= g′.

Hence, G1 = H1 = g1, g2, g3, g4.

step 2:

• H0 = g1, g2, g3, g4• H1 = g2, g3, g4 since lt(g4) divides lt(g1)

• H2 = g3, g4 since lt(g4) divides lt(g2)

• H3 = g4 since lt(g4) divides lt(g3)

Hence, G2 = g4 = 1.

step 3: G3 = 1× 1 = 1.

step 4:

• H0 = 1

• H1 = H0 since H0\1 = ∅ hence, clearly, 1H0\1−→ 1 and 1 is H0\1-

ireducible.

Hence, G4 = H0 = 1.

We conclude that Buch(g1, g2, g3) = 1 and therefore 1 is a reduced Grobnerbasis of the ideal (g1, g2, g3).


In Example 3.3.25 we added the polynomial 1 to the basis in the first step ofthe algorithm and at the end we got 1 as reduced Grobner basis. Note thatthis is not just a consequence of this particular example. In general, if we addthe polynomial 1 to the basis in step 1 of the Buchberger algorithm, then weimmediately have a Grobner basis, since lt(1) divides any term. Then, in step 2,all the elements of the basis are removed except the polynomial 1, given that theirleading terms are divisible by lt(1). Step 3 does not modify anything because 1 ismonic. Similarly with respect to step 4 since the basis has now only one element.Note that, in general, whenever the input set G3 for step 4 is a singular set g,then g can never be reduced to a distint polynomial modulo G3\g since thisset is empty, and therefore no changes occur in step 4.


g1 = 2x21x2 + x2 and g2 = x3

2 + x1.

in Z5[x1, x2]. We use the Buchberger algorithm to compute a reduced Grobnerbasis of the ideal (g1, g2). Let us compute Buch(g1, g2):

step 1: From Example 3.3.21, we have that

• H0 = g1, g2• H1 = g1, g2, g3 with g3 = 4x3

1 + 2x1.

Moreover, B(g, g′)H1−→ 0 for all polynomials g, g′ ∈ H1 such that g′ 6= g.

Hence, G1 = H1 = g1, g2, g3.

step 2:

• H0 = g1, g2, g3• H1 = H0 since neither lt(g2) nor lt(g3) divide lt(g1)

• H2 = H1 since lt(g3) does not divide lt(g1)

Hence, G2 = H0 = g1, g2, g3.

step 3: G3 = 3× g1, 1× g2, 4× g3 = x21x2 + 3x2, x3

2 + x1, x31 + 3x1.

step 4: Let g′1 = x2

1x2 + 3x2, g′2 = x3

2 + x1 and g′3 = x3

1 + 3x1. Then,

H0 = g′1, g

′2, g

′3

H1 = H0 since g′1 is irreducible modulo g′

2, g′3


2, g′3


1, g′2


Hence, G4 = H0 = x21x2 + 3x2, x

32 + x1, x

31 + 3x1.

We conclude that Buch(g1, g2) = x21x2 + 3x2, x

32 + x1, x

31 + 3x1. This set of

polynomials is then a reduced Grobner basis of the ideal (g1, g2).

If G and H are reduced Grobner bases of I then they have the same numberof polynomials. Moreover, letting k be the number of polynomials in G, thereis an enumeration g1, . . . , gk of the elements of G and an enumeration h1, . . . , hk

of the elements of H such that lt(gi) = lt(hi) for each 1 ≤ i ≤ k (Exercise 30 inSection 3.5).

In Figure 3.22 we present a possible implementation of the Buchberger algo-rithm in Mathematica. Although some software packages already provide func-tions to compute Grobner bases, it is interesting to look at some of the details ofthe computation. In Figure 3.17 we have already presented an implementationof the first step of the algorithm. We now present implementations of the othersteps.

rempol=Function[G1,n,Module[ntddivQ,H,j,G2,ntddivQ=Function[K,p,

Apply[And,

Map[Function[h,

Not[divisibleQ[lt[p,n],lt[h,n],n]]],K]]];

G2=Sort[G1,Function[x,y,degmon[lt[x,n],n]<degmon[lt[y,n],n]]];

H=; j=1;

While[j<=Length[G2],

If[ntddivQ[H,G2[[j]]],

H=Append[H,G2[[j]]]];

j=j+1];

H]];

Figure 3.19: Second step of the Buchberger algorithm in Mathematica

The second step of the Buchberger algorithm can be implemented using thefunction rempol in Figure 3.19. The function receives has input a list G1 ofpolynomials and a positive integer n. It returns a list that results from G1 byremoving all the polynomials whose leading term is divisible by the leading termof another polynomial in the list. The function rempol uses the auxiliary functionntddivQ that given a list K of polynomials and a polynomial p returns True ifthe leading term of each polynomial in K does not divide the leading term of


p. The function rempol first creates the list G2 that results from ordering thepolynomials in G1 according to the degree of their leading terms, and sets H tothe empty list. After that the function passes through G2 appending to H eachelement of G2 whose leading term is not divisible by the leading terms of thepolynomials already in H.

The function makemonic in Figure 3.20 implements the third step of the Buch-berger algorithm. It receives as input a list G of polynomials and a positive integern. The function returns a list of monic polynomials that results from G by mul-tiplying each polynomial in the list by the inverse of its leading term coefficient.It uses the function coeflt presented in Figure 3.14.

makemonic=Function[G,n,Map[Function[p,Expand[PolynomialMod[p*PowerMod[coeflt[p,n],-1,n],n]]],G]];

Figure 3.20: Third step of the Buchberger algorithm in Mathematica

The function reduce in Figure 3.21 implements the fourth step of the Buch-berger algorithm. It receives as input a list G of polynomials and a positive integern. A copy K of the input list G is first created, and the output list U is set to theempty list. Then there is a loop that, at each step, removes the first polynomialin K and reduces it modulo the remaining polynomials in the list and the poly-nomials in already in U. The resulting polynomial is inserted in the output listU. The loop stops when K becomes the empty list. The function reduce uses thefunction redmod presented Figure 3.13.

reduce=Function[G,n,Module[K,U,h,K=G;

U=;While[Length[K]!=0,

h=redmod[First[K],Join[Rest[K],U],n];

K=Rest[K];

If[h=!=0,U=Append[U,h]]];

U]];

Figure 3.21: Fourth step of the Buchberger algorithm in Mathematica

Finally, the function reducedGrobnerbase in Figure 3.22 is a possible imple-mentation of the Buchberger algorithm. It receives as input a list G of polynomials


and a positive integer n, and returns the reduced Grobner basis of the ideal gen-erated by the polynomials in G. The functions calcBuch, rempol, makemonicand reduce that implement the four steps of the algorithm are used as expected.

reducedGrobnerbase=Function[G,n,Module[K,K=calcBuch[G,n];

K=rempol[K,n];

K=makemonic[K,n];

K=reduce[K,n];

K]];

Figure 3.22: Buchberger algorithm in Mathematica

3.3.4 Properties of Grobner basis

When we have a Grobner basis of an ideal I of polynomials, some importantquestions about I can be answered in an easy way. In this section we presentsome properties of Grobner basis.

First of all note that given an ideal I over C[x1, . . . , xn] with a Grobner basisG and a polynomial p in C[x1, . . . , xn] it is quite easy to determine whether ornot p ∈ I, that is, to solve the membership problem.

Proposition 3.3.27 Let I be an ideal over C[x1, . . . , xn] with a Grobner basis

G and let p be a polynomial in C[x1, . . . , xn]. Then pG−→ 0 if and only if p ∈ I.

Proof:(→) If p

G−→ 0 then, by Proposition 3.3.9, p ∈ I.(←) Conversely, let us assume that p ∈ I. Since G is a Grobner basis, from

Proposition 3.3.12 we conclude that pG−→ 0. QED


g1 = 2x21x2 + x2 g2 = x3

2 + x1 g3 = 4x31 + 2x1.

in Z5[x1, x2]. From Example 3.3.18 we know that G = g1, g2, g3 constitutes aGrobner basis of the ideal I = (g1, g2, g3). Let us consider the polynomial

p = 2x21x2 + x2

2


in Z5[x1, x2]. We cannot reduce p modulo g2 or g3 but we can reduce it modulog1:

pg1−→ x2

2 − x2

The polynomial x22 − x2 cannot be reduced modulo g1, g2 or g3. Hence p cannot

be reduced modulo G to 0. By Proposition 3.3.27, p /∈ I.

Note that reductions modulo a Grobner basis G are unique, that is, if wereduce a polynomial modulo a Grobner basis G of an ideal I until no morereductions in one step are possible, then we always get the same polynomial.

Proposition 3.3.29 Let I be an ideal over C[x1, . . . , xn] with a Grobner basis

G and let p be a polynomial in C[x1, . . . , xn] such that pG−→ p′ and p

G−→ p′′. Ifp′ and p′′ are G-irreducible then p′ = p′′.

Proof: By Proposition 3.3.9, p− p′ ∈ I and p− p′′ ∈ I. Hence, we have that

(p− p′′)− (p− p′) ∈ I, that is, p′ − p′′ ∈ I. By Proposition 3.3.27, p′ − p′′G−→ 0.

But, since p′ and p′′ are G-irreducible, so is p′−p′′. As a consequence, p′ − p′′ = 0,that is, p′ = p′′. QED

The converse of Proposition 3.3.29 also holds, that is, if for all p ∈ C[x1, . . . , xn]

we have that p′ = p′′ whenever pG−→ p′, p

G−→ p′′ and p′ and p′′ are G-irreducible,then G is a Grobner basis (Exercise 26 in Section 3.5). This property is sometimesused to define the notion of Grobner basis.

The fact that we can solve the ideal membership problem, allows us to solveother problems involving ideals. For instance, given two ideals I1 and I2 withGrobner bases G1 and G2 respectively, we can determine whether or not I1 ⊆ I2.As a consequence, we can also determine whether or not I1 = I2. (Exercise 27 inSection 3.5)

Reduced Grobner basis have a very important property: each ideal as a uniquereduced Grobner basis. Recall that we are considering a particular order onmonomials. The uniqueness result in Proposition 3.3.30 below assumes thatsome order on monomials is fixed. Different orders may lead to different reducedGrobner basis of a given ideal.

Proposition 3.3.30 Let I 6= 0 be an ideal over C[x1, . . . , xn]. Then I has aunique reduced Grobner basis.

Proof (sketch): We start by recalling that by the Buchberger algorithm we canalways get a reduced Grobner basis of I.

Let G and H be reduced Grobner bases of I. Hence, recall that G e Hhave the same number of polynomials and, moreover, letting k be the number


of polynomials in G, there is an enumeration g1, . . . , gk of the elements of G andan enumeration h1, . . . , hk of the elements of H such that lt(gi) = lt(hi) for each1 ≤ i ≤ k (Exercise 30 in Section 3.5).

By contradiction, assume that there is some 1 ≤ i ≤ k such that gi 6= hi.Then, since gi − hi ∈ I, Definition 3.3.11 ensures that lt(gi − hi) is divisible bylt(gj) (and lt(hj)) for some 1 ≤ j ≤ k. Note that lt(gi) and lt(hi) cancel eachother in gi − hi and therefore lt(gi) >glx lt(gi − hi). Hence, j 6= i. Then eithera nonzero term of gi is divisible by lt(gj) or a nonzero term of hi is divisible bylt(hj), thus contradicting the fact that G and H are reduced Grobner bases of I.Hence, gi = hi for each 1 ≤ i ≤ k, that is G = H . QED

Proposition 3.3.30 provides another method to determine whether two idealsI1 and I2 are equal. Using the Buchberger algorithm we compute the reducedGrobner bases of I1 from its basis and similarly with respect to I2. Then we justhave to compare them.

Corollary 3.3.31 Let I1 6= 0 and I2 6= 0 be ideals over C[x1, . . . , xn]. ThenI1 = I2 if and only if their reduced Grobner bases are equal.

Another consequence of Proposition 3.3.30 is that the reduced Grobner basisof the nonproper ideal C[x1, . . . , xn] is 1. In fact, 1 is a Grobner basis, has wehave already remarked above, and 1 is a reduced Grobner basis. Proposition3.3.30 ensures that it is the only one.

The following propositions illustrate the relevance of Grobner basis for solvingsystems of (nonlinear) polynomial equations. We first introduce some notationand present a preliminary result. In the sequel, given a set S of polynomials inC[x1, . . . , xn], we assume that

Z(S) = (u1, . . . , un) ∈ Cn : p (u1, . . . , un) = 0 for all p ∈ S.

Proposition 3.3.32 Let p1, . . . , pm be polynomials in C[x1, . . . , xn] with m ∈ N,and let I = (p1, . . . , pm). We have that Z(p1, . . . , pm) = Z(I).

Proof: Let (u1, . . . , un) ∈ Z(p1, . . . , pm). Since (p1, . . . , pm) is a basis of I,given any p ∈ I we have that p =

∑m

i=1 ai × pi for some a1, . . . , am in C[x1, . . . , xn].Then,

p(u1, . . . , un) =

m∑

i=1

ai(u1, . . . , un)× pi(u1, . . . , un) = 0.

Therefore, (u1, . . . , un) ∈ Z(I). Hence, Z(p1, . . . , pm) ⊆ Z(I). The inclusionZ(I) ⊆ Z(p1, . . . , pm) is immediate since p1, . . . , pr ⊆ I. QED

A consequence of the above result is that we can use ideals and ideal bases tocheck whether two systems of polynomial equations have the same solutions.


Proposition 3.3.33 Let p1, . . . , pr and q1, . . . , qs be polynomials in C[x1, . . . , xn],for r, s ∈ N. If (p1, . . . , pr) = (q1, . . . , qs) then the systems of polynomial equations

p1 = 0

. . .

pr = 0

and

q1 = 0

. . .

qs = 0

have the same solutions.

Proof: Assume that I = (p1, . . . , pr) and J = (q1, . . . , qs). By Proposition 3.3.32,

Z(p1, . . . , pr) = Z(I) and Z(q1, . . . , qs) = Z(J).

Therefore,

Z(p1, . . . , pr) = Z(q1, . . . , qs)

that is, the two systems have the same solutions. QED

As remarked above, to check whether two ideals are equal it is enough toverify that their reduced Grobner bases are equal. Hence, if we want to determinewhether two systems of polynomial equations have the same solutions, a goodstarting point is to compute the reduced Grobner basis of the ideal correspondingto each system. If the bases are equal the systems have the same solutions. Letus present an illustrative example.

Example 3.3.34 Consider the systems of polynomial equations

x1x2 − x1 − x2 + 1 = 0

x21x2 − 2x2

1 + 2x1x2 − 4x1 + x2 − 2 = 0(3.1)

and

x1x2 + x22 − x1 − 4x2 + 3 = 0

x21 + x1x2 + x1 − 5x2 + 6 = 0

x21 + x2

2 + 2x1 − 7x2 + 7 = 0

(3.2)

in R[x1, x2]. We first compute the reduced Grobner basis of the ideal gen-erated by the polynomials involved in (3.1), that is, the ideal (g1, g2) whereg1 = x1x2 − x1 − x2 + 1 and g2 = x2

1x2 − 2x21 + 2x1x2 − 4x1 + x2 − 2. We briefly

sketch the computation of Buch(g1, g2):


step 1:

• H0 = g1, g2

• B(g1, g2) = x21 − 3x1x2 + 5x1 − x2 + 2

B(g1, g2)H0−→ x2

1 + 2x1 − 4x2 + 5

H1 = g1, g2, g3 with g3 = x21 + 2x1 − 4x2 + 5

• B(g1, g3) = −x21 − 3x1x2 + 4x2

2 + x1 − 5x2

B(g1, g3)H1−→ 4x2

2 − 12x2 + 8

H2 = g1, g2, g3, g4 with g4 = 4x22 − 12x2 + 8


Hence, G1 = H2 = g1, g2, g3, g4.

step 2: G2 = g1, g3, g4

step 3: G3 = g1, g3, g′4 where g′

4 = x22 − 3x2 + 2

step 4: G4 = G3.

We conclude that

Buch(g1, g2) = x1x2 − x1 − x2 + 1, x21 + 2x1 − 4x2 + 5, x2

2 − 3x2 + 2

and therefore this set of polynomials is the reduced Grobner basis of (g1, g2).

We now consider the system (3.2). In this case we have to compute thereduced Grobner basis of the ideal (r1, r2, r3), where r1 = x1x2 +x2

2−x1−4x2 +3,r2 = x2

1 + x1x2 +x1− 5x2 + 6 and r3 = x21 +x2

2 + 2x1− 7x2 + 7. We briefly sketchthe computation of Buch(r1, r2, r3):

step 1:

• H0 = r1, r2, r3• B(r1, r2) = −x2

1 − 5x1x2 + 5x22 + 3x1 − 6x2

B(r1, r2)H0−→ 9x2

2 − 27x2 + 18

H1 = r1, r2, r3, r4 with r4 = 9x22 − 27x2 + 18


Hence, G1 = H1 = r1, r2, r3, r4.

step 2: G2 = r1, r3, r4


step 3: G3 = r1, r3, r′4 where r′4 = x2

2 − 3x2 + 2

step 4: since r1

r′4−→ x1x2 − x1 − x2 + 1 and r3

r′4−→ x21 + 2x1 − 4x2 + 5

then G4 = x1x2 − x1 − x2 + 1, x21 + 2x1 − 4x2 + 5, x2

2 − 3x2 + 2.

We conclude that

Buch(r1, r2, r3) = x1x2 − x1 − x2 + 1, x21 + 2x1 − 4x2 + 5, x2

2 − 3x2 + 2

and therefore this set is the reduced Grobner basis of (r1, r2, r3).Since the reduced Grobner basis of the ideals (g1, g2) and (r1, r2, r3) is the

same, the ideals are equal and the systems have the same solutions.

Another consequence of Proposition 3.3.32 is the following. Given polynomialsp1, . . . , pm in C[x1, . . . , xn] we can conclude that the system p1 = 0, . . . , pm = 0has no solutions in C if 1 ∈ (p1, . . . , pm) (Exercise 31 in Section 3.5). We can alsoconclude that the system has no solution whenever 1 is the reduced Grobnerbasis of (p1, . . . , pm).

Proposition 3.3.35 Let p1, . . . , pm be polynomials in C[x1, . . . , xn], for m ∈ N.If 1 is the reduced Grobner basis of the ideal (p1, . . . , pm) then the system

p1 = 0

. . .

pm = 0

has no solution in C.

Proof: Since 1 is the reduced Grobner basis of the ideal (p1, . . . , pm), then(p1, . . . , pm) = C[x1, . . . , xn]. So, Z(p1, . . . , pm) = Z(C[x1, . . . , xn]) by Proposi-tion 3.3.32. But, Z(C[x1, . . . , xn]) = ∅, since, for instance, p1 = x1−1 and p2 = x1

are both polynomials in C[x1, . . . , xn] and there is no c ∈ C such that p1(c) = 0and p2(c) = 0, given that 1 6= 0 in any field. QED

Let us present an illustrative example.

Example 3.3.36 Consider the system of polynomial equations

x1x22 + x1x2 + 2 = 0

x22 − x1 = 0

x21 + x1x2 + 1 = 0


in R[x1, x2]. We have to compute the reduced Grobner basis of the ideal (p1, p2, p3)where p1 = x1x

22 + x1x2 + 2, p2 = x2

2 − x1 and p3 = x21 + x1x2 + 1. We have that

B(p1, p2) = x21 + x1x2 + 2

andx2

1 + x1x2 + 2p3−→ 1.

Assuming p4 = 1, we are able to conclude that B(pi, pj) reduces to 0 modulop1, p2, p3, p4, for all 1 ≤ i, j ≤ 4, i 6= j. Hence, a Grobner basis of (p1, p2, p3) isG = p1, p2, p3, p4.

The other steps of the Buchberger algorithm are not necessary. In fact, notingthe remarks after Example 3.3.25, we can already conclude that 1 is the reducedGrobner basis of the ideal (p1, p2, p3). Hence, the system has no solutions in R.

The converse of Proposition 3.3.35 also holds but only when the coefficientfield is algebraically closed. A field C is said to be algebraically closed [20]whenever for each polynomial p of degree m ∈ N in C[x1] there is c ∈ C suchthat p(c) = 0. The field C of complex numbers, for instance, is algebraicallyclosed. The field R of real numbers is not algebraically closed, since, for instance,x2

1 + 1 = 0 has no solution in R. Moreover, no finite field is algebraically closed.An example that illustrates the fact that the converse of Proposition 3.3.35

does not hold for arbitrary fields is the following. Consider the equation x21+1 = 0.

This equation has no solution in R. However, the reduced Grobner basis of theideal (x2

1 + 1) over R[x1] is x21 + 1 and 1 /∈ x2

1 + 1.

As the following example illustrates, a consequence of Proposition 3.3.33 isthat we can often use Grobner bases, or reduced Grobner bases, to find thesolutions of systems of polynomial equations.

Example 3.3.37 Consider the system (3.1) in Example 3.3.34. We know thatthe reduced Grobner basis of the ideal generated by the two polynomials involvedis

x1x2 − x1 − x2 + 1, x21 + 2x1 − 4x2 + 5, x2

2 − 3x2 + 2and that the corresponding system

x1x2 − x1 − x2 + 1 = 0

x21 + 2x1 − 4x2 + 5 = 0

x22 − 3x2 + 2 = 0

(3.3)

and the system (3.1) have the same solutions. But note that the last equation ofthe system (3.3) involves only one variable, x2, thus it is easier to solve. With thesolutions for x2 we can transform the other two equations in equations involvingonly the variable x1 that can also be easily solved.


From the equation x22 − 3x2 + 2 = 0 we get x2 = 1 or x2 = 2. For x2 = 1, we

the get the system

0x1 = 0

x21 + 2x1 + 1 = 0

and therefore x1 = −1. For x2 = 2 we get

x1 − 1 = 0

x21 + 2x1 − 3 = 0

and therefore x = 1. Thus, the system (3.3), and therefore the system (3.1), hastwo solutions

x1 = −1, x2 = 1 and x1 = 1, x2 = 2.

If a system has three or more variables, once we have eliminated one of thevariables as above, we can repeat the process with the remaining equations andtry to eliminate a new variable. Observe that this process of eliminating variablesis similar to Gauss elimination algorithm for systems of linear equations.

Note that it may not be possible to obtain Grobner bases where all the vari-ables but one have been eliminated from some equation. However, recall that theuniqueness of reduced Grobner bases depend on the monomial order considered.Hence, if in the reduced Grobner basis with respect to the order >glx, for instance,no equation with only one variable exists, it may be the case that consideringanother order, such as the order >lx for example, such equation occurs. In fact,whenever the system satisfies some suitable conditions, using the order >lx en-sures that in the corresponding reduced Grobner basis an equation with only onevariable always exists. We do not further develop this subject herein and referthe reader to [10], for instance.

3.4 Motivating examples revisited

3.4.1 Equivalence of digital circuits

We address the problem of how Grobner bases can be used for checking theequivalence of two combinational circuits.

As we have discussed in Section 3.1.1, we can check the equivalence of twocombinational circuits checking whether two suitable propositional formulas are

3.4. MOTIVATING EXAMPLES REVISITED 119

equivalent. A combinational circuit with one output variable computes a Booleanfunction and can be represented by a propositional formula.

Let P be a set (of propositional symbols). The set FP of propositional formulasover P is inductively defined as follows:

• p ∈ FP for every p ∈ P ;

• if ϕ1, ϕ2 ∈ FP then (¬ϕ1) ∈ FP and (ϕ1 ⇒ ϕ2) ∈ FP .

The connectives ∧, ∨ and ⇔ can be defined as abbreviations as expected (fordetails see [9]).

A valuation over P is a map v : P → 0, 1. A valuation v over P can beextended to propositional formulas over P considering the map

v : FP → 0, 1

such that v(p) = v(p) for every p ∈ P , and such that v(¬ϕ1) = 1 − v(ϕ1) andv(ϕ1 ⇒ ϕ2) = 1−v(ϕ1)+v(ϕ1)×v(ϕ2) for every ϕ1, ϕ2 ∈ FP . Given ϕ ∈ FP , wesay that ϕ is satisfiable whenever there is a valuation v over P such that v(ϕ) = 1,and we say that ϕ is valid whenever v(ϕ) = 1 for all valuations v over P .

We first describe how a propositional formula is converted into a polynomialin Z2[x1, . . . , xn]. For simplicity, in the sequel we use x1, x2, . . . as propositionalsymbols.

Definition 3.4.1 Let P = x1, . . . , xn with n ∈ N. The function

conv : FP → Z2[x1, . . . , xn]

is inductively defined as follows:

• conv(xi) = xi for each 1 ≤ i ≤ n;

• conv(¬ϕ) = 1− conv(ϕ);

• conv(ϕ1 ⇒ ϕ2) = conv(ϕ1)× conv(ϕ2)− conv(ϕ2).

It is straightforward to conclude that conv(ϕ1 ∨ ϕ2) = conv(ϕ1) × conv(ϕ2)and conv(ϕ1 ∧ ϕ2) = conv(ϕ1) + conv(ϕ2) − conv(ϕ1) × conv(ϕ2). Moreover,conv(ϕ1 ⇔ ϕ2) = conv((ϕ1 ⇒ ϕ2) ∧ (ϕ2 ⇒ ϕ1)) (see Exercise 36 in Section 3.5).

Example 3.4.2 Consider ϕ1 = x1 ∨ (¬x1) and ϕ2 = (¬x1) ∨ (x1 ∧ x2). Then,

conv(ϕ1) = conv(x1)× conv(¬x1)

= conv(x1)× (1− conv(x1))

= x1 × (1− x1)

= −x21 + x1


conv(ϕ2) = conv(¬x1)× conv(x1 ∧ x2)

= (1− conv(x1))×(conv(x1)+ conv(x2)− conv(x1)× conv(x2))

= (1− x1)× (x1 + x2 − x1 × x2)

= −x21x2 − x2

1 + x1 + x2

Given the properties of the field Z2, ϕ1 also corresponds to x21 + x1 and conv(ϕ2)

to x21x2 + x2

1 + x1 + x2.

The Mathematica rewriting rules in Figure 3.23 convert propositional formu-las into polynomials.

conv[neg[x ]]:=1-conv[x];

conv[imp[x1 ,x2 ]]:=conv[x2]conv[x1]-conv[x2];

conv[or[x1 ,x2 ]]:=conv[x1]conv[x2];

conv[and[x1 ,x2 ]]:=conv[neg[or[neg[x1],neg[x2]]]];

conv[eqv[x1 ,x2 ]]:=conv[and[imp[x1, x2],imp[x2, x1]]];

Figure 3.23: Converting propositional formulas into polynomials

Let ϕ be a propositional formula and let p ∈ Z2[x1, . . . , xn] be the polynomialconv(ϕ). It is straightforward to conclude that ϕ is satisfiable if and only if thereare u1, . . . , un ∈ Z2 such that p(u1, . . . , un) = 0 (see Exercise 37 in Section 3.5).

The following proposition establishes how Grobner bases can be used to checkthe validity of a propositional formula.

Proposition 3.4.3 Consider P = x1, . . . , xn and ϕ ∈ FP . Let p = conv(¬ϕ)and pi = conv(xi ∨ (¬xi))) for each 1 ≤ i ≤ n. If 1 is in the reduced Grobnerbasis of the ideal (p, p1, . . . , pn) over Z2[x1, . . . , xn] then ϕ is valid.

Proof: If 1 is in the reduced Grobner basis of (p, p1, . . . , pn), then this reducedGrobner basis is 1 and therefore, by Proposition 3.3.35, the system

p = 0

p1 = 0

. . .

pm = 0

has no solutions. Given that pi(u) = 0 for each 1 ≤ i ≤ n and u ∈ Z2, thenp(u1, . . . , un) 6= 0 for all u1, . . . , un ∈ Z2. As a consequence, ¬ϕ is not satisfiable


and therefore ϕ is valid. QED

To check whether two circuits corresponding to the formulas ϕ1 and ϕ2 areequivalent we check whether the formulas are equivalent, that is, we check whetherϕ1 ⇔ ϕ2 is valid . To this end we can check if 1 is in the reduced Grobnerbasis of the ideal over Z2[x1, . . . , xn] generated by the polynomial correspondingto ¬(ϕ1 ⇔ ϕ2) and the polynomials corresponding to the formulas xi ∨ (¬x1)for each propositional symbol xi in ϕ1 ⇔ ϕ2. Let us illustrate equivalence ofcircuits with the following example. Herein, we do not give details on how toconvert combinational circuits into propositional formulas. The interested readeris referred to [23].

Example 3.4.4 In this example we are going to test the equivalence of twocircuits using Grobner basis. Consider circuits 1 and 2 corresponding to propo-sitional formulas ϕ1 and ϕ2, respectively, where

ϕ1 = (¬x1) ∨ (x1 ∧ x2) and ϕ2 = (¬x1) ∨ x2.

First, we have to convert the formulas ¬(ϕ1 ⇔ ϕ2), x1 ∨ (¬x1) and x2 ∨ (¬x2)into polynomials in Z2[x1, x2]. From Example 3.4.2 we have that

conv(x1 ∨ (¬x1)) = x21 + x1

and, similarly

conv(x2 ∨ (¬x2)) = x22 + x2.

After some computations we get

conv(¬(ϕ1 ⇔ ϕ2)) = x61x

42 +x6

1x22 +x5

1x32 +x4

1x42 +x5

1x2 +x41x

22 +x2

1x42 +x4

1x2

+ x21x

22 + x1x

32 + x4

2 + x21 + x2

2 + x1 + 1.

To check the equivalence of the two circuits we are going to check if 1 is inthe reduced Grobner basis of the ideal (g1, g2, g3) over Z2[x1, x2] where

g1 = conv(¬(ϕ1 ⇔ ϕ2) g2 = conv(x1 ∨ (¬x1)) g3 = conv(x2 ∨ (¬x))

Observe that

B(g1, g2) = x51x

42 + x6

1x22 + x5

1x32 + x4

1x42 + x5

1x2 + x41x

22 + x2

1x42 + x4

1x2

+ x21x

22 + x1x

32 + x4

2 + x21 + x2

2 + x1 + 1

and

B(g1, g2)H0−→ 1.

Assuming g4 = 1, B(gi, gj) reduces to 0 modulo g1, g2, g3, g4, for all 1 ≤ i, j ≤ 4,i 6= j. Hence, G = g1, g2, g3, g4 is a Grobner basis of (g1, g2, g3).

The other steps of the Buchberger algorithm are not necessary. We can alreadyconclude that 1 is the reduced Grobner basis of the ideal (g1, g2, g3). Therefore,the circuits are equivalent.


3.4.2 Inverse kinematics

In Section 3.1.2 we have described the inverse kinematics task: given the intendedcoordinates (a, b, c) of the end effector of a particular robot, we want to determinewhat are the suitable angles β, θ1 and θ2 at the base and joints for reaching thatposition. To this end we have to find the solutions of the system of polynomialequations

l2v2v1 + l3v3v1 − a = 0

l2v2u1 + l3v3u1 − b = 0

l2u2 + l3u3 − c = 0

u21 + v2

1 − 1 = 0

u22 + v2

2 − 1 = 0

u23 + v2

3 − 1 = 0

where l2 and l3 are the lengths of the arm links. The system has 6 variables, ui

and vi for i = 1, 2, 3, where u1 = sin β, v1 = cos β, and u2 = sin θ1, v2 = cos θ1,u3 = sin θ3, v3 = cos θ3.

For simplicity, we are going to solve this system for particular values of a, b, c,l2 and l3. We use Grobner bases as described in Section 3.3.4. Let us assume thata = b = c = l2 = l3 = 1. We also assume a change of variables: x1 = u1, x2 = v1,x3 = u2, x4 = v2, x5 = u3 and x6 = v3. Thus, the system becomes

x2x4 + x2x6 − 1 = 0

x1x4 + x1x6 − 1 = 0

x3 + x5 − 1 = 0

x21 + x2

2 − 1 = 0

x23 + x2

4 − 1 = 0

x25 + x2

6 − 1 = 0

where the polynomials are already ordered.

Let us compute the reduced Grobner basis of the ideal (p1, p2, p3, p4, p5, p6)over R[x1, x2, x3, x4, x5, x6], where

p1 = x4x2 + x6x2 − 1 p2 = x4x1 + x6x1 − 1 p3 = x3 + x5 − 1

p4 = x21 + x2

2 − 1 p5 = x23 + x2

4 − 1 p6 = x25 + x2

6 − 1

considering the order >lx. Note that the polynomials are already ordered accord-ing to this order. Using the Buchberger algorithm we get

x3 + x5, x1 − 2x6, x2 − 2x6, x4 − x6, x25 − 1

2, x2

6 − 12.

3.5. EXERCISES 123

Hence, the solutions of the original system, assuming a = b = c = l2 = l3 = 1,can be computed just solving the system

x3 + x5 = 0

x1 − 2x6 = 0

x2 − 2x6 = 0

x4 − x6 = 0

x25 − 1

2= 0

x26 − 1

2= 0

.

3.5 Exercises

1. Compute p1 + p2 and p1 × p2 where

(a) p1 = 3x2 + 4x and p2 = 2x3 + 3x2 are polynomials in Z5[x].

(b) p1 = 6x21x

32 + 4x2

2 and p2 = 5x31 + 3x5

1x2 are polynomials in Z7[x1, x2].

2. Compute the quotient and the remainder of the division of p by d where

(a) p = 2x16 + x1

5 + x14 + x1 and d = x1

2 − x1 are polynomials in Q[x1].

(b) p = 2x21x2 − x2

1 + x1x2 + x1 and d = x1x2 + x1 are polynomials inZ3[x1, x2].

(c) p = 3x21x2−x2

1+x1x2 and d = 2x1x2 + x1 are polynomials in Z5[x1, x2].

(d) p = x1x2x33 + 4x2

2x33 + 2x2

1x33 + x1x2x

23 and d = 2x1x3 + x2x3 are poly-

nomials in Z7[x1, x2, x3].

3. Consider the polynomials p = 3x31x

32 +5x2

1x32−6x3

1x2 +2x21x

22−10x2

1x2−4x21

and d = x1x22 − 2x1 in R[x1, x2]. Prove that p

d−→ 0.

4. Consider the polynomials p = 2x31x

32 + x3

1x3 + x32x3, d1 = x3

1 + x3 and

d2 = x32 + x3 + 2 in Z5[x1, x2, x3]. Prove that p

d1,d2−→ 2x3.

5. Let C be a field and n ∈ N. Prove that (C[x1, . . . , xn], +, 0,−,×), where +and × are the sum and product of polynomials and −p is the symmetric ofp for ecah p ∈ C[x1, . . . , xn], is a ring over C.

6. Prove that every ring of polynomials is unitary.

7. Prove that >lx and >glx are well founded total orders on monomials inx1, . . . , xn.

8. Prove that 0 ∈ I for every ideal I over a ring (A, +, 0,−,×).


9. Prove that the set Zeven of the integer numbers that are even is an idealover Z.

10. Prove that the ideal (6, 9) over Z is the set of multiples of 3.

11. Prove that the ideal (g1, g2) over Z is the set gcd(g1, g2)× n : n ∈ Z.

12. Let I and J be ideals over the same ring A = (A, +, 0,−,×). Prove that

(a) I ∩ J

(b) I + J = a + b : a ∈ I, b ∈ J(c) I − J = i− j : i ∈ I, j ∈ J

are also ideals over A.

13. Let (A, +, 0,−,×) be a commutative unitary ring and let I ⊆ A. Prove thatI is an ideal over the ring if and only if I 6= ∅ and ((a1 × b1) + (a2 × b2)) ∈ Ifor every b1, b2 ∈ I and a1, a2 ∈ A.

14. Let p1, p2 be two polynomials in C[x1, . . . , xn] and let D be a finite subset

of C[x1, . . . , xn]. Prove that B(p1, p2)D−→ 0 if and only if B(p2, p1)

D−→ 0.

15. Let I be an ideal over C[x1, . . . , xn], G a finite subset of C[x1, . . . , xn] andg ∈ C[x1, . . . , xn]. Prove that

(a) g is a Grobner basis of the ideal (g).

(b) if G is a basis of I and 1 ∈ G then G is a Grobner basis.

(c) if g is a G-irreducible nonzero polynomial then g /∈ G.

16. Consider the polynomials g1 = x31+x3 and g2 = x3

2+x3+2 in Z5[x1, x2, x3].Prove that g1, g2 is a Grobner basis of the ideal (g1, g2) and check whether2x3

1x32 + x3

1x3 + x32x3 ∈ (g1, g2).

17. Consider the polynomials g1 = x1x3+x22+2, g2 = 4x2

2x3+x3 and g3 = 2x22+3

in Z5[x1, x2, x3]. Prove that g1, g2, g3 is a Grobner basis of the ideal(g1, g2, g3) and check whether x1x

22x3 + x1x3 + 1 ∈ (g1, g2, g3)

18. Consider the polynomials g1 = 2x22 + 2x1, g2 = x1x3 + x2

2 + x1 + 6x2 andg3 = 4x1x

22 + 4x2

1 in Z7[x1, x2, x3].

(a) Prove that g1, g2, g3 is a Grobner basis of the ideal (g1, g2, g3).

(b) Check whether 2x21x3 + 6x1x

22 + 6x2

1 + 5x1x2 ∈ (g1, g2, g3).

(c) Compute the reduced Grobner basis of (g1, g2, g3).

3.5. EXERCISES 125

19. Let G be a finite set of polynomials in Z2[x1, x2, x3, x4] and let the setG′ = x1−x2, −x2

2+x3x4, x3+x1, −x1−x4, −x2+x3, x2+x4, −x2x3x4+x3x4

2, x3 + x4 be a Grobner basis of the ideal generated by G. Computethe reduced Grobner basis of the ideal generated by G. Check whetherx1

2 + x2x4 is in the ideal generated by G.

20. Consider the polynomials g1 = −3x1x2 + 2 and g2 = 3x1 − 1 in Z5[x1, x2].

(a) Check whether g1, g2 is a Grobner basis of the ideal (g1, g2).

(b) Compute the reduced Grobner basis of (g1, g2).

(c) Check whether 4x12x2 + 3x2

3 ∈ (g1, g2).

21. Consider the polynomials g1 = 3x1x2 − 2x1 and g2 = −2x22 + 3x1 in Z5[x1, x2].

(a) Check whether g1, g2 is a Grobner basis of the ideal (g1, g2).

(b) Compute the reduced Grobner basis of (g1, g2).

(c) Check whether 4x12x2

2 + 3x1 ∈ (g1, g2).

22. Consider the polynomials g1 = x31 and g2 = x4

1 +4x2 in Z5[x1, x2]. Computethe reduced Grobner basis of the ideal (g1, g2).

23. Consider the polynomials g1 = x21 + 2 and g2 = x2

1 + 4x2 in Z5[x1, x2].Compute the reduced Grobner basis of the ideal (g1, g2).

24. Consider the polynomials g1 = x1, g2 = x1x2 − x2 and g3 = x2 + 1 inZ2[x1, x2].Compute the reduced Grobner basis of the ideal (g1, g2, g3). Checkwhether x2

1 + x32 ∈ (g1, g2, g3).

25. Consider the polynomials g1 = x22 +x2

3, g2 = x21x2 +x2x3 and g3 = x3

3 +x1x2

in Z2[x1, x2, x3]. Compute the reduced Grobner basis of the ideal (g1, g2, g3).Check whether x2

1x23 + x2

2x3 ∈ (g1, g2, g3).

26. Let G be a set of polynomials in C[x1, . . . , xn] and assume that for all

p ∈ C[x1, . . . , xn] we have that p′ = p′′ whenever pG−→ p′, p

G−→ p′′ and p′

and p′′ are G-irreducible. Prove that G is a Grobner basis.

27. Let G1 and G2 be Grobner bases of the ideals I1 and I2 over C[x1, . . . , xn],respectively. Use the properties of Grobner bases to determine whetherI1 ⊆ I2.

28. Let I1 = (4x1 + 2x2, 3x2) and I2 = (x21, 3x

21 − x2,−x2

1 + 2x2) be ideals overZ7[x1, x2]. Prove that I1 ∩ I2 = I2.

29. Let I1 = (x1, 3x2) and I2 = (4x1, 3x21 − x1,−x2

1 + 2x2) be ideals overZ7[x1, x2]. Prove that I1 ∪ I2 = I1 ∩ I2.


30. Let G and H be reduced Grobner bases of an ideal I over C[x1, . . . , xn].Then G and H have the same number of polynomials and, moreover, lettingk be the number of polynomials in G, there is an enumeration g1, . . . , gk ofthe elements of G and an enumeration h1, . . . , hk of the elements of H suchthat lt(gi) = lt(hi) for each 1 ≤ i ≤ k.

31. Let p1, . . . , pm ∈ C[x1, . . . , xn]. Prove that the system p1 = 0, . . . , pm = 0has no solutions in C if 1 ∈ (p1, . . . , pm).

32. Solve the following system of polynomial equations in R using Grobnerbases

x21 + x1x2 = 1

x1x2 + x22 = 2

33. Solve the following system of polynomial equations in Z5 using Grobnerbases

x21 − x2

2 = 1

x21x2 − x3

2 − x2 = 1

34. Solve the following system of polynomial equations in R using Grobnerbases

x1 + x2 + x23 − x3 − 1 = 0

x1 + 2x2 + x23 − 1 = 0

x1 + x2 + x23 − 3 = 0

35. Solve the following system of polynomial equations in Z3 using Grobnerbases

x1x2x3 + x22x3 + x2 = 0

x1x3 + x2x3 − x3 = −1

x22 + x2x3 = 2

36. Prove that

(a) conv(ϕ1 ∨ ϕ2) = conv(ϕ1)× conv(ϕ2).

(b) conv(ϕ1 ∧ ϕ2) = conv(ϕ1) + conv(ϕ2)− conv(ϕ1)× conv(ϕ2).

(c) conv(ϕ1 ⇔ ϕ2) = conv((ϕ1 ⇒ ϕ2) ∧ (ϕ2 ⇒ ϕ1)).

37. Let ϕ be a propositional formula and let p ∈ Z2[x1, . . . , xn] be the polyno-mial conv(ϕ). Prove that ϕ is satisfiable if and only if there are u1, . . . , un ∈ Z2

such that p(u1, . . . , un) = 0.

3.5. EXERCISES 127

38. Consider the digital circuits corresponding to the formulas

(x1 ∨ (¬x2)) ∧ (x1 ∨ (¬x1)) and (¬x1)⇒ (¬x2).

(a) Present the polynomials in Z2[x1, x2] resulting from their conversion.

(b) Check whether they are equivalent using Grobner basis, assuming thatthe polynomial

1 + x1 + x31 + x4

1 + x51 + x7

1 + x81 + x3

1x2 + x41x2 + x5

1x2 + x71x2 + x2

1x22+

x61x

22 + x7

1x22 + x4

1x32 + x7

1x32 + x4

1x42 + x6

1x42 + x8

1x42

in Z2[x1, x2] corresponds to the formula

¬(((x1 ∨ (¬x2)) ∧ (x1 ∨ (¬x1)))⇔ ((¬x1)⇒ (¬x2))).

39. Consider the digital circuits corresponding to the formulas (¬x1) ∧ x2 and¬((¬x2) ∨ x1). Check whether they are equivalent using Grobner bases.

Chapter 4

Euler-Maclaurin formula

In this chapter, we present several techniques to compute summations. Givena set of integers A, a set of numbers B, and a map f : A→ B, the sum, orsummation, of f on A, is denoted by

∑

k∈A

f(k) (4.1)

and is the sum of all images f(k) for every k ∈ A. When A is a finite set, say,A = a1, . . . , an, then

∑

k∈A

f(k) = f(a1) + · · ·+ f(an).

Moreover, when A is an integer interval [a..b], that is, the set of all integersbetween a and b, then the summation (4.1), can be written as

b∑

k=a

f(k).

Summations play an important role in several different fields from science toengineering. The goal of this chapter is to provide several techniques for comput-ing summations. We give illustrations on analysis of algorithms in Bioinformat-ics, linear algebra and sorting. In Section 4.1 we present a motivating examplein Bioinformatics. In Section 4.2 we introduce summation expressions and someof their relevant properties. The Euler-Maclaurin formula is presented in Section4.3. In Section 4.4, we illustrate the relevance of summations to the analysis ofthe Gauss elimination technique and the insertion sort algorithm. In Section 4.5we present some exercises.

129

130 CHAPTER 4. EULER-MACLAURIN FORMULA

4.1 Motivation

Bioinformatics is related to searching and data mining DNA sequences. A DNAstring, or gene, is a sequence in the alphabet A, C, G, T, where the symbolsrepresent the molecules: adenine (A), cytosine (C), guanine (G) and thymine(T). Pattern matching of DNA is related to the search a sequence of letters in alarger sequence, for instance, the genome. From the computational point of view,the problem we have to address is to determine if a given sequence (pattern) is asubsequence of another sequence (word). A naive algorithm to solve this problemis presented in Figure 4.1.

SPatMatch=Function[w,p,Module[i,j,r,s,r=False;

i=0;

While[i<=Length[w]-Length[p]&&!r,

j=1;

s=True;

While[j<=Length[p]&&s,

s=s&&w[[i+j]]===p[[j]];

j=j+1];

r=r||s;i=i+1];

r]];

Figure 4.1: Naive pattern matching algorithm

The function SPatMatch in Figure 4.1 receives two lists, stored in w and p,corresponding to the word and the pattern, respectively. It returns a Booleanvalue stored in variable r. If the value of r is True then the pattern p is asubsequence of the word w.

Let n be the length of the word w and let m be the length of the patternp. The first iteration of the outer loop consists in comparing the pattern p withthe first m elements of w. If they all match we are done, that is, the pattern pis a subsequence of w (in fact, a prefix of w). Otherwise, we necessarily reach aposition 1 ≤ k ≤ m such that

w[[k]] 6= p[[k]].

This means that the element in the k-th position of the list w differs from theone in the k-th position of the list p. At this point the first iteration of the outerloop ends and we start the next iteration. This step consists in comparing thepattern p with

w[[2], w[[3]], . . . , w[[m + 1]].

4.2. EXPRESSIONS 131

If no pattern is found, at the k-th step the algorithm compares the pattern p with

w[[k], w[[k + 1]], . . . , w[[k + (m− 1)]].

It will finish if either a pattern is found or k + (m− 1) > n.

We now analyze the complexity of the algorithm in Figure 4.1 in terms of thenumber of comparisons performed between the input word w and pattern p, thatis, the comparisons

w[[i+j]] === p[[j]].

This number should be expressed as a function of the input sizes of both w andp.

We can consider, in particular, worst-case analysis. In worst-case analysis, weconcentrate on characterizing the worst possible situation, that is, on evaluatingthe maximum number of comparisons needed to determine wether or not thepattern p occurs in the word w. Observe that the maximum number of suchcomparisons occurs when the algorithm always executes m comparisons at eachiteration of the outer loop. For instance, the case when the mismatch occursalways in the last iteration of the inner loop (eg. w = C, C, C, C, C, C, C, C,Cand p = C, C, A). In this case the number of comparisons executed in the innerloop is

m∑

j=1

1.

Moreover, since the inner loop is executed when i ranges from 0 to n −m, thetotal number of comparisons is given by the following summation:

n−m∑

i=0

(m∑

j=1

1

)

.

In this case the summation is very simple to compute, it yields (n−m + 1)m.

4.2 Expressions

In this section, we introduce the syntax and semantics of summation expressions.

Syntax

We introduce integer and real expressions. Let I be a nonempty set of integervariables. The set of integer expressions EI is the smallest set containing allinteger numbers and variables that is closed under addition and multiplication.That is, EI , is inductively defined as follows:


• Z ∪ I ⊆ EI ;

• (d1 + d2), (d1 × d2) ∈ EI for every d1, d2 ∈ EI .

In the sequel we can omit parentheses when no ambiguity occurs. The followingusual simplifications will be used:

• d1d2 for d1 × d2;

• −d for (−1)× d.

Let X be a set of real variables such that I ⊆ X. The set of real expressionsEX is the smallest set containing all real numbers and variables and is closedunder addition, multiplication, exponentiation, logarithm and summation. Thatis, EX is inductively defined as follows:

• R ∪X ⊆ EX ;

• (e1 + e2), (e1 × e2), (ee21 ), loge1

(e2) ∈ EX for every e1, e2 ∈ EX ;

•(

d2∑

k=d1

e

)

for every d1, d2 ∈ EI and e ∈ EX .

Observe that EI is contained in EX . As before we can omit parentheses when noambiguity occurs. We use the following abbreviations:

• e1

e2

for e1 × e−12 ;

• −e for (−1)× e;

• e1e2 for e1 × e2.

We need to compare two expressions e1 and e2, namely, checking whethere1 = e2 or e2 ≤ e2.

Otherwise stated, we use the letters i, j, k, m, n, i1, i2, j1, . . . for integer vari-ables and the letters x, y, z, x1, x2, y1, . . . for real variables.

Semantics

We will interpret integer expressions as integers and real expressions as real num-bers. For this purpose we need the notion of assignment.

An assignment ρ is a map that assigns to each variable a real number, that is,ρ : X → R such that ρ(i) ∈ Z for all i ∈ I. Given an assignment, the denotationis a function that associates an expression with a real number. However, someexpressions can not be interpreted, e.g. 1

0. Hence, the denotation is a partial

function [[·]]ρ : EX 6→ R defined as follows:


• [[a]]ρ = a for each a ∈ R;

• [[x]]ρ = ρ(x) for each x ∈ X;

• [[e1 + e2]]ρ =

[[e1]]ρ + [[e2]]ρ if [[e1]]ρ and [[e2]]ρ are defined,

undefined otherwise;

• [[e1 × e2]]ρ =

[[e1]]ρ × [[e2]]ρ if [[e1]]ρ and [[e2]]ρ are defined,


• [[ee21 ]]ρ =

[[e1]][[e2]]ρρ if [[e2]]ρ is defined and ([[e1]]ρ >0 or

([[e1]]ρ =0 and[[e2]]ρ >0) or ([[e1]]ρ <0 and [[e2]]ρ∈Z),


• [[loge1(e2)]]ρ =

log[[e1]]ρ([[e2]]ρ) if [[e1]]ρ > 0 and [[e2]]ρ > 0,


•[[

d2∑

k=d1

e

]]

ρ

=

0 if [[d1]]ρ > [[d2]]ρ,

[[e]]ρ′ +

[[d2∑

k=d1+1

e

]]

ρ

if [[d1]]ρ ≤ [[d2]]ρ and both terms

are defined,


where

ρ′(x) =

ρ(x) if x 6= k

[[d1]]ρ otherwise.

To check if e1 = e2 amounts to verify if [[e1]]ρ = [[e2]]ρ for all ρ such that bothdenotations are defined. Similarly for e1 ≤ e2.

Example 4.2.1 In order to check that

2∑

k=1

k2 = 5

holds, we have to establish that

[[2∑

k=1

k2

]]

ρ

= [[5]]ρ


for every assignment ρ. Indeed:

[[2∑

k=1

k2

]]

ρ

= 12 +

[[2∑

k=1+1

k2

]]

ρ

= 1 + 22 +

[[2∑

k=1+1+1

k2

]]

ρ

= 1 + 4 + 0 = 5 = [[5]]ρ.

The denotation of an expression only depends on the values assigned to thevariables occurring in the expression.

Lemma 4.2.2 Let e ∈ EX . Then

[[e]]ρ = [[e]]ρ′

for all assignments ρ, ρ′ such that ρ(x) = ρ′(x) for every variable x occurring ine when both denotations are defined.

Proof: The proof follows by structural induction

Basis: e ∈ R ∪X. Let ρ, ρ′ be assignments such that ρ(x) = ρ′(x) for everyx ∈ R occurring in e. If e ∈ R then [[e]]ρ = e = [[e]]ρ′ . Otherwise, e ∈ X and then[[e]]ρ = ρ(e) = ρ′(e) = [[e]]ρ′ .

Step: We only consider the cases where e is e1 + e2 and where e is a summation.

(1) e is e1 + e2. Let ρ, ρ′ be such that ρ(x) = ρ′(x) for every variable x occurringin e and [[e]]ρ and [[e]]ρ′ are both defined. Then [[ei]]ρ = [[ei]]ρ′ for i = 1, 2, by theinduction hypothesis. Hence,

[[e1 + e2]]ρ = [[e1]]ρ + [[e2]]ρ = [[e1]]ρ′ + [[e2]]ρ′ = [[e1 + e2]]ρ′ .

(2) e is∑d2

k=d1e′. Let ρ, ρ′ be such that ρ(x) = ρ′(x) for every variable x occurring

in e and [[e]]ρ and [[e]]ρ′ are both defined. By the induction hypothesis, [[di]]ρ = [[di]]ρ′for i = 1, 2.

(2.1) [[d1]]ρ > [[d2]]ρ. Then also [[d1]]ρ′ > [[d2]]ρ′ and therefore

[[d2∑

k=d1

e′

]]

ρ

= 0 =

[[d2∑

k=d1

e′

]]

ρ′


(2.2) [[d1]]ρ ≤ [[d2]]ρ. Then also [[d1]]ρ′ ≤ [[d2]]ρ′ . We have to prove that

[[d2∑

k=d1

e′

]]

ρ

=

[[d2∑

k=d1

e′

]]

ρ′

(4.2)

Let n = [[d2]]ρ − [[d1]]ρ(= [[d2]]ρ′ − [[d1]]ρ′). Note that proving (4.2) is equivalent toproving

[[d2∑

k=d2−n

e′

]]

ρ

=

[[d2∑

k=d2−n

e′

]]

ρ′

The proof follows by induction on n.

Basis: n = 0. Let ρ′′ be the assignment such that ρ′′(x) = ρ(x) for x 6= k andρ′′(k) = [[d2]]ρ and let ρ′′′ be the assignment such that ρ′′′(x) = ρ′(x) for x 6= kand ρ′′′(k) = [[d2]]ρ′ . Clearly, ρ′′(x) = ρ′′′(x) for every x ∈ X occurring in e, andtherefore in e′. Hence, by the induction hypothesis of the structural induction,[[e′]]ρ′′ = [[e′]]ρ′′′ . As a consequence,

[[d2∑

k=d2

e′

]]

ρ

= [[e′]]ρ′′ = [[e′]]ρ′′′ =

[[d2∑

k=d2

e′

]]

ρ′

.

Induction hypothesis:[[

d2∑

k=d2−n+1

e′

]]

ρ

=

[[d2∑

k=d2−n+1

e′

]]

ρ′

(4.3)

Step: Let ρ′′ be the assignment such that ρ′′(x) = ρ(x) for x 6= k and ρ′′(k) =[[d2 − n]]ρ and let ρ′′′ be the assignment such that ρ′′′(x) = ρ′(x) for x 6= k andρ′′′(k) = [[d2 − n]]ρ′ . Again ρ′′(x) = ρ′′′(x) for every x ∈ X occurring in e′, andtherefore, by the induction hypothesis of the structural induction, [[e′]]ρ′′ = [[e′]]ρ′′′ .Hence, using also (4.3),

[[d2∑

k=d2−n

e′

]]

ρ

= [[e′]]ρ′′+

[[d2∑

k=d2−n+1

e′

]]

ρ

= [[e′]]ρ′′′+

[[d2∑

k=d2−n+1

e′

]]

ρ′

=

[[d2∑

k=d2−n

e′

]]

ρ′

QED

We now establish some properties of summation that we will use latter on forsymbolically reasoning with summations.

Proposition 4.2.3 Let e, e′, c ∈ EX , d1, d2, d′1, d

′2 ∈ EI and k ∈ I such that k

does not occur in c. Then the following properties hold:


1. Distributivity

d2∑

k=d1

c e = c

d2∑

k=d1

e.

2. Associativity

d2∑

k=d1

(e + e′) =

d2∑

k=d1

e +

d2∑

k=d1

e′.

3. Constantd2∑

k=d1

c =

0 if d1 < d2

c(d2 − d1 + 1) otherwise.

4. Additivity of indices

d2∑

k=d1

e +

d3∑

k=d2+1

e =

0 if d1 > d2 and d2 + 1 > d3∑d2

k=d1e if d1 ≤ d2 and d2 + 1 > d3

∑d3

k=d2+1 e if d1 > d2 and d2 + 1 ≤ d3∑d3

k=d1e otherwise.

5. Change of variable

d2∑

k=d1

e =d+d2∑

k=d+d1

ek(k−d) =

d−d1∑

k=d−d2

ek(d−k)

where ekd′ is the expression obtained from e by replacing the occurrences of

k by d′.

Proof: We only present the proof of first property, leaving the proofs of the otherproperties as exercises.

We have to prove that

[[d2∑

k=d1

c e

]]

ρ

=

[[

c

d2∑

k=d1

e

]]

ρ

(4.4)

for every assignment ρ where the denotation is defined. We consider two cases.


(1) [[d1]]ρ > [[d2]]ρ. Then,

[[d2∑

k=d1

c e

]]

ρ

= 0 = [[c]]ρ × 0 = [[c]]ρ ×[[

d2∑

k=d1

e

]]

ρ

=

[[

cd2∑

k=d1

e

]]

ρ

.

(2) [[d1]]ρ ≤ [[d2]]ρ. Then, let n = [[d2]]ρ − [[d1]]ρ. Observe that showing (4.4) isequivalent to showing

[[d2∑

k=d2−n

c e

]]

ρ

=

[[

cd2∑

k=d2−n

e

]]

ρ

.

The proof follows by induction on n.

Basis: n = 0. Take ρ′ to be the assignment such that ρ′(x) = ρ(x) for x 6= k andρ′(k) = [[d2]]ρ. Then,

[[d2∑

k=d2

c e

]]

ρ

= [[c e]]ρ′ = [[c]]ρ′ × [[e]]ρ′ = [[c]]ρ ×[[

d2∑

k=d2

e

]]

ρ

=

[[

c

d2∑

k=d2

e

]]

ρ

since [[c]]ρ′ = [[c]]ρ given that k does not occur in c.

Induction Hypothesis:

[[d2∑

k=d2−n+1

c e

]]

ρ

=

[[

c

d2∑

k=d2−n+1

e

]]

ρ

.

Step: Take ρ′ to be the assignment such that ρ′(x) = ρ(x) for x 6= k and ρ′(k) =[[d2 − n]]ρ.


[[d2∑

k=d2−n

c e

]]

ρ

= [[c e]]ρ′ +

[[d2∑

k=d2−n+1

c e

]]

ρ

= [[c]]ρ′ × [[e]]ρ′ +

[[

c

d2∑

k=d2−n+1

e

]]

ρ

(induction hypothesis)

= [[c]]ρ × [[e]]ρ′ + [[c]]ρ ×[[

d2∑

k=d2−n+1

e

]]

ρ

= [[c]]ρ

[[e]]ρ′ +

[[d2∑

k=d2−n+1

e

]]

ρ

= [[c]]ρ ×[[

d2∑

k=d2−n

e

]]

ρ

=

[[

c

d2∑

k=d′

e

]]

ρ

QED

Symbolic reasoning

We will use the properties described in Proposition 4.2.3, together with the well-known properties of real expressions, to reason about summations in a symbolicway (that is, not invoking semantic arguments). We will check the followingproperty

5∑

j=3

e +

7∑

j=4

e =

7∑

j=3

e +

5∑

j=4

e.

Clearly,


5∑

j=3

e +7∑

j=4

e =5∑

j=3

e +

(5∑

j=4

e +7∑

j=5

e

)

(additivity)

=

(5∑

j=3

e +

7∑

j=5

e

)

+

5∑

j=4

e

=

7∑

j=3

e +

5∑

j=4

e (additivity)

Using the above properties of summations we can now compute the worst-casenumber of comparisons carried out by the algorithm presented in Section 4.1. Infact, we only need the constant sequence property.

Example 4.2.4 motivating example revisited

In Section 4.1 we had to compute the summation

n−m∑

i=0

(m∑

j=1

1

)

which can now be computed as follows:

n−m∑

i=0

(m∑

j=1

1

)

=n−m∑

i=0

m (constant)

= m(n−m + 1) (constant)

using Proposition 4.2.3.

We will use symbolic reasoning to simplify summations. In many cases, givena summation

d2∑

k=d1

e

we want to obtain an expression e′ such that

d2∑

k=d1

e = e′

and e′ does not have summations. In this case, e′ is called a closed form for thesummation.

We will consider closed forms for the summations of members of both arith-metic and geometric progressions.


Example 4.2.5 We start by finding a closed form for the sum of members ofan arithmetic progression. Recall that an arithmetic progression is a sequenceukk∈N0 such that uk+1−uk is the same for all k ∈ N0. All arithmetic progressionsare of the form

uk = (c + rk).

We would like to consider the summation of the first n + 1 members of ukk∈N0,that is

n∑

k=0

(c + rk).

The following equality holds for n ≥ 0

n∑

k=0

(c + rk) =(

c +rn

2

)

(n + 1).

The equality is established as follows.

(1)n∑

k=0

(c + rk) =

n−0∑

k=n−n

(c + r(n− k)) (change of variable)

=

n∑

k=0

(c + rn− rk)

(2)

2

n∑

k=0

(c + rk) =

n∑

k=0

(c + rk) +

n∑

k=0

(c + rn− rk)

=n∑

k=0

(c + rk + c + rn− rk) (associativity)

=n∑

k=0

(2c + rn)

= (2c + rn)n∑

k=0

1 (distributivity)

= (2c + rn)(n + 1) (constant)


(3)

n∑

k=0

(c + rk) =(

c +rn

2

)

(n + 1)

Example 4.2.6 We now find a closed form for the sum of members of an geo-metric progression. Recall that an geometric progression is a sequence ukk∈N0

such thatuk+1

uk

is the same for every k ∈ N0. All geometric progressions are of the form

uk = crk.

We would like to consider the summation of the first n + 1 members of ukk∈N0,that is

n∑

k=0

crk.

The following equality holds

n∑

k=0

crk =c− crn+1

1− r.

In fact:

(1)

(n∑

k=0

crk

)

+ crn+1 =

n+1∑

k=0

crk (additivity)

= cr0 +

n+1∑

k=1

crk (additivity)

= cr0 +

n∑

k=1

crk+1 (change of variable)

= c + r

n∑

k=1

crk (distributivity)


(2)

n∑

k=0

crk =c− crn+1

1− r

As a consequence of Examples 4.2.5 and 4.2.6 we get the equalities

n∑

k=0

k =n(n + 1)

2and

n∑

k=0

rk =1− rn+1

1− r

where n ≥ 0 and r 6= 1.

The perturbation technique is an useful way of finding closed forms of sum-mations. It works as follows. Assume we want to find a closed form for thesummation

n∑

k=0

ak

where n > 0. For simplicity let sn abbreviate∑n

k=0 ak. We can rewrite sn+1 intwo different ways: one by splitting off its last term, an+1, and another by splittingoff its first term, a0. In many cases, when we rewrite both the expressions forsn+1 in terms of sn we obtain an equation for sn whose solution is a closed formof the summation.

In fact, using additivity, on one hand we have

sn+1 =

n+1∑

k=0

ak =

n∑

k=0

ak + an+1

and, on the other hand,

sn+1 =n+1∑

k=0

ak = a0 +n+1∑

k=1

ak

= a0 +

n∑

k=0

ak+1

using also a change of variable. Thus, we get the equation

n∑

k=0

ak + an+1 = a0 +

n∑

k=0

ak+1


that is

sn + an+1 = a0 +

n∑

k=0

ak+1 (4.5)

The goal is now to express the right-hand side of 4.5 in terms of sn in such a waythat solving the equation for sn results in a closed form for sn. Note that this isnot always possible. Let us illustrate this method with the following example.

Example 4.2.7 Consider the summation

n∑

k=0

kak

where n is a nonnegative integer and a is an integer not equal to 1. Let us usethe perturbation technique to find a closed form for this summation. Let sn

abbreviate∑n

k=0 kak. Using equation 4.5 we get

sn + (n + 1)an+1 = 0a0 +n∑

k=0

(k + 1)ak+1

and, using associativity and distributivity for rewriting the right-hand side interms of sn,

sn + (n + 1)an+1 = a

n∑

k=0

kak + a

n∑

k=0

ak

that is, the following equation involving the unknown sum sn

sn + (n + 1)an+1 = asn + a1− an+1

1− a(4.6)

Solving equation 4.6 for sn we obtain the intended closed form

− 1

1− aan+1(n + 1) + a

1− an+1

(1− a)2.

Summations over countable sets can also be considered. Let f : A→ B be amap where A is a countable set and B is set of real numbers. If A = a1, . . . , anis a finite set then

∑

a∈A

f(a) =

n∑

i=1

f(ai).

If A is an infinite countable set then∑

a∈A

f(a) = L

where L = min(L ∈ R :∑

a∈C f(a) ≤ L for all finite C ⊂ A), whenever this realnumber L exists. Note that, in particualr,

∑

a∈N0f(a) = lim

n

∑na=0 f(a).


4.3 Main results

One of the most important techniques to obtain a closed formula for a sum is byapproximation to an integral. The central result was discovered independentlyby Leonhard Euler and Colin Maclaurin in the XVIII century.

Let f : [0, n]→ R be p times differentiable. Denote by f (p) the pth derivativeof f . Then the Euler-Maclaurin formula is as follows:

n∑

k=0

f(k) =

∫ n

0

f(x) dx−B1(f(n) + f(0)) +

p∑

k=2

Bk

k!

(f (k−1)(n)− f (k−1)(0)

)+ Rp.

(4.7)where Bkk∈N0 is the sequence of Bernoulli numbers inductively defined as fol-lows

B0 = 1 and Bk+1 = − 1

k + 2

k∑

j=0

(k + 2

j

)

Bj (4.8)

and Rp, p ≥ 2, is the remainder term such that

|Rp| ≤2ζ(p)

(2π)p

∫ n

0

∣∣f (p)(x)

∣∣ dx. (4.9)

Given a non-negative integer n and an integer k the binomial coefficient isdefined as follows:

(nk

)

=

n!

(n− k)!k!if 0 ≤ k ≤ n

0 otherwise.

Informally, it represents the number of ways that k objects can be chosen amongn objects when order is irrelevant. Hence, we can see the binomial coefficient asrepresenting the number of k-element subsets of an n-element set. Its name isderived from the fact that they constitute the coefficients of the series expansionof a power of a binomial, that is,

(1 + x)n =

∞∑

k=0

(nk

)

xk.

There is also an alternative recursive definition of binomial coefficient:(

n0

)

=

(nn

)

= 1

where (nk

)

=

(n− 1k − 1

)

+

(n− 1

k

)

4.4. EXAMPLES 145

which is the reason why the Pascal’s triangle is valid.

Bernoulli numbers are associated to the interesting fact that the first com-puter program is recognized to be a Bernoulli calculation program for specializedcalculus operations in the Babbage’s engine, program proposed by Ada Byron toCharles Babbage approximately in the year 1850.

In (4.9), ζ is the Riemmann zeta function. Since 1 < ζ(p) < 2 for everyinteger p ≥ 2, it holds that

|Rp| ≤4

(2π)p

∫ n

0

∣∣f (p)(x)

∣∣ dx.

The proof of the validity of the Euler-Maclaurin formula can be seen in [24].

Example 4.3.1 Consider the map f such that f(k) = k2 and take p = 3. Thenf (p) is such that f (p)(x) = 0 and so Rp = 0. Using (4.8) we get that B1 = −1/2,B2 = 1/6 and B3 = 0. Moreover f (1)(n) = 2n and f (2)(n) = 2 and so from theformula (4.7) we get

n∑

k=0

k2 =1

3n3 +

1

2n2 +

1

6n.

Observe that the Euler-Maclaurin formula gives an exact value for the sum-mation whenever f(k) is a polynomial kp, for any p ∈ N. In fact, in this casef (p+1) is such that f (p+1)(x) = 0 and therefore Rp+1 is 0.

Proposition 4.3.2 For each n ∈ N and p ∈ N0

n∑

k=0

kp =1

p + 1

(p∑

k=0

(p + 1

k

)

Bk (n + 1)p+1−k

)

.

The Euler-Maclaurin formula is useful when obtaining a closed formula of asummation

∑n

k=0 f(k) where f (p) is 0 for some natural number p.

4.4 Examples

We further illustrate the relevance of summations in algorithm analysis. We firstconsider the Gaussian elimination technique, then we analyze the insertion sortalgorithm.


4.4.1 Gaussian elimination

We now briefly describe Gaussian elimination technique (for a general descriptionsee [29] and [13]).

Consider a system Ax = b of n linear equations with n variables where n > 0.Assume that A is nonsingular, that is, det(A) 6= 0. Hence the system has a uniquesolution x = A−1b. The Gaussian elimination technique computes the solution x.The objective is to obtain an equivalent system of equations A′x = b′, that is, asystem of equations with the same solution as Ax = b and where the computationof the solution is much easier. For this purpose, we define a sequence of systemsof equations

A(k)x = b(k)k∈0,...,n−1

such that

1. A(0) = A, b(0) = b, A(n−1) = A′ and b(n−1) = b′;

2. A(k) is a triangular matrix up to column k;

3. The first k rows of A(k−1) are equal to the first k rows of A(k).

From 2 and 3, we only need to say how to obtain a(k)ij and b

(k)i from A(k−1) and

b(k−1) for 0 < k < i, j ≤ n. Assuming that a(k−1)kk 6= 0 then

a(k)ij = a

(k−1)ij − a

(k−1)ik

a(k−1)kk

a(k−1)kj b

(k−1)i = b

(k−1)i − a

(k−1)ik

a(k−1)kk

b(k−1)k . (4.10)

The Gaussian elimination technique described above can be easily imple-mented in Mathematica as depicted in Figure 4.2. It is straightforward to provethat A(k)x = b(k) is equivalent to Ax = b for k = 0, . . . , n− 1.

The Mathematica function GaussElimination in Figure 4.2 receives as inputa nonsingular square matrix A and a vector b such that the dimension of b isequal to the order of A. Moreover, A is such that the denominators in (4.10) arealways nonzero. The output is a triangular matrix Aux and a vector baux suchthat the linear equation systems Ax = b and Auxx = baux are equivalent. Itconsists of three nested loops that implement the Gaussian elimination techniquedescribed above. The most inner loop computes the elements of Aux and baux asdescribed in (4.10).

We now analyze the algorithm in terms of the number of arithmetic operationsinvolving matrix elements. Note that this number only depends on the numberof rows and columns of the given matrix.

Proposition 4.4.1 The algorithm for Gaussian elimination presented in Figure4.2 is cubic in the order of the input matrix, that is, the map f : N→ N0 such thatf(n) is the number of arithmetic operations involving matrix entries performedby the algorithm when it receives as input a n× n matrix is in O(λn.n3).

4.4. EXAMPLES 147

GaussElimination=

Function[A,b,Module[k,i,j,m,n,Aux,baux,Aux=A;

baux=b;

n=Length[b];

k=1;

While[k<=n-1,

i=k+1;

While[i<=n,

m=Aux[[i,k]]/Aux[[k,k]];

Aux[[i,k]]=0;

j=k+1;

While[j<=n,

Aux[[i,j]]=Aux[[i,j]]-m*Aux[[k,j]];

j=j+1];

baux[[i]]=baux[[i]]-m*baux[[k]];

i=i+1];

k=k+1];

Aux,baux]];

Figure 4.2: Gaussian elimination in Mathematica


Proof: Consider a n× n input matrix.(1) To begin with let us count the number of sums and subtractions involvingmatrix elements. The execution of each iteration of the most inner loop performsthe subtraction Aux[[i,j]]-m*Aux[[k,j]] and therefore the execution of theloop involves exactly

n∑

j=k+1

1

subtractions. Looking now at the next loop, the execution of each iterationperforms

n∑

j=k+1

1

subtractions plus the subtraction baux[[i]]-m*baux[[k]]. Thus, there are

n∑

i=k+1

(

1 +n∑

j=k+1

1

)

subtractions. Finally, the total number of sums and subtractions is

n−1∑

k=1

n∑

i=k+1

(

1 +n∑

j=k+1

1

)

We can use the techniques in Section 4.2 to compute this summation. Moreprecisely, we prove that

n−1∑

k=1

n∑

i=k+1

(

1 +n∑

j=k+1

1

)

=n3 − n

3(4.11)

In fact,

n−1∑

k=1

n∑

i=k+1

(

1 +n∑

j=k+1

1

)

=n−1∑

k=1

n∑

i=k+1

(1 + (n− k)) (constant)

=n−1∑

k=1

((n− k)2 + (n− k)) (constant)

=

n−1∑

k=1

(k2 + k) (change of variable)

=

(n−1∑

k=1

k2

)

+

(n−1∑

k=1

k

)

(associativity)

4.4. EXAMPLES 149

The equality (4.11) then follows since from Example 4.3.1

n−1∑

k=1

k2 =(n− 1)3

3+

(n− 1)2

2+

n− 1

6

and from Example 4.2.5n−1∑

k=1

k =n(n− 1)

2

(2) With respect to multiplications and divisions involving matrix elements, wecan reason as in the case of sums and subtractions and conclude that the totalnumber of such multiplications and divisions is

n−1∑

k=1

n∑

i=k+1

(

2 +

n∑

j=k+1

1

)

.

The computation of this sum is similar to the one presented in (1) and we get

n−1∑

k=1

n∑

i=k+1

(

2 +

n∑

j=k+1

1

)

=

(n−1∑

j=1

j2

)

+ 2

(n−1∑

j=1

j

)

=2n3 + 3n2 − 5n

6.

(3) From (1) and (2) we conclude that the map f is such that

f(n) =n3 − n

3+

2n3 + 3n2 − 5n

6=

5n3 + 3n2 − 7n

6

for each n ∈ N, and therefore f ∈ O(λn.n3). QED

4.4.2 Insertion sort

The problem of sorting consists in, given a list w of numbers, obtaining an orderedlist that is a permutation of w.

There are many sorting algorithms, among them, the insertion sort algorithm(for more details on sort algorithms, the reader can consult [2]). At the kth stepof the insertion sort algorithm the list w is such that the elements w1, . . . , wk arealready ordered and the goal is to put wk+1 in the proper position with respectto w1, . . . , wk. An implementation in Mathematica of the insertion sort algorithmis given in Figure 4.3.

The Mathematica function InsertSort in Figure 4.3 receives as input a listw to be ordered and it gives as output the corresponding ordered list v. At thebeginning v is set to w. The function consists of two nested loops. At the kth


InsertSort=Function[w,Module[v,i,j,m,i=2;

v=w;

While[i<=Length[w],

j=i-1;

m=v[[i]];

While[j>0&&v[[j]]>m,

v[[j+1]]=v[[j]];

j=j-1];

v[[j+1]]=m;

i=i+1];

v]];

Figure 4.3: Insertion sort algorithm in Mathematica

iteration of the outer loop the list v is considered to be divided in two parts. Thefirst k elements of v are the first k elements of w already ordered. The elementsof v and w from position k + 1 to the last position are the same. The orderedsublist is extended to the k + 1th position by inserting the k + 1th element of win the correct position The loop ends when there are no more elements to sort.Then, v is the intended output list.

A run of the insertion sort algorithm for the input list w = 2,−5, 43,−10, 8is depicted in Figure 4.4. We denote by v′ the ordered part of the list and by v′′

the remaining part. This run has four steps. In the first step, −5 is comparedwith 2 and becomes the first element of v′. Then, 43 is compared with the lastelement of v′ and remains in the original position since it is greater than 2. Inthe last two steps −10 and 8 are inserted in the correct positions.

In the analysis of a sorting algorithm the relevant operations are the compar-isons between list elements carried out by the algorithm to sort the list. We nowanalyze the insertion sort algorithm in terms of the number of comparisons

v[[j]] > m

performed by the algorithm to sort a list v of length n. This number depends onthe length n of the list and also on how “unsorted” the list is.

Worst-case analysis

In worst-case analysis we concentrate on finding the maximum number of com-parisons needed to sort an input list of length n or, at least, an upper bound forthis value.

4.4. EXAMPLES 151

v = 2︸︷︷︸

,−5, 43,−10, 8︸︷︷︸

v′ v′′

⇓v = −5, 2

︸︷︷︸, 43,−10, 8︸︷︷︸

v′ v′′

⇓v = −5, 2, 43

︸︷︷︸, −10, 8︸︷︷︸

v′ v′′

⇓v = −10,−5, 2, 43

︸︷︷︸, 8︸︷︷︸

v′ v′′

⇓v = −10,−5, 2, 8, 43

︸︷︷︸︸︷︷︸

v′ v′′

Figure 4.4: A run of the insertion sort algorithm

Proposition 4.4.2 The insertion sort algorithm presented in Figure 4.4 is, in theworst-case, quadratic in the length of the input list, that is, the map f : N→ N0

such that f(n) is the maximum number of comparisons between list elementsperformed by the algorithm when it receives as input a list of length n is inO(λn.n2).

Proof: Consider an input list w = w1, . . . wn of length n ≥ 1. The maximumnumber of comparisons occurs whenever the input list is sorted by (strict) de-creasing order, that is, wi > wi+1 holds for all 1 ≤ i < n. In fact, in this case, toinsert the i-th element in the correct position we have to compare it with all theelements to its left. Hence, i − 1 comparisons are executed. Note that Mathe-matica evaluates conjunctions in order, that is, when evaluating j>0&&v[[j]]>m,for instance, if j>0 is false then v[[j]]>m is not evaluated. Since 2 ≤ i ≤ n, thetotal number of comparisons to sort the input list is given by the value of thesummation

n∑

i=2

(i− 1).

Using a change of variable and Example 4.2.5,

n∑

i=2

(i− 1) =

n−1∑

j=1

j =(n− 1)n

2.


Hence, the map f is such that

f(n) =(n− 1)n

2=

n2 − n

2


Average-case analysis

The average-case analysis is in general more elaborated than the worst-case anal-ysis since it involves probabilistic hypothesis. Recall that the (i − 1)-th step ofthe outer loop of the function in Figure 4.3 places the i-th element of the list inthe right ordered position, ranging from 1 to i. In the sequel we assume that theprobability of the element to be placed in any position is

1

i

that is, the probability of the element to be placed in any position is the same(uniform distribution).

As we will see in the proof of Proposition 4.4.4 below, the average-case analysisof the insertion sort algorithm involves sums such as

n∑

i=1

1

i. (4.12)

The sum (4.12) is the n-th harmonic number and it is usually denoted by Hn.Harmonic numbers constitute the discrete version of the natural logarithm. Thereis no closed form for Hn, but it is easy to establish upper and lower bounds forthe value of Hn.

Proposition 4.4.3 For each n ∈ N

ln(n) ≤ Hn ≤ ln(n) + 1.

Proof: The result follows by definition of the Riemann integral. On one hand,

Hn − 1 =n∑

i=2

1

i≤∫ n

1

1

xdx = ln(n).

On the other hand,

Hn =n∑

i=1

1

i≥∫ n

1

1

xdx = ln(n).

QED

We now present the average-case analysis of the insertion sort algorithm pre-sented in Figure 4.3.

4.4. EXAMPLES 153

Proposition 4.4.4 The insertion sort algorithm presented in Figure 4.4 is, inthe average case, quadratic in the length of the input list, that is, the mapf : N→ N0 such that f(n) is the average number of comparisons between listelements performed by the algorithm when it receives as input a list of lengthn ≥ 1 is in O(λn.n2).

Proof: Consider an input list of length n ≥ 1.(1) At the (i− 1)-th step of the outer loop of the function in Figure 4.3 the i-thelement of the list is placed in the right ordered position, ranging from 1 to i.Recall that we are assuming that the probability of this element to be placed inany position is the same, that is,

1

i.

If the i-th element is placed in position k ≥ 2, the number of required comparisonsis i− k + 1. Moreover, if it is placed in the first position the number of requiredcomparisons is i − 1. Furthermore, the average-case number of comparisons toplace the i-th element in the correct position is

1

i(i− 1) +

(i∑

k=2

1

i(i− k + 1)

)

.

Hence, the average-case number of required comparisons to sort the input list isgiven by the summation

n∑

i=2

(

1

i(i− 1) +

(i∑

k=2

1

i(i− k + 1)

))

.

Let us find a closed form of the above summation. Since

i∑

k=2

1

i(i− k + 1) =

1

i

i∑

k=2

(i− k + 1) (distributivity)

=1

i

i−2∑

j=0

(j + 1) (change of variable)

=1

i

i(i− 1)

2(Example 4.2.5)

=i− 1

2


we have that

n∑

i=2

(1

i(i− 1)+

(i∑

k=2

1

i(i− k + 1)

))

=n∑

i=2

(1

i(i− 1) +

i− 1

2

)

=

(n∑

i=2

1 + i

2

)

−(

n∑

i=2

1

i

)

(associativity)

=1

2

(n∑

i=0

(i + 1)

)

−(

n∑

i=1

1

i

)

− 1

2(distributivity)

=n2 + 3n

4−(

n∑

i=1

1

i

)

(Example 4.2.5)

(2) The map f is then such that

f(n) =n∑

i=2

(

1

i(i− 1) +

(i∑

k=2

1

i(i− k + 1)

))

=n2 + 3n

4−(

n∑

i=1

1

i

)

for each n ∈ N. From Proposition 4.4.3 it follows that

f(n) ≤ n2 + 3n

4− ln(n)


4.5 Exercises

1. Compute the Bernoulli numbers B1, B2 and B3.

2. Prove that the Bernoulli number Bk is equal to 0 for every odd integer kgreater than 1.

3. Prove that

(a)∑n

k=1 3k2k = 6(1− 2n + 2nn)

4.5. EXERCISES 155

(b)∑n

k=1 2k(−2)k = 49(−1 + (−2)n + 3(−2)nn)

(c)∑n

k=1 k2 = n3

3+ n2

2+ n

6

(d)∑n

i=1 i3 = n2(n+1)2

4

(e)∑n

i=12

i(i+1)= 2n

1+n

(f)∑n

k=11

3(k+k2)= n

3n+3

where n is a positive integer.

4. Compute the following sums where n is a positive integer

(a)∑n

k=0(n2k + 3k)

(b)∑n

k=−2 n(k2 − k)

(c)∑n

k=0(6k2 + n2k + 3k)

(d)∑n

k=0 3kk

(e)∑n+1

k=1(3k2 + k3k + 5k)

(f)∑n

k=0(6(n− k)2 + k2k)

(g)∑n+1

k=−1((n− k)5n−k+2 + 2(k − 4))

(h)∑n+2

k=−3 k3

5. Prove that∑n−1

k=0(ak+1−ak)bk = anbn−a0b0− (∑n−1

k=0(bk+1− bk)ak+1) wheren is a positive integer.

6. Consider the Mathematica function f that receives as input a matrix m ofinteger numbers and computes an integer a using a function h.

f=Function[m,Module[i,j,a,nlin,ncol,nlin=Length[m];

ncol=Length[First[m]];

a=1;

i=2;

While[i<nlin,

j=1;

While[j<=ncol,

a=a*h[m,i,j];

j=j+1];


i=i+1];

a]]

Compute the number of multiplications performed by an evaluation of f[m],assuming that the evaluation of h[m,i,j] performs i + j multiplications.

7. Consider the Mathematica function f that receives as input a n× n matrixw of integer numbers, where n > 1, and computes an integer a using afunction p and a function h.

f=Function[w,Module[a,i,j,n,n=Length[w];

a=1;

i=1;

While[i≤n-1,a=a+p[i+1];

j=1;

While[j≤n,a=w[[i,j]]*a*t[j,i];

j=j+1];

i=i+1];

a]];

Compute the number of multiplications performed by an evaluation of f[w],assuming that the evaluation of p[x] performs 3x multiplications and theevaluation of t[x,y] performs x + y multiplications.

8. Consider the Mathematica function matrixMemberQ implementing an algo-rithm for linear search.

matrixMemberQ=Function[m,x,Module[i,j,found,i=1;

found=False;

While[!found&&i<=Length[m],

j=1;

While[!found&&j<=Length[m[[1]]],

If[m[[i,j]]==x,found=True];

j=j+1];

i=i+1];

found]]

Assuming that the integer x is uniformly distributed in the matrix of inte-gers m, analyze the average-case number of times the expression m[[i,j]]==x

is evaluated during an evaluation of matrixMemberQ[m,x].

4.5. EXERCISES 157

9. Consider the following Mathematica function f

f=Function[m,x,Module[i,j,r,menoresQ,i=; r=0;

While[i<=Length[m],

j=1;

menoresQ=True;

While[menoresQ&&j<=Length[m[[1]]],

If[m[[i,j]]<x,r=r+1,menoresQ=False];

j=j+1];

i=i+1];

r]]

Consider an evaluation of f[m,x] where m is a matrix of integers and x

is an integer.

(a) Analyze the average-case number of times the expression m[[i,j]]<x

is evaluated assuming that the rows of m are ordered in increasing orderand that x occurs in each row exactly once.

(b) Analyze the average-case number of times the expression m[[i,j]]<x

is evaluated assuming that the rows of m are ordered in increasingorder.

Chapter 5

Discrete Fourier transform

In this chapter we introduce the discrete Fourier transform The discrete Fouriertransform is widely used in many fields, ranging from image processing to effi-cient multiplication of polynomials and large integers. In section 5.1 we presenta motivating example that illustrates the use of the discrete Fourier transformfor efficient polynomial multiplication. In Section 5.2 we introduce the discreteFourier transform. In Section 5.3 we present the fast Fourier transform, an ef-ficient method for computing the discrete Fourier transform. In Section 5.4 werevisit polynomial multiplication based on the discrete Fourier transform. Imageprocessing using the fast Fourier transfer is discussed in Section 5.5.

Several exercises are proposed in Section 5.6.

5.1 Motivation

Polynomial multiplication is crucial in many tasks, from signal processing to largeinteger multiplication and therefore efficient algorithms for polynomial multipli-cation are of utmost importance. Consider the polynomials of degree n− 1

p =

n−1∑

i=0

aixi and q =

n−1∑

j=0

bjxj

in R[x], for instance, and recall from Chapter 3 that their product is the polyno-mial

p× q =2n−2∑

k=0

ckxk

where, for each 0 ≤ k ≤ 2n− 2

159

160 CHAPTER 5. DISCRETE FOURIER TRANSFORM

ck =

k∑

i=0

ai × bk−i if 0 ≤ k ≤ n− 1

n−1∑

i=k−n+1

ai × bk−i if n− 1 < k ≤ 2n− 1.

The naive way of computing the polynomial p × q involves computing ai × bj

for each 0 ≤ i, j ≤ n− 1, that is, computing n2 multiplications of real numbers.Moreover, it also involves computing (n − 1) + 2

∑n−2i=1 i = (n − 1)2 sums. As

a consequence, we conclude that the naive way of computing the product oftwo polynomials of degree n− 1 in R[x] involves a O(n2) number of sums andmultiplications of real numbers.

There is however a very efficient technic for computing the product of twopolynomials in R[x] (or C[x]) that only involves a O(n logn) number of sums andmultiplications. It is based on the discrete Fourier transform and it also uses thepoint-value representation of polynomials. The celebrated Shor algorithm for fac-torizing integers in polynomial time in a quantum computer uses this technique.The main idea is depicted in Figure 5.1.

p, qcoefficient rep.

p× qcoefficient rep.

evaluationO(n log n)

p, qpoint-value rep.

pointwise multiplication

O(n)p× qpoint-value rep.

interpolationO(n log n)

Figure 5.1: Polynomial multiplication

Roughly speaking, the method can be described as follows. It first involvesthe evaluation of p and q at suitable values (the complex roots of unity), thusobtaining point-value representations of p and q. Then, from these point-valuerepresentations we get a point-value representation of p× q. Finally, interpola-tion is used to obtain the coefficients of p× q from its point-value representation.The evaluations of p and q can be computed with a O(n logn) number of sums andmultiplications using a particular way of computing the discrete Fourier trans-form (DFT), named fast Fourier transform (FFT). The interpolation can also becomputed using FFT again with a O(n log n) number of sums and multiplications.

5.2. DISCRETE FOURIER TRANSFORM 161

This technique can also be used for efficient multiplication of large integers.For this purpose, each integer number k is considered as the value of the evalu-ation of a suitable polynomial at a positive integer b, corresponding to the basewe are considering. The coefficients of the polynomial correspond to the digitsthe representation of k in base b.

5.2 Discrete Fourier transform

In this section we first recall some properties of complex numbers, namely, ofthe nth-roots of unity. Then we introduce discrete Fourier transform and somerelated properties.

5.2.1 Complex roots of unity

The set of complex number C is the extension of the set of real numbers for whichall polynomials with real coefficients of degree n have precisely n roots (includingmultiplicity). For this reason, the set of complex number is called the algebraicclosure of the set of real numbers.

Any complex number can be expressed as a + b i where a and b are real num-bers and i is called the imaginary unit. Addition and multiplication of complexnumbers are defined in the usual way, taking into account that i2 = −1, that is,

• (a + b i) + (c + d i) = (a + c) + (b + d)i;

• (a + b i)× (c + d i) = (ac− bd) + (ad + bc)i.

The set of complex numbers endowed with their addition and multiplicationconstitutes a field. A useful representation of complex numbers is their polarform, that is

(a + bi) = reiθ = r(cos(θ) + i sin(θ))

where r =√

a2 + b2 is called the modulus and θ = neg(a)π + arctan b/a is calledthe phase where neg(a) = 0 iff a > 0 and neg(a) = 1 otherwise.

Recall that in the field of complex numbers C, given n ∈ N, each nonzerocomplex number reiθ has exactly n distinct n-th roots

n√

r ei θ+2kπn for k ∈ 0, 1, . . . , n− 1.

Herein, we are particularly interested in the distinct n-th roots of the unity

zkn = ei 2kπ

n for k ∈ 0, 1, . . . , n− 1The n-th root of unity zn = z1

n = ei 2πn is the principal root. In the sequel we may

also refer to zkn for k ∈ Z. As expected, also in this case zk

n = cis(2kπn

) = ei 2kπn .

The n-th roots of unity enjoy several interesting properties. The followingproperties are useful in the sequel (see Exercises 1 and 2 in Section 5.6).


Proposition 5.2.1 Let n ∈ N and k ∈ Z. Then zkn = zn+k

n .

Proposition 5.2.2 Let k ∈ N0 and n, d ∈ N where n is an even number. Then,

zn2+d

n = −zdn and z2k

n = zkn2.

Using the above properties of the complex roots of unity we can speed up thecomputation of the n-th roots of unity when n is even. On one hand, note thatonce we have computed the first n

2roots of unity, that is

z0n, z1

n, . . ., zn2−1

n

then we can easily compute the other n2

roots: we have just to consider thesymmetric of the above since

• zn2n = z

n2+0

n = −z0n

• zn2+1

n = −z1n

. . .

• zn−1n = z

n2+(n

2−1)

n = −zn2−1

n

On the other hand, when we have already computed the n2-th roots of unity, we

can use those values to obtain half of the n-th roots:

• z0n = z2×0

n = z0n2

• z2n = z2×1

n = z1n2

. . .

• zn−2n = z

2(n2−1)

n = zn2−1

n2

Another relevant property is the following (see Exercise 3 in Section 5.6).

Proposition 5.2.3 Let k ∈ N0 and n ∈ N such that k is not divisible by n. Then

n−1∑

i=0

(zkn)i = 0.


5.2.2 Discrete Fourier transform

We now present the discrete Fourier transform and some related properties.

Definition 5.2.4 Let ~a = (a0, a1, . . . , an−1) ∈ Cn where n ∈ N. The discrete

Fourier transform of ~a is ~b = (b0, . . . , bn−1) where

bk =

n−1∑

j=0

ajzkjn

for each 1 ≤ k ≤ n− 1, and it is denoted by DFTn(a0, a1, . . . , an−1).

Note that, in particular, DFTn(a) = a for each a ∈ Cn, since z01 = 1.

In the sequel, DFTn(a0, a1, . . . , an−1)k denotes the kth component of the tupleDFTn(a0, a1, . . . , an−1), for each 1 ≤ k ≤ n.

Example 5.2.5 Consider ~a = (3,−2, 1, 0). Observe that z04 = 1, z1

4 = i, z24 = −1

and z34 = −i. Then,

• b0 =∑3

j=0 ajz04 =

∑3j=0 aj = 2;

• b1 =∑3

j=0 ajzj4 =

∑3j=0 aji

j = 3i0 + (−2)i1 + i2 = 2− 2i;

• b2 =∑3

j=0 ajz2j4 =

∑3j=0 aj(−1)j = 3(−1)0 + (−2)(−1)1 + (−1)2 = 6;

• b3 =∑3

j=0 ajz3j4 =

∑3j=0 aj(−i)j = 3(−i)0 + (−2)(−i)1 + (−i)2 = 2 + 2i.

Therefore, DFT4(3,−2, 1, 0) = (2, 2− 2i, 6, 2 + 2i).

Observe that the discrete Fourier transform is a particular linear transforma-tion described by a Vandermonde matrix. An m × n Vandermonde matrix for(α1, . . . , αm), denoted by Vn(α1, . . . , αm) is such that its entry at row i column jis αj

i , and so each row i is a sequence of a geometric progression with ratio αi.Indeed

DFTn(~a) = V · ~awhere

V =

1 z0n (z0

n)2 . . . (z0n)n−1

1 z1n (z1

n)2 . . . (z1n)n−1

......

.... . .

...1 zn−1

n (zn−1n )2 . . . (zn−1

n )n−1

is the square Vandermonde matrix for (z0n, z

1n, . . . , zn−1

n ). Moreover U = 1√nV is

unitary and so, for this normalized version, the DFT can be seen as a change ofbasis preserving the norm.

This matrix is invertible and therefore the matrix product


V −1

y0

y1...

yn−1

=

a0

a1...

an−1

corresponds to DFT−1n (y0, y1, . . . , yn−1) = (a0, a1, . . . , an−1). The matrix V −1 can

be characterized as follows.

Proposition 5.2.6 The inverse of the Vandermond matrix

V =

1 z0n (z0

n)2 . . . (z0n)n−1

1 z1n (z1

n)2 . . . (z1n)n−1

......

.... . .

...1 zn−1

n (zn−1n )2 . . . (zn−1

n )n−1

is the matrix

V −1 =1

n

1 z−1×0n z−2×0

n . . . z−(n−1)×0n

1 z−1×1n z−2×1

n . . . z−(n−1)×1n

......

.... . .

...

1 z−1×(n−1)n z

−2×(n−1)n . . . z

−(n−1)×(n−1)n

Proof: For each 0 ≤ j, j′ ≤ n− 1 the entry in row j + 1 and column j′ + 1 ofV −1 × V is

1

n

n−1∑

k=0

z−kjn zkj′

n

Since z−kjn zkj′

n = zk(j−j′)n , the above summation equals 1 when j′ = j and, by

Proposition 5.2.3, it equals 0 otherwise. Note that −(n− 1) < j′ − j < n− 1and therefore j′ − j is not divisible by n. Hence, V −1 × V is an identity matrix.QED

We can now write DFT−1n (y0, y1, . . . , yn−1) as the discrete Fourier transform

of a suitable tuple. Hence, the inverse of of the discrete Fourier transform can becomputed also using the discrete Fourier transform.

Proposition 5.2.7 Let Vn be the Vandermonde matrix V (z0n, z

1n, . . . , z

n−1n ). Then

V −1n

y0

y1...

yn−1

=1

nDFTn(y0, yn−1, yn−2, . . . , y1)


where in the above equality we assume DFTn(y0, yn−1, yn−2, . . . , y1) written acolumn matrix.

Proof: Considering

V −1n

y0

y1...

yn−1

=

a0

a1...

an−1

it holds that

ak =1

n

n−1∑

j=0

yjz−kjn

for each 0 ≤ k ≤ n− 1. By Proposition 5.2.1, z−kn = zn−k

n , and therefore

ak =1

n

n−1∑

j=0

yjz(n−k)jn .

QED

Hence,

DFT−1n (y0, y1, . . . , yn−1) =

1

nDFTn(y0, yn−1, yn−2, . . . , y1) (5.1)

that is, the inverse of the discrete Fourier transform can be computed using thediscrete Fourier transform itself.

Another relevant way of presenting the discrete Fourier transform is basedon polynomials. In fact, we may associate to a tuple (a0, a1, . . . , an−1) ∈ Cn thepolynomial p =

∑n−1j=0 ajx

j in C[x]. Then

DFTn(a0, a1, . . . , an−1) = (p(z0n), p(z1

n), . . . , p(zn−1n )). (5.2)

We illustrate this fact with a simple example.

Example 5.2.8 Consider again the tuple (3,−2, 1, 0). Recall that z04 = 1, z1

4 = i,z24 = −1 and z3

4 = −i. The polynomial p is in this case 0x3 + x2 − 2x + 3, thatis, x2 − 2x + 3. Since

• p(1) = 1− 2 + 3 = 2

• p(i) = i2 − 2i + 3 = 2− 2i

• p(−1) = 1 + 2 + 3 = 6


• p(−i) = (−i)2 + 2i + 3 = 2 + 2i

we conclude that DFT4(3,−2, 1, 0) = (2, 2− 2i, 6, 2 + 2i).

Taking into account the equality (5.2), computing DFT reduces to evaluatinga polynomial in the roots of the unity. The naive way to evaluate the poly-nomial p used in (5.2) at some value u consists of computing ai × ui for each1 ≤ i ≤ n− 1 and then add all these values to a0. Moreover, if to compute ui wealways perform i − 1 multiplications (that is, we do not take advantage of thepreviously computed value of ui−1 to get ui = ui−1 × u), this evaluation involves∑n−1

i=1 i = n(n−1)2

multiplications and n− 1 sums. Hence, we have a O(n2) numberof multiplications and sums. If we use this naive way to evaluate DFTn, takinginto account Equality (5.2), it would require O(n3) multiplications and sums.

This bound can be improved using the Horner’s rule (see Proposition 5.2.9),which uses only a O(n) number of multiplications and sums to evaluate p at u.

Proposition 5.2.9 Let p =∑n−1

i=0 aixi be a polynomial in C[x]. Consider the

sequence of polynomials qj in C[x] with j = 0, ..., n− 1 defined as follows:

• q0 = an−1

• qj = qj−1 × x + an−(j+1).

Then p = qn−1 and the evaluation of qn−1 using the above sequence of polynomialsinvolves n− 1 multiplications and n− 1 sums.

Proof: The proof follows by induction on the degree n− 1 of p.

Basis: the degree is 0. Then, n − 1 = 0 and no multiplications or sums areperformed.

Step: Assuming that the degree is n, then p(u) = qn(u) = qn−1(u) ∗ u + a0. Byinduction hypothesis, computing qn−1(u) takes n − 1 multiplications and n − 1sums. So computing p(u) takes n multiplication and n sums. QED

Example 5.2.10 Consider the polynomial x3 − 4x + 2 of degree 3. Let ai be thecoefficient of xi for each 0 ≤ i ≤ 3. Then, a3 = 1, a2 = 0, a1 = −4 and a0 = 2.Using Proposition 5.2.9,

• q0 = a3 = 1;

• q1 = q0 × x + a2 = x + 0 = x;

• q2 = q1 × x + a1 = x2 − 4;

• q3 = q2 × x + a0 = (x2 − 4)× x + 2 = x3 − 4x + 2.

5.3. FAST FOURIER TRANSFORM 167

Let us compute p(3):

• q0 = a3 = 1

• q1(3) = q0 × 3 + a2 = 3 + 0 = 3

• q2(3) = q1(3)× 3 + a1 = 33 − 4 = 5

• p(3) = q3(3) = q2(3)× 3 + a0 = 5× 3 + 2 = 17.

The evaluation of DFTn needs O(n2) multiplications and sums using Horner’srule and Equality (5.2). Note that we are not taking into account that the poly-nomial p to be evaluated is always the same throughout all components of DFTn

and that p is evaluated at the roots of the unity. These two facts combined allowto further improve the complexity of computing DFTn to O(n log(n)) multiplica-tions and sums. In the following section we present the algorithm that achievesthis complexity, known as the Fast Fourier Transform.

5.3 Fast Fourier transform

In this section we describe an efficient method to evaluate the discrete Fouriertransform DFTn(a0, a1, . . . , an−1) whenever n is a power of 2. This method relieson some properties of the n-th roots of unity and it is usually referred to as fastFourier transform (FFT).

Proposition 5.3.1 Let (a0, a1, . . . , an−1) ∈ Cn where n is an even number andlet p =

∑n−1j=0 ajx

j . Consider the polynomials

• p0 =

n2−1∑

j=0

a2jxj

• p1 =

n2−1∑

j=0

a2j+1xj

Then p(u) = p0(u2) + up1(u2) for all u ∈ Cn.

Proof: We equalities

p0(u2) =

n2−1∑

j=0

a2ju2j and p1(u2) =

n2−1∑

j=0

a2j+1u2j

hold. As a consequence


up1(u2) =

n2−1∑

j=0

a2j+1u2j+1

and therefore p(u) = p0(u2) + up1(u2). QED

Hence, the evaluation at u of a polynomial p =∑n−1

j=0 ajxj of degree n− 1,

where n is an even number, can be computed using the evaluation of two poly-nomials of degree less than or equal to n

2− 1 at u2.

When considering the n-th roots of unity, we have that

p(zkn) = p0(z2k

n ) + zknp1(z2k

n ) = p0(zkn2) + zk

np1(zkn2)

for each 0 ≤ k ≤ n2− 1. Moreover,

p(zn2+k

n ) = p(−zkn) = p0(z2k

n )− zknp

1(z2kn ) = p0(zk

n2)− zk

np1(zkn2)

for each 0 ≤ k ≤ n2− 1. Hence, the evaluations of p at the n-th roots of unity

can be computed from zkn for 0 ≤ k ≤ n

2− 1 and the evaluations of p0 and p1 at

the n2-th roots of unity, that is, DFTn(a0, a1, . . . , an−1) can be computed from

DFTn2(a0, a2, . . . , an−2) and DFTn

2(a1, a3, . . . , an−1).

Since n is a power of 2, we have that n2

is also an even number and we canreason in a similar way with respect to p0 and p1. Similarly, n

4is also a even

number and we can keep reasoning in this way until we only to have computeDFT1(aj) for 0 ≤ j ≤ an−1 (see Figure 5.2).

DFTn(a0, a1, . . . , an−1)

DFTn2(a0, a2 . . . , an−2) DFTn

2(a1, a3 . . . , an−1)

. . . . . . . . . . . .

DFT1(a0) . . . DFT1(an−2) DFT1(a1) . . . DFT1(an−1)

Figure 5.2: Discrete Fourier transform

Hence, the discrete Fourier transform can be computed recursively as follows:

DFTn(a0, a1, . . . , an−1) =

(b0 + z0nc0, . . . , b(n

2−1) + z

(n2−1)

n c(n2−1), b0 − z0

nc0, . . . , b(n2−1) − z

(n2−1)

n c(n2−1))

where

5.3. FAST FOURIER TRANSFORM 169

• DFTn2(a0, a2, . . . , an−2) = (b0, . . . , b(n

2−1));

• DFTn2(a1, a3, . . . , an−1) = (c0, . . . , c(n

2−1)).

The algorithm to compute the DFT using this recursion is called Fast FourierTransform (FFT). In Figure 5.3 a Mathematica implementation of the FFT isgiven, assuming that the lenght of w is a power of 2.

FFT=Function[w,Module[n,z,zp,az,au,ptz,ptu,k,r,n=Length[w];

If[n==1,

w,

zp=EI2πn ;

z=1;

az=Table[w[[2i+1]],i,0,n/2-1];au=Table[w[[2i+2]],i,0,n/2-1];ptz=FFT[az];

ptu=FFT[au];

r=Table[0,i,1,n];For[k=0,k<=n/2-1,k=k+1,

r[[k+1]]=ptz[[k+1]]+z*ptu[[k+1]];

r[[k+n/2+1]]=ptz[[k+1]]-z*ptu[[k+1]];

z=z*zp];

r]]];

Figure 5.3: FFT in Mathematica

The analysis of the FFT algorithm follows straightforwardly. Let oFFT(n)be the number of sums and multiplications used in FFT for an input of lengthn. For such input, the FFT performs O(n) multiplications and sums, and makestwo recursive calls of order n

2. So we have to find the solution for

oFFT(n) = 2oFFT(n/2) + O(n)

that is:

oFFT(n) = O(n) + 2O(n/2) + 4O(n/4) + . . . + 2log2(n)O(n/2log2(n))= log2(n)O(n)= O(n log2(n)).

In the following example we illustrate the computation a discrete Fouriertransform using the FFT algorithm.


Example 5.3.2 Consider the tuple (3,−2, 1, 0). The goal is to compute thediscrete Fourier transform DFT4(3,−2, 1, 0) using the FFT algorithm. Thistranform can be computed from DFT2(3, 1) and DFT2(−2, 0), and these fromDFT1(3), DFT1(1), DFT1(−2) and DFT1(0). The computation involves 2th-roots of unity and 4th-roots of unity. Only two of them, z0

2 and z14 , have to be

computed since

z02 = z0

4 = 1 z12 = −z0

2 = z24 = −1 z1

4 = i z34 = −z1

4 = −i.

The computation proceeds as follows:

DFT2(3, 1) = (4, 2) since

DFT2(3, 1)1 = DFT1(3) + z02DFT1(1) = 3 + 1 = 4;

DFT2(3, 1)2 = DFT1(3)− z02DFT1(1) = 3− 1 = 2.

DFT2(−2, 0) = (−2,−2) since

DFT2(−2, 0)1 = DFT1(−2) + z02DFT1(0) = −2 + 0 = −2;

DFT2(−2, 0)2 = DFT1(−2)− z02DFT1(0) = −2− 0 = −2.

and therefore

DFT4(3,−2, 1, 0) = (2, 2− 2i, 6, 2 + 2i) since

DFT4(3,−2, 1, 0)1 = DFT2(3, 1)1 + z04DFT2(−2, 0)1 = 2;

DFT4(3,−2, 1, 0)2 = DFT2(3, 1)2 + z14DFT2(−2, 0)2 = 2− 2i;

DFT4(3,−2, 1, 0)3 = DFT2(3, 1)1 − z04DFT2(−2, 0)1 = 6;

DFT4(3,−2, 1, 0)4 = DFT2(3, 1)2 − z14DFT2(−2, 0)2 = 2 + 2i.

and it can also be briefly sketched as follows

DFT4(3,−2, 1, 0) = (4 + z04 .(−2), 2 + z1

4 .(−2), 4− z04 .(−2), 2− z1

4 .(−2))

( = (2, 2− 2i, 6, 2 + 2i) )

DFT2(3, 1) = (3 + z02 .1, 3− z0

2 .1)

( = (4, 2) )

DFT2(−2, 0) = (−2 + z02 .0, −2− z0

2 .0)

( = (−2,−2) )

DFT1(3) = 3 DFT1(1) = 1 DFT1(−2) = −2 DFT1(0) = 0

5.4. POLYNOMIAL MULTIPLICATION REVISITED 171

5.4 Polynomial multiplication revisited

We now detail the polynomial multiplication technique briefly skteched in Section5.1, starting by introducing some remarks related to coefficient and point-wiserepresentation of polynomials.

5.4.1 Coefficient and point-value representations

Herein we refer to coefficient and point-value representation of polynomials.

Definition 5.4.1 Let p ∈ C[x] be a polynomial with degree m and let n ∈ N0

be such that m ≤ n. The coefficient representation of degree n of the polynomialp is the tuple

(a0, a1, . . . , an)

where ai is the coefficient of xi in p for all 0 ≤ i ≤ n.

Example 5.4.2 Consider the polynomial x3 − 4x + 2 of degree 3 in C[x]. Itscoefficient representation of degree 3 is the tuple (2,−4, 0, 1). Its coefficient rep-resentation of degree 4 is the tuple (2,−4, 0, 1, 0).

We now refer to the point-value representation of polynomials in C[x]. Given

(u0, v0), (u1, v1), . . . , (un, vn) ∈ C2

where n ∈ N0 and ui 6= uj for all 0 ≤ i, j ≤ n such that i 6= j, there is one andonly one polynomial p in C[x] with degree less than or equal to n such thatp(ui) = vi for all 0 ≤ i ≤ n. We get the coefficients ai of xi for each 0 ≤ i ≤ nby interpolation, that is, by solving a system of n + 1 linear equations on theunknowns a0, a1, . . . , an whose matrix form is

1 u0 u20 . . . un

0

1 u1 u21 . . . un

1...

......

. . ....

1 un u2n . . . un

n

a0

a1...

an

=

v0

v1...

vn

Recall that the coefficient matrix is the Vandermonde matrix Vn(u0, u1, . . . , un).This matriz is invertible, since its determinant, |Vn|, is

∏

0≤i<j≤n(uj − ui) andtherefore |Vn| is different from 0 whenever ui 6= uj. If an 6= 0 then p has degreen. Otherwise the degree is less than n.

Example 5.4.3 Consider the set (0, 2), (1,−1), (2, 2, (3, 17) of pairs of realnumbers. There is one (and only one) polynomial p in R[x] with degree less thanor equal to 3 such that

p(0) = 2 p(1) = −1 p(2) = 2 p(3) = 17

since solving the system


1 0 0 01 1 1 11 2 4 81 3 9 27

a0

a1

a2

a3

=

2−1217

we geta0 = 2 a1 = −4 a2 = 0 a3 = 1

and therefore x3 − 4x + 2 is the intended polynomial p. In this case the degreeof p is 3.

Hence, each polynomial p of degree n ∈ N0 in C[x] can be represented by a setof suitable n + 1 pairs of complex numbers, obtained by evaluating p at distinctn + 1 complex numbers.

Definition 5.4.4 Let p = anxn + . . . + a0 ∈ C[x] be a polynomial with degreen ∈ N0 and let u0, . . . , un ∈ C where ui 6= uj for all 0 ≤ i, j ≤ n such that i 6= j.The set

(u0, p(u0)), . . . , (un, p(un))is a point-value representation of p, more precisely, the point-value representationof p at u0, . . . , un.

Clearly, point-value representation is not unique, in the sense that any setof n + 1 pairs of complex numbers (v0, p(v0)), . . . , (vn, p(vn)) with distinct firstcomponents is also a point-value representation of p.

In certain situations it is useful to consider extended point-value represen-tations of a polynomial p with degree n: any set of m > n + 1 pairs of realvalues

(u0, p(u0)), (u1, p(u1)), . . . , (um, p(um))where ui 6= uj for all 0 ≤ i, j ≤ m such that i 6= j, is an extended point-valuerepresentation of p. Note that the solution of the system of m linear equations onthe n + 1 unknowns a0, a1, . . . , an we get from the above extended representationof p is equal to the solution of the system of n+1 linear equations on the unknownsa0, a1, . . . , an that results from the interpolation with the first n + 1 pairs

(u0, p(u0)), (u1, p(u1)), . . . , (un, p(un))

of the extended representation. For simplicity, in the sequel we often refer onlyto point-value representations even in the case of extended point-value represen-tations.

Example 5.4.5 Let p = x3 − 4x + 2 be a polynomial in C[x]. Since p(0) = 2,p(1) = −1, p(2) = 2 and p(3) = 17, the set (0, 2), (1,−1), (2, 2), (3, 17) is apoint-value representation of p. Given that p(−2) = 2 and p(−1) = 5, the set(−2, 2), (−1, 5), (0, 2), (1,−1) is another possible point-value representation ofp. The set (−2, 2), (−1, 5), (0, 2), (1,−1), (2, 2), (3, 17) is an extended point-value representation of the polynomial p.


Note that using the discrete Fourier transform we get a point-value representa-tion of a polynomial p =

∑n−1i=0 aix

i of degree n−1: the point-value representationat the n-th roots of unity. In fact, from

DFTn(a0, a1, . . . , an−1) = (p(zn0), p(zn

1), . . . , p(znn−1))

we get the set

(zn0, p(zn

0)), (zn1, p(zn

1)), . . . , (znn−1, p(zn

n−1)).

Conversely, given a point-value representation of a polynomial p at the n-th rootsof unity

(zn0, y0)), (zn

1, y1)), . . . , (znn−1, yn−1)

the inverse of the discrete Fourier transform is used to perform interpolation.Since

DFTn(a0, a1, . . . , an−1) = (y0, y1, . . . , yn−1)

getting the coefficients a0, a1, . . . , an−1 from y0, y1, . . . , yn−1 corresponds to com-puting the inverse of the discrete Fourier transform, that is,

DFT−1n (y0, y1, . . . , yn−1) = (a0, a1, . . . , an−1).

We now refer to sum and multiplication of polynomials using only their point-value representations.

Given two polynomials p and q with degree n, from point-value representationsof p and q at the same complex numbers u0, . . . , un we easily get a point-valuerepresentation of p+q at u0, . . . , un. Recall that deg(p+q) ≤ maxdeg(p), deg(q).

Proposition 5.4.6 Consider the polynomials p and q in C[x] with degree n.If (u0, v0), . . . , (un, vn)) and (u0, w0), . . . , (un, wn) are point-value representa-tions of p and q, respectively, then the pointwise sum

(u0, v0 + w0), . . . , (un, vn + wn)is a (possibly extended) point-value representations of p + q.

If the polynomials p and q do not have the same degree, let us say, for instance,deg(q) < deg(p), then deg(p+q) = deg(p) and it is easy to conclude that taking asuitable extended point-value representation of q we can also obtain a point-valuerepresentation of p + q as described in Proposition 5.4.6.

Example 5.4.7 Consider the polynomials p = x3 − 4x + 2 and q = x2 − 2x + 1.Since (0, 2), (1,−1), (2, 2), (3, 17) and (0, 1), (1, 0), (2, 1), (3, 4) are a point-value representation of p and an extended point-value representation of q, re-spectively, then (0, 3), (1,−1), (2, 3), (3, 21) is a point-value representation ofp + q.


Extended point-value representations are also useful when multiplying poly-nomials in point-value representation. Recall that deg(p× q) = deg(p) + deg(q).Hence, if p and q have degree n, any point value-representation of p × q has2n + 1 elements. As a consequence, to obtain a point value representation forp× q we always have to consider extended point-values representations of p andq whenever deg(p) and deg(q) are not both 0.

Proposition 5.4.8 Let p and q be polynomials in C[x] with degree n > 0. If(u0, v0), . . . , (u2n, v2n)) and (u0, w0), . . . , (u2n, w2n) are extended point-valuerepresentations of p and q, respectively, then the pointwise multiplication

(u0, v0 × w0), . . . , (u2n, v2n × w2n)

is a point-value representations of p× q.

If deg(p) = deg(q) = 0 then deg(p× q) is also 0 and if (u0, v0) and (u0, w0)are point-value representations of p and q, respectively, then (u0, v0 × w0) isa point-value representations of p × q. If p and q do not have the same degreewe just have to consider extended point-value representations of p and q withdeg(p) + deg(q) + 1 elements.

Example 5.4.9 Consider p = x3 − 4x + 2 and q = x2 − 2x + 1. Given that(−2, 2), (−1, 5), (0, 2), (1,−1), (2, 2), (3, 17) is an extended point-value repre-sentation of p and (−2, 9), (−1, 4), (0, 1), (1, 0), (2, 1), (3, 4) is an extended point-value representation of q, then (−2, 18), (−1, 20), (0, 2), (1, 0), (2, 2), (3, 68) is apoint-value representation of p× q.

Note that the pointwise multiplication in Proposition 5.4.8 only involves aO(n) number of multiplications of real numbers.

5.4.2 Polynomial multiplication and FFT

We now revisit the polynomial multiplication technique briefly sketched in Section5.1. The main steps are depicted in Figure 5.4. We use two discrete Fouriertransforms to obtain point-value representations of p and q (in fact, only thesecond component of each pair is relevant). Then, using pointwise multiplication,we can compute a point-value representations of p × q. Finally, using a discreteFourier transform to interpolate we get the coefficients of p× q. The fast Fouriertransform is used to compute the discrete Fourier transforms involved.

Given that deg(p× q) = deg(p) + deg(q), we need a point-value representa-tion of the polynomial p × q with deg(p) + deg(q) + 1 elements. Since we wantto use the fast Fourier transform to efficiently compute the coefficients of p× q, ifdeg(p) + deg(q) + 1 is not a power of 2, we have to consider an extended point-value representation of p× q with n elements, where n = 2k and k is such that


p, qcoefficient rep.

p× qcoefficient rep.

DFTn for pDFTn for q

p, qpoint-value rep.

pointwise multiplicationp× qpoint-value rep.

DFT−1n

Figure 5.4: Polynomial multiplication and DFT

2k−1 < deg(p) + deg(q) + 1 ≤ 2k. Therefore, we need (possibly extended) point-values representations of p and q with n elements and, as a consequence, we haveto consider n coefficients of p and q, that is, we have to consider coefficient rep-resentations of degree n − 1 of p and q, in order to compute the correspondingdiscrete Fourier transforms.

In Section 5.3 we concluded that DFTn(~a) can be computed using a O(n log2 n)number of sums and multiplications where n is a power of 2. The computation ofDFT−1

n (~a) can also be performed using a O(n log2 n) number of sums and multi-plications since, in Section 5.2.2, we proved that to get DFT−1

n (~a) we only have

to compute DFT−1n (~b), where ~b results from ~a just by changing the order of the

components. It is also trivial to conclude that the pointwise multiplication ofthe point-value representations of p and q mentioned above just involves a O(n)number of multiplications. Therefore, as the following proposition ensures, poly-nomial multiplication computed as described above involves a O(n log n) numberof sums and multiplications.

Proposition 5.4.10 Let f1, f2, f3 : N0 → N0 such that f1(n), f2(n) ∈ O(n log2 n)and f3(n) ∈ O(n). Then f1(n)+f2(n) ∈ O(n log2 n) and f1(n)+f3(n) ∈ O(n log2 n).

Proof: For i = 1, 2, let pi ∈ N0 be such that | fi(n) |≤ ci | n log2 n | for all n ≥ pi

for some ci ∈ R+. Hence, letting p = max(p1, p2)

f1(n) + f2(n) ≤ c1n log2 n + c2n log2 n

for all n ∈ N0 such that n > p. Moreover,

f1(n) + f2(n) ≤ 2cn log2 n

considering c = max(c1, c2). Since all the values involved are nonnegative, itholds that | f1(n) + f2(n) |≤ 2c | n log2 n | for all n ∈ N0 such that n > p, andtherefore f1(n) + f2(n) ∈ O(n log2 n).


Let p1 ∈ N0 be as above and let p3 ∈ N0 be such that | f3(n) |≤ c3 | n | forall n ≥ p3 for some c3 ∈ R+. If n > 2 then log2 n > 1 and therefore n < n log2 n.Hence, letting p = max(p1, p3, 2) and c = max(c1, c3)

f1(n) + f3(n) ≤ 2cn log2 n

for all n ∈ N0 such that n > p. Since all the values involved are nonnegative,it holds that | f1(n) + f3(n) |≤ 2c | n log2 n | for all n ∈ N0 such that n > p andtherefore f1(n) + f3(n) ∈ O(n log2 n). QED

Let us now relate the number of sums and multiplications involved in this com-putation of p×q with the degrees of p and q. Assuming that deg(p) = deg(q) = m,and recalling that n is the power of 2 used for computing the discrete Fourer trans-forms, it holds that n

2< 2m + 1 ≤ n, It is then easy to conclude this computation

of p×q indeed involves a O(m log2 m) number of sums and multiplications, wherem is the degree of p and q.

We now present an example that illustrates polynomial multiplication usingthe discrete Fourier transform and the fast Fourier transform.

Example 5.4.11 Let us consider the polynomials p = x2−2x+3 and q = 2x−1.The goal is to compute p × q using the discrete Fourier transform and the fastFourier transform.

(1) Since deg(p× q) = 3 then we need a point-value representation for p× q with4 (= 22) elements and therefore, also point-value representations for p and q with4 elements. As a consequence, we begin with coefficient representations of degree3 of p and q, that is, respectively

(3,−2, 1, 0) and (−1, 2, 0, 0)

(2) We first compute DFT4(3,−2, 1, 0). Recalling Example 5.3.2, it holds thatDFT4(3,−2, 1, 0) = (2, 2− 2i, 6, 2 + 2i).

We now compute DFT4(−1, 2, 0, 0):

DFT2(−1, 0) = (−1,−1) since

DFT2(−1, 0)1 = DFT1(−1) + z02DFT1(0) = −1 + 1× 0 = −1

DFT2(−1, 0)2 = DFT1(−1)− z02DFT1(0) = −1− 1× 0 = −1

DFT2(2, 0) = (2, 2) since

DFT2(2, 0)1 = DFT1(2) + z02DFT1(0) = 2 + 1× 0 = 2

DFT2(2, 0)2 = DFT1(2)− z02DFT1(0) = 2− 1× 0 = 2


and therefore

DFT4(−1, 2, 0, 0) = (1,−1 + 2i,−3,−1− 2i) since

DFT4(−1, 2, 0, 0)1 = DFT2(−1, 0)1 + z04DFT2(2, 0)1 = −1 + 1× 2 = 1

DFT4(−1, 2, 0, 0)2 = DFT2(−1, 0)2 + z14DFT2(2, 0)2 = −1 + 2i

DFT4(−1, 2, 0, 0)3 = DFT2(−1, 0)1 − z04DFT2(2, 0)1 = −1− 1× 2 = 1

DFT4(−1, 2, 0, 0)4 = DFT2(−1, 0)2 − z14DFT2(2, 0)2 = −1− 2i

(3) Using DFT4(3,−2, 1, 0), DFT4(−1, 2, 0, 0) and pointwise multiplication (de-noted by ⊗) we get a point-value representation for p×q (in fact, only the secondcomponents):

(2, 2− 2i, 6, 2 + 2i)⊗ (1,−1 + 2i,−3,−1− 2i) = (2, 2 + 6i,−18, 2− 6i)

(4) Finally, we compute

DFT−14 (2, 2 + 6i,−18, 2− 6i) =

1

4DFT4(2, 2− 6i,−18, 2 + 6i)

DFT2(2,−18) = (−16, 20) since

DFT2(2,−18)1 = DFT1(2) + z02DFT1(−18) = −16

DFT2(2,−18)2 = DFT1(2)− z02DFT1(−18) = 20

DFT2(2 + 6i, 2− 6i) = (4,−12i) since

DFT2(2 + 6i, 2− 6i)1 = DFT1(2 + 6i) + z02DFT1(2− 6i) = 4

DFT2(2 + 6i, 2− 6i)2 = DFT1(2 + 6i)− z02DFT1(2 + 6i) = −12i

DFT4(2, 2− 6i,−18, 2 + 6i) = (−12, 32,−20, 8) since

DFT4(2, 2− 6i,−18, 2 + 6i)1 = −16 + z044 = −12

DFT4(2, 2− 6i,−18, 2 + 6i)2 = 20 + z14(−12i) = 32

DFT4(2, 2− 6i,−18, 2 + 6i)3 = −16− z044 = −3

DFT4(2, 2− 6i,−18, 2 + 6i)4 = 20− z14(−12i) = 8

Hence,

DFT−14 (2, 2 + 6i,−18, 2− 6i) =

1

4(−12, 32,−20, 8) = (−3, 8,−5, 2).

The product of p and q is therefore the polynomial 2x3 − 5x2 + 8x− 3.


5.5 Image processing

5.6 Exercises

1. Prove that znk = zn

n+k where n ∈ N and k ∈ Z.

2. Let k ∈ N0 and n, d ∈ N where n is an even number. Prove that

(a) zn

n2 = −1

(b) zn

n2+d = −zn

d

(c) zn2k = zk

n2

3. Let k ∈ N0 and n ∈ N such that k is not divisible by n. Prove that theequality

∑n−1i=0 (zk

n)i = 0 holds.

4. Using the Fast Fourier transform compute

(a) DFT2(3, 4)

(b) DFT4(0, 1, 1, 2)

(c) DFT4(2, 1, 1, 0)

(d) DFT4(4,−7, 3, 0)

(e) DFT8(2, 0, 1,−1, 0, 1,−2, 1)

(f) DFT−12 (−2, 1)

(g) DFT−14 (1, 0, 2,−1)

(h) DFT−14 (4,−1− i,−2,−1 + i)

(i) DFT−18 (−1, 1, 0,−1, 2, 1,−1,−2)

5. Compute p× q in a efficient way, using the discrete Fourier transform andits inverse, where

(a) p = 3x2 − 7x + 4 e q = −2

(b) p = 5x + 3 e q = −2x + 4

(c) p = 3x− 2 e q = −4x2 − 5x + 3

(d) p = 5x e q = −2x2 − 3x

(e) p = 5x2 e q = −2x4 − 3x− 1

Chapter 6

Generating functions

In this chapter we introduce generating functions. Given any sequence of real orcomplex numbers, we can associate with it a generating function, that is, a seriesinvolving all the terms of the sequence. There are several kinds of generatingfunctions for a sequence, such as ordinary generating functions, exponential gen-erating functions and Poisson generating functions [19, 18, 31]. Herein, we onlyconsider ordinary generating functions.

In Section 6.1 we present motivating examples in algorithm analysis. Gen-erating functions are introduced in section 6.2. In section 6.3 we revisit themotivating examples and in Section 6.4 we propose some exercises.

6.1 Motivation

In this section, we refer to the use of generating functions in algorithm analy-sis. We first refer to the average case analysis of a search algorithm involving ahash function. Then we refer to the Euclid’s algorithm The second example alsoillustrates the relevance of generating functions for solving recurrence relations.

6.1.1 Search by hashing

Assume we want store information in a computer using of a set of records, whereeach record has a key K and some data D(K). Given a key K, it should beeasy to get the corresponding data D(K). To this end we can then consider atable Key for storing keys and a table Data for storing the corresponding data.Assuming that nr is the maximum number of records that can be stored, foreach 1 ≤ j ≤ nr, if Key[j] is some key K then Data[j] is the corresponding dataD(K). The tables are filled sequentially, in the sense that Key[1] is the keyof first record that has been stored (and therefore Data[1] is the correspondingdata), Key[2] is the key of second record that has been stored, etc. The value ofa variable rstored indicates the number of records that have already been stored.

179

180 CHAPTER 6. GENERATING FUNCTIONS

To insert a new record, we assign its key value to Key[rstored + 1] and its datato Data[rstored + 1] and then increment rstored. The task of searching for somekey K among the keys already stored can be accomplished going through thetable Key sequentially, comparing each Key[j] to the given key K. This can berather slow when a large number of records have already been stored.

In order to improve this situation we can use the hashing technique thatinvolves splitting the storing memory space for keys into m lists, and consider ahash function h that transforms each key K into an integer h(K) ∈ 1, ..., m.The integer h(K) indicates the list where to search for the key K. We have toconsider also the tables First and Next. For each 1 ≤ i ≤ m,

First[i] ∈ −1, 1, ..., nr

indicates the position in table Key of the first key of list i. The value −1 indicatesthat the list is empty. For each 1 ≤ j ≤ nr,

Next[j] ∈ 0, 1, ..., nr

indicates the position in table Key of the key that follows Key[j] in the listh(Key[j]). The value 0 indicates that Key[j] is the last key in its list.

Example 6.1.1 Consider an university library maintaining a database storingrelevant data about its readers, the university students. In order to borrowbooks, each student has to register first as a reader at the library. The universitysequentially assigns an identification number to each of its students, lets say, anonnegative integer number less than 100000. The database library uses theseidentification numbers as record keys.

To keep things easy, assume in this example that we have only 10 lists, that ism = 10, and that the hash function h : 1, . . . , 99999 → 1, . . . , 10 is such that

h(K) =

⌊K

10000

⌋

+ 1

Moreover, for simplicity, let us assume that only 6 students have registered atthe library so far and that Key[1] = 35346, Key[2] = 15367, Key[3] = 43289,Key[4] = 32128, Key[5] = 38532 and Key[6] = 46238. We can sketch the keydistribution as follows

L1 L2 L3 L4 L5 L6 L7 L8 L9 L10

15367 35346 4328932128 4623838532

that is, there are keys in the lists L2, L4 and L5 and the other lists are empty.This situation corresponds to

6.1. MOTIVATION 181

• rstored=6

• First[i] = −1 for i ∈ 1, 3, 6, 7, 8, 9, 10and

First[2] = 2, First[4] = 1, First[5] = 3

• Next[i] = 0 for i ∈ 2, 5, 6and

Next[1] = 4, Next[3] = 6, Next[4] = 5

Using the hashing technique described above, the task of searching for keyscan be performed faster, since when looking for a given key K we only have tocompare it with the stored keys K ′ such that h(K ′) = h(K). Clearly, if the hashfunction h is such that h(K) = h(K ′) for all the keys K ′ that have already beenstored, then all the lists are empty but one and K has to be compared with allthe stored keys. Therefore, the worst case number of comparisons is equal to theworst case number of comparisons when no hash function is involved. However,the average case number of comparisons is smaller when the hashing techniqueis used. Our goal is to determine this average case number of comparisons.

The function keySearch in Figure 6.1 determines whether a given key k hasalready been stored, assuming that the hash function h is already known and thatthe lists key, first and next record the tables Key, First and Next, respec-tively. Using the auxiliary lists first and next, function keySearch compares kwith all the elements in key whose hash function value equals that of k. It returnsthe position of the key k in key if k has already been stored and the string “thekey has not been stored” otherwise.

keySearch=Function[k,Module[i,j,r,i=h[k];

j=first[[i]];

r=False;

While[j>0&&!r,

If[key[[j]]==k,r=True,j=next[[j]]]];

If[r,j,Print["the key has not been stored"]]]];

Figure 6.1: Key search function in Mathematica

If the key we are searching for has been already stored we say that the searchis successful. Otherwise, the search is unsuccessful.


We want to determine the average number of comparisons key[[j]]==k thatare executed when searching for some given key k. To this end we have to considera discrete random variable NC whose values correspond to the possible numbersof such comparisons. We also have to assume some probabilistic hypothesis,such as, for instance, what is the probability that h(k) = i for each 1 ≤ i ≤ m.The characterization of the random variable NC depends on the kind of search:successful or unsuccessful. But, in both cases, the average number of comparisonsis the expected value, or mean, of NC.

Generating functions are useful for computing the mean and the varianceof discrete random variables whose values are nonnegative. In Section 6.3.1 wediscuss the relevance of generating functions in the average case analysis of theabove key search algorithm.


Recall the Euclid’s algorithm for computing the greatest common divisor of twononnegative integers presented in Figure 1.4. The analysis of this algorithminvolves counting the number of recursive calls performed when euclid[m,n] isevaluated.

When evaluating euclid[m,n], there are no recursive calls if m = 0 and ifm = n there is just one recursive call. The analysis of the Euclid’s algorithmoften assumes that the first argument is less than the second. In fact, note thatif m 6= 0 and m < n, there is one recursive call to euclid[Mod[n,m],m] where,again, the first argument is less than the second, since mod(n, m) < m. Reasoningin a similar way, it is easy to conclude that the first argument is going to be alsoless than the second in all the following recursive calls. If m > n, there is againa first recursive call to euclid[Mod[n,m],m], but the first argument is now lessthan the second. Hence, we can reason as in the previous case and conclude thatagain the first argument is going to be less than the second in all the followingrecursive calls.

The Lame theorem (see Theorem 6.3.16) establishes an upper bound for thenumber of recursive calls that are performed when euclid[m,n] is evaluated. Itstates that for k, m, n ∈ N, with m < n, the evaluation of euclid[m,n] involvesless than k recursive calls whenever m < sk+1, where sk+1 is the (k + 2)th Fi-bonacci number. Recall that the sequence of Fibonacci numbers is the sequences = snn∈N0 such that

s0 = 0 s1 = 1 and sn = sn−1 + sn−2 for n ≥ 2. (6.1)

Moreover (see Proposition 6.3.17), the worst case number of recursive calls occurswhen evaluating euclid[sk,sk+1]. In this case there are k − 1 recursive calls.

6.2. GENERATING FUNCTIONS 183

The Fibonacci number sk, for some k ∈ N0, can be computed using (6.1), butthe closed form

sk =1√5

(

1 +√

5

2

)k

−(

1−√

5

2

)k

(6.2)

is often useful. To get the equality (6.2) we have to solve the recurrence relation(6.1). In Section 6.3.2 we discuss how generating functions can be used for solving,in particular, the recurrence relation in (6.1). The analysis of recursive algorithmsoften involves recurrence relations.

6.2 Generating functions

In this section we introduce the notion of generating function and some relatedproperties. A generating function associates a formal power series with eachsequence of elements of a field. Herein, we refer to sequences of real or complexnumbers.

Definition 6.2.1 Let s = snn∈N0 be a sequence of real or complex numbers.The generating function for s is

+∞∑

i=0

sizi. (6.3)

A generating function for a sequence s records all the elements of s. In thesequel, we often use Gs(z) or Gs to denote the generating function for a sequences. The subscript may be omitted when no confusion arises.

For simplicity, it is usual to introduce the following notations to represent thegenerating function for s = snn∈N0.

If si = 1 or si = −1 for all i ∈ N0, in (6.3) we may just use zi or −zi,respectively.

If for some k ∈ N it holds that si = 0 for each 0 ≤ i ≤ k − 1, it is oftenconvenient to write

+∞∑

i=k

sizi or

+∞∑

i=0

si+kzi+k (6.4)

to denote Gs(z).If s is such that sn = 0 for all n > k, for some k ∈ N0, we can write

s0 + s1z1 + s2z

2 + . . . + skzk (6.5)


together with the usual conventions regarding polynomials: using z instead of z1,omitting s0 when s0 = 0 and siz

i when si = 0 for some 1 ≤ i ≤ k, etc. Note thatwe can associate a polynomial in R[z] (or C[z]) to each generating function Gs(z)for a sequence s such that sn = 0 for all n > k, for some k ∈ N0 (and vice-versa).As expected, the coefficient of zn in the polynomial associated to Gs(z) is sn, foreach n ∈ N0.

If, for some k ∈ N, s is such that sn = 1 when n is a multiple of k and sn = 0otherwise, then we can use

+∞∑

i=0

zki (6.6)

to denote Gs(z).

Example 6.2.2 The following are examples of generating functions.

(i) The generation function for s = (sn)n∈N0 where sn = 2n+1 for each n ∈ N0

is+∞∑

i=0

(2i + 1)zi.

(ii) The generation function for v = (vn)n∈N0 where vn = 1 for each n ∈ N0 is

+∞∑

i=0

zi.

(iii) The generation function for r = (rn)n∈N0 where rn = n for each n ∈ N0 is

+∞∑

i=0

izi

and, taking into account the observations above, we can also write

+∞∑

i=1

izi or+∞∑

i=0

(i + 1)zi+1

for this generation function.

(iv) We can write 1 − z2 for the generating function for q = (qn)n∈N0 whereq0 = 1, q1 = 0, q2 = −1 and qn = 0 for each n > 2.


(v) We can write+∞∑

i=0

z3i

to denote the generation function for w = (wn)n∈N0 where wn = 1 if n is amultiple of 3 and wn = 0 otherwise, for n ∈ N0.

Let s = snn∈N0 be a sequence. Note that the expression∑+∞

i=0 sizi can be

seen as defining a map but it can also be seen as a formal power series.In the first case, we have the map f that associates each real or complex

number c to the real or complex number f(c), where f(c) is the sum of the series∑+∞

i=0 sici. Convergence issues are clearly relevant in this case since the domain

of f is the interval or disc of convergence of∑+∞

i=0 sizi.

When we consider∑+∞

i=0 sizi as a formal power series, convergence issues are

not relevant and∑+∞

i=0 sizi is only seen as a way of recording all the elements

of the sequence s. This is, in general, the point of view we are interested inalong this chapter. Generating functions represent sequences as formal powerseries. They are algebraic objects that we can manipulate using some suitable(ring) operations, without taking into account intervals (discs) of convergence.However, after some manipulations we may end up with a power series that indeedconverges in some interval (disc) I and therefore it defines a map with domainI. Then we can take advantage of this fact and use it as a map. But even whenthis is not the case, the result is useful for several purposes as we will discuss inthe sequel.

Sum and product of generating functions

We now introduce several operations over generating functions for sequences ofreal numbers and some related properties. Similar results also hold for sequencesof complex numbers. In the sequel, when we refer to a sequence we always assumethat it is a sequence of real numbers.

To begin with it is relevant to recall several notions regarding sequences ofreal numbers. Consider two sequences s = snn∈N0 and t = tnn∈N0 . The sumof s and t is the sequence s + t = (s + t)nn∈N0 where (s + t)n = sn + tn foreach n ∈ N0 . Moreover, as = (as)nn∈N0 is the sequence such that (as)n = asn

for each n ∈ N0, where a is any real or complex number (we just write −s whena = −1). The convolution of s and t is the sequence s ∗ t = (s ∗ t)nn∈N0 where(s ∗ t)n =

∑n

k=0 sktn−k for all n ∈ N0.

We now introduce the sum of generating functions.


Definition 6.2.3 Consider two sequences s = snn∈N0 and t = tnn∈N0. Thesum of Gs(z) and Gt(z), denoted by Gs(z) + Gt(z), is Gs+t(z) that is, the gener-ating function for the sequence s + t.

Example 6.2.4 Let r and v be the sequences presented in Example 6.2.2. Then,

Gr(z) + Gv(z) = Gr+v(z) =

+∞∑

i=0

(ri + vi)zi =

+∞∑

i=0

(i + 1)zi.

The product of generating functions is defined as follows.

Definition 6.2.5 Consider two sequences s = snn∈N0 and t = tnn∈N0. Theproduct of Gs(z) and Gt(z), denoted by Gs(z) × Gt(z), is Gs∗t(z), that is, thegenerating function for the sequence s ∗ t.

Hence,

Gs(z)×Gt(z) =+∞∑

i=0

(i∑

k=0

skti−k

)

zi.

For simplicity we often just write Gs(z)Gt(z) for Gs(z)×Gt(z).

Example 6.2.6 Let t=tnn∈N0 be any sequence and let s be the sequence

a, 0, 0, 0, . . .

that is, s=(sn)n∈N0 is such that s0 =a and sn =0 for all n>0, where a∈R. Since

i∑

k=0

skti−k = s0ti = ati

for each i ∈ N0, we get

Gs(z)×Gt(z) =+∞∑

i=0

atizi

and therefore Gs(z)×Gt(z) is the generation function for the sequence at. Notethat we can also write

a×Gt(z) =

+∞∑

i=0

atizi or a×Gt(z) = Gat(z)

since in this case we can write just a for Gs(z).


Example 6.2.7 Let t=tnn∈N0 be any sequence and let u be the sequence

0, 0, 0, 1, 0, 0, 0, 0, 0, . . .

that is, u = unn∈N0 is such that u3 = 1 and un = 0 for all n ∈ N0\3. Notethat

i∑

k=0

ukti−k =

i∑

k=0

0ti−k = 0

for i < 3, and thati∑

k=0

ukti−k = u3ti−3 = ti−3

for i ≥ 3. Hence, Gu(z)×Gt(z) is the generating function for the sequence

0, 0, 0, t0, t1, t2, t3, t4 . . .

that is, the sequence t′ = t′nn∈N0 such that t′0 = t′1 = t′2 = 0 and t′n = tn−3 forn ≥ 3, and therefore

Gu(z)×Gt(z) =

+∞∑

i=0

t′izi = Gt′(z). (6.7)

Taking into account the notation introduced above and the fact that ti = t′i+3 foreach i ≥ 0, we can also write

z3 ×Gt(z) =

+∞∑

i=3

t′izi or z3 ×Gt(z) =

+∞∑

i=0

tizi+3 (6.8)

Equalities similar to (6.7) hold for sequences u such that um = 1 is the onlynonzero term of the sequence, where m is any nonnegative integer. The productGu(z)×Gt(z) can then be also denoted as in (6.8) with the integer 3 substitutedfor m.

Observe that the sum and product of generating functions corresponding topolynomials indeed correspond to the sum and product of polynomials. Notealso that the sum and product of generating functions coincide with the sum andproduct of real functions admitting a power series expansion within its intervalof convergence.

Let G denote the set of generating functions for sequences of real numbers. Itis easy to conclude that the operation + : G2 → G that associates to each pairof generating functions their sum is a commutative and associative operation.Moreover, Gs(z) + 0 = Gs(z) and Gs(z) + G−s(z) = 0, for all Gs(z) ∈ G (recall


that 0 denotes the generating function for s = snn∈N0 such that sn = 0 for eachn ∈ N0).

Since∑i

k=0 skti−k =∑i

k=0 si−ktk for all i ∈ N0, it is also easy to conclude thatthe operation × : G2 → G that associates to each pair of generating functionstheir product is a commutative and associative operation. It also holds thatGs(z)×1 = Gs(z) for all Gs(z) ∈ G (recall that 1 denotes the generating functionfor s = snn∈N0 such that s0 = 1 and sn = 0 for each n > 0).

Moreover, the product of generating functions is distributive with respect totheir sum. Hence, the set G endowed with the operations defined above and− : G → G such that −(Gs) = G−s constitutes a unitary commutative ring. Themultiplicative identity is the generating function 1.

Proposition 6.2.8 The tuple (G, +, 0,−,×) constitutes a unitary commutativering.

Not all the generating functions have multiplicative inverses. If s = snn∈N0

is such that s0 6= 0 then Gs(z) has multiplicative inverse (and vice-versa).

Proposition 6.2.9 The generating function Gs(z) for the sequence s = snn∈N0

has a multiplicative inverse if and only if s0 6= 0.

Proof:(→) Assume that Gt(z) =

∑+∞i=0 tiz

i is the multiplicative inverse of Gs(z). Then,

Gs(z) × Gt(z) = 1. Therefore,∑0

k=0 skt0−k = s0t0 = 1 and∑i

k=0 skti−k = 0 fori > 0. In particular, s0 × t0 = 1 and, as a consequence, s0 6= 0.(←) Assume that s0 6= 0. Let t = tnn∈N0 be such that

t0 =1

s0

and

tn = − 1

s0

n∑

k=1

sktn−k (6.9)

for n > 0. Then,0∑

k=0

skt0−k = s0t0 = s01

s0= 1

and, taking into account (6.9),

n∑

k=0

sktn−k = s0tn +n∑

k=1

sktn−k = s0

(

− 1

s0

n∑

k=1

sktn−k

)

+n∑

k=1

sktn−k = 0

for n > 0. Hence, Gs(z)×Gt(z) is the generating function 1. QED

Besides Gs(z)−1 we may also use 1Gs(z)

to denote the multiplicative inverse of

a generating function Gs(z), when it exits.


Example 6.2.10 Let v be the sequence 1, 1, 1, . . . presented in Example 6.2.2.Since v0 6= 0 the multiplicative inverse of Gv(z) exists. The proof of Proposition6.2.9 describes a method for obtaining Gv(z)−1. Assuming that Gv(z)−1 is thegenerating function for a sequence t it holds that:

• t0 =1

v0

=1

1= 1

• t1 = − 1

v0

1∑

k=1

vkt1−k = − 1

v0v1t0 = −1

• t2 = − 1

v0

2∑

k=1

vkt2−k = − 1

v0

(v1t1 + v2t0) = − (−1 + 1) = 0

• t3 = − 1

v0

3∑

k=1

vkt3−k = − 1

v0(v1t2 + v2t1 + v3t0) = − (0 +−1 + 1) = 0

• . . .

It can be easily proved by induction that tn = 0 for all n ≥ 2. Hence, we concludethat Gv(z)−1 = 1− z.

Example 6.2.11 Let a be a real number and let ga be the sequence 1, a, a2, a3,. . ., that is, ga = (ga)nn∈N0 with (ga)n = an for all n ∈ N0 (geometric progressionwith ratio a and first term 1). Since (ga)0 6= 0 the multiplicative inverse of Gga

(z)exists. Using again the proof of Proposition 6.2.9, and assuming that Gga

(z)−1

is the generating function for a sequence t it holds that:

• t0 =1

(ga)0=

1

1= 1

• t1 = − 1

(ga)0

1∑

k=1

(ga)kt1−k = − 1

(ga)0(ga)1t0 = −a

• t2 = − 1

(ga)0

2∑

k=1

(ga)kt2−k = − 1

(ga)0((ga)1t1 + (ga)2t0) = −

(−a2 + a2

)= 0

• . . .

It can be easily proved by induction that tn = 0 for all n ≥ 2. Hence, we concludethat Gga

(z)−1 = 1− az.Note that the sequence s considered in Example 6.2.10 is just g1 and, as ex-

pected, the inverse of Gs(z) computed therein complies with the general expres-sion obtained above for the inverse of generation functions Gga

(z) for geometricprogressions with ratio a and first term 1.


Derivative and integral of generating functions

We can also define the derivative and the integral of a generating function.

Definition 6.2.12 Consider the sequence s = snn∈N0 . The derivative of Gs(z),denoted by G′

s(z), is the generating function for the sequence t = tnn∈N0 wheretn = (n + 1)sn+1 for all n ∈ N0.

Hence, the derivative of Gs(z) is

G′s(z) =

+∞∑

i=0

(i + 1)si+1zi.

Example 6.2.13 Let v be the sequence 1, 1, 1, . . . in Example 6.2.2. Thederivative of Gv(z) is the generating function for 1v1, 2v2, 3v3, . . .. Hence, G′

v(z)is the generating function for 1, 2, 3, 4, . . ., the sequence x = (xn)n∈N0 wherexn = n + 1 for each n ∈ N0, and therefore

G′v(z) =

+∞∑

i=0

(i + 1)zi.

Example 6.2.14 Let t = (tn)n∈N0 be such that tn = 0 for all n > k for somek ∈ N0, and recall that we can use t0 + t1z + t2z

2 + . . . + tkzk to denote Gt(z).

The derivative G′t(z) is the generating function for de 1t1, 2t2, . . ., ktk, 0, 0,. . .,

the sequence x = (xn)n∈N0 where xn = (n + 1)tn+1 for 0 ≤ n ≤ k − 1 and xn = 0for n ≥ k, and therefore

G′t(z) = t1 + 2t2z + 3t3z

2 + . . . + ktkzk−1.

Note that the derivative of t0 + t1z + t2z2 + . . . + tkz

k is just the derivative of thecorresponding polynomial function.

Considering the particular case of 1, 0,−1, 0, 0 . . ., the sequence q in Example6.2.2, it holds

G′q(z) = (1− z2)′ = −2z.

We now define the integral of a generating function.

Definition 6.2.15 Consider the sequence s = snn∈N0 . The integral of Gs(z),denoted by

∫ z

0Gs(z), is the generating function for the sequence t = tnn∈N0

where t0 = 0 and tn = sn−1

nfor all n ∈ N.


Example 6.2.16 Consider again 1, 1, 1, . . ., the sequence v in Example 6.2.2.The integral of Gv(z) is the generating function for 0, v0

1, v1

2, v2

3, . . .. Hence,

∫ z

0Gv(z) is the generating function for 0, 1, 1

2, 1

3, . . ., the sequence y = (yn)n∈N0

where y0 = 0 and yn = 1n

for each n > 0, and therefore

∫ z

0

Gv(z) =

+∞∑

i=0

1

i + 1zi+1.

Note that (∫ z

0Gv(z))′ = Gv(z) since the derivative of

∫ z

0Gv(z) is the generat-

ing function for 1× 1, 2× 12, 3× 1

3, . . ., that is, the sequence whose terms are all

equal to 1.

Example 6.2.17 Recall again the sequence t = (tn)n∈N0 in Example 6.2.14 andthe generating function Gt(z) = t0 + t1z + t2z

2 + . . .+ tkzk. The integral of Gt(z)

is the generating function for

0, t01, t1

2, t2

3, . . ., tk

k+1, 0, 0,. . .

the sequence y = (yn)n∈N0 where y0 = 0, yn = tn−1

nfor each 1 ≤ n ≤ k + 1 and

yn = 0 para n ≥ k + 2. Hence,∫ z

0

Gt(z) = t0z +t12

z2 +t23

z3 + . . . +tk

k + 1zk+1.

In the particular case of the sequence q in Example 6.2.2, it holds∫ z

0

Gq(z) =

∫ z

0

(1− z2) = z − 1

3z3.

The usual properties regarding the derivative of the sum and of the producthold, that is, given generating functions Gs(z) e Gt(z)

(Gs(z) + Gt(z))′ = G′s(z) + G′

t(z) (6.10)

and(Gs(z)×Gt(z))′ = G′

s(z)×Gt(z) + Gs(z)×G′t(z). (6.11)

Moreover, if Gs(z) has a multiplicative inverse then(

1

Gs(z)

)′= − G′

s(z)

(Gs(z))2. (6.12)

The derivative of the integral of a generating function is the original generatingfunction, that is, (

∫ z

0Gs(z))′ = Gs(z). The proofs of these properties are left as

an exercise to the reader.Observe that the notion of derivative of a generating function coincides with

the usual notion of derivative of a function admitting a power series expansion,within its domain. Similarly, with respect to the integral.


Closed forms

It is often useful to get a closed form for a generating function Gs(z), that is,to get an equality Gs(z) = e where the expression e does not explicitly involvepower series. The following examples illustrate how we can obtain closed formsfor some generating functions.

Example 6.2.18 Let v be the sequence 1, 1, 1, . . ., presented in Example 6.2.2.The equalities

zGv(z) =+∞∑

i=0

zi+1 =

(+∞∑

i=0

zi

)

− 1 = Gv(z)− 1

hold. SolvingzGv(z) = Gv(z)− 1

for Gv(z) (within the ring (G, +, 0,−×)) we get

Gv(z) =1

1− z(6.13)

thus obtaining a closed form for Gv(z). This technique can be generalized toconclude that

Gs(z) = Gav(z) =a

1− z

where s is a sequence whose terms are all equal to a real number a, that is, s = av.Observe that we can also use the fact that Gv(z)−1 = 1− z (see Example

6.2.10) to conclude that Gv(z) = (1− z)−1 and, as a consequence, the equality(6.13).

Example 6.2.19 Let a be a real number and let ga be the sequence presentedin Example 6.2.11 (geometric progression with ratio a and first term 1). From

Gga(z) =

+∞∑

i=0

aizi

we get

azGga(z) =

+∞∑

i=0

ai+1zi+1 =

(+∞∑

i=0

aizi

)

− 1 = Gga(z)− 1

SolvingazGga

(z) = Gga(z)− 1.

for Gga(z) we conclude that

Gga(z) =

1

1− az(6.14)


thus obtaining a closed form for Gga(z).

Again we could have used use the fact that Gga(z)−1 = 1− az (see Example

6.2.11) to conclude that Gga(z) = (1 − az)−1, and therefore the equality (6.14).

Note also that (6.13) is a particular case of (6.14), since v is g1.

Example 6.2.20 Let w be the sequence 1, 0, 0, 1, 0, 0, 1, 0, 0, 1 . . . presented inExample 6.2.2. Then

z3Gw(z) =

+∞∑

i=0

z3i+3 = Gw(z)− 1

and therefore, solving z3Gw(z) = Gw(z)− 1 for Gw(z), we get the closed form

Gw(z) =1

1− z3. (6.15)

We can reason as above when for some k ∈ N the sequence w = (wn)n∈N0 is suchthat wn = 1 if n is multiple of k and wn = 0 otherwise, for each n ∈ N0. Theresulting closed form is analogous to (6.15) with k instead of 3.

The following example uses the sum of generating functions.

Example 6.2.21 Let p be the sequence 2, 0, 2, 0, . . ., that is, the sequencep = pnn∈N0 where pn = 2 when n is even and pn = 0 otherwise. Note that

p = v + g−1

where v is the sequence 1, 1, 1, 1, . . . in Example 6.2.18 and g−1 is the geometricprogression with ratio −1 and first term 1. Hence, recalling the closed forms(6.13) and (6.14), we conclude that

Gp(z) = Gv(z) + Gg−1(z) =1

1− z+

1

1 + z.

In the next example we use the derivative of generating functions to obtain aclosed form.

Example 6.2.22 Recall from Example 6.2.13 that

Gx(z) = G′v(z)

where x is the sequence 1, 2, 3, 4,. . . and v is the sequence 1, 1, 1, 1,. . .. We canget a closed form for Gx(z) using (6.12) and the closed form (6.13):

Gx(z) = G′v(z) =

(1

1− z

)′=

1

(1− z)2. (6.16)


From the closed form obtained in Example 6.2.22 we can obtain closed formsfor other generating function.

Example 6.2.23 Let r be the sequence 0, 1, 2, 3, . . ., in Example 6.2.2. From(6.16) we conclude that

Gr(z) = zGt(z) =z

(1− z)2. (6.17)

6.3 Motivating examples revisited

6.3.1 Search by hashing

We first describe how to use generating functions for computing the expectedvalue and the variance of some discrete random variables. We then present theaverage case analysis of the key search algorithm discussed in Section 6.1.

Discrete random variables and probability generating functions

We briefly recall several basic notions concerning discrete random variables, be-ginning with the notion of discrete probability space.

Definition 6.3.1 A discrete probability space is a pair (Ω, p) where Ω is a count-able set and p : Ω → [0, 1] is a map such that

∑

ω∈Ω p(ω) = 1. Each element ofΩ is an elementary event.

Given a discrete probability space (Ω, p), the map p can be extended to subsetsof Ω considering the map p : 2Ω → [0, 1] such that p(A) =

∑

ω∈A p(ω). When thereis no ambiguity we just use p for p. We now introduce discrete random variables.

Definition 6.3.2 A random variable over a discrete probability space (Ω, p) isa map X : Ω→ R.

A random variable over a discrete probability space is said to be a discreterandom variable. In the sequel, X(Ω) denotes the set X(w) : w ∈ Ω. More-over, we say that the random variable X takes only values in a set C wheneverX(Ω) ⊆ C. For instance, if X(Ω) ⊆ N0 we say that X takes only nonnegativeinteger values. We can associate a probability function with each discrete randomvariable.

Definition 6.3.3 Let X : Ω→ R be a discrete random variable over (Ω, p). Theprobability function associated with X is the map PX : R → [0, 1] such thatPX(x) = p(w ∈ Ω : X(w) = x).


We also write P (X = x) to denote PX(x). Note that Definition 6.3.3 impliesthat PX(x) is always 0 whenever x /∈ X(Ω).

When characterizing a discrete random variable X over a probability space(Ω, p) we often do not explicitly define X(w) for each w ∈ Ω, but we just say whichvalues X takes, that is, we just define X(Ω). Moreover, we may not explicitlyrefer to the the function p, defining instead the values of PX(x) for each x ∈ X(Ω)in a suitable way.

In some situations it is relevant to consider several random variables over thesame probability space.

Definition 6.3.4 Let X and Y be discrete random variables over the same prob-ability space (Ω, p). The joint probability function associated with X and Y isthe map PXY : R2 → [0, 1] such that

PXY (x, y) = p(w ∈ Ω : X(w) = x ∩ w ∈ Ω : Y (w) = y).

The two variables are independent whenever PXY (x, y) = PX(x)PY (y).

Some parameters are useful to characterize the probability function of a ran-dom variable, such as the expected value and the variance. Since in the sequelwe only refer to discrete random variables taking only nonnegative integer values,we just introduce expected value and variance of such random variables.

Definition 6.3.5 Let X be a discrete random variable over (Ω, p) taking onlynonnegative integer values. For each m ∈ N, the m-th moment of X, denoted byE(Xm), is

+∞∑

k=0

kmPX(k)

whenever this summation is a real number. The first moment of X, that is,E(X) =

∑+∞k=0 kPX(k) is the expected value, or mean, of X. The variance of X

is V (X) = E((X − E(X))2).

It can be easily proved that

V (X) = E((X − E(X))2) = E(X2)− (E(X))2 (6.18)

and therefore the variance of X can be computed using the first and the secondmoments of X.

We now introduce probability generating functions. Let X be a discrete ran-dom variable taking only nonnegative integer values. We can consider the se-quence PX(0), PX(1), PX(2), . . ., and therefore the corresponding the generatingfunction, the probability generating function associated with X.


Definition 6.3.6 Let X be a discrete random variable over (Ω, p) taking onlynonnegative integer values. The probability generating function of X, denoted byGX(z), is the generating function sX = (sX)nn∈N0 such that (sX)n = PX(n) forall n ∈ N0, that is,

GX(z) =+∞∑

k=0

PX(k)zk.

The sets w ∈ Ω : X(w) = k and w ∈ Ω : X(w) = k′ are disjoint for dis-tinct k, k′ ∈ N0, and

⋃

k∈N0(w ∈ Ω : X(w) = k) = Ω. As a consequence, recall-

ing also Definition 6.3.1, the equalities∑+∞

k=0 PX(k) =∑

w∈Ω p(w) = 1 hold, thatis, GX(1) = 1.

The expected value of a discrete random variable X taking only nonnegativeinteger values can be computed using the derivative of the probability generatingfunction of X.

Proposition 6.3.7 Let X be a discrete random variable over (Ω, p) taking onlynonnegative integer values. Then E(X) = G′

X(1).

Proof: The first derivative of the probability generating function of X is

G′X(z) =

+∞∑

k=0

(k + 1)PX(k + 1)zk. (6.19)

Therefore,

G′X(1) =

+∞∑

k=0

(k + 1)PX(k + 1) =+∞∑

k=0

kPX(k) = E(X).

Note that E(X) is a real number if and only if G′X(1) is a real number. QED

The second moment of X can be computed using the first and the secondderivatives of GX(z).

Proposition 6.3.8 Let X be a discrete random variable over (Ω, p) taking onlynonnegative integer values.

1. E(X2) = G′′X(1) + G′

X(1)

2. V (X) = G′′X(1) + G′

X(1)− (G′X(1))2


Proof:1. The second derivative of the probability generating function of X is

G′′X(z) =

+∞∑

k=0

(k + 1)(k + 2)PX(k + 2)zk

Therefore,

G′′X(z) + G′

X(z) =+∞∑

k=0

(k − 1)kPX(k)zk ++∞∑

k=0

kPX(k)zk

=

+∞∑

k=0

((k − 1)kPX(k) + kPX(k))zk

=+∞∑

k=0

k2PX(k)zk

Hence, whenever G′′X(z) and G′

X(z) converge at z = 1 it holds that

G′′X(1) + G′

X(1) =

+∞∑

k=0

k2PX(k) = E(X2).

2. It follows from 1, Proposition 6.3.7 and the equality (6.18). QED

The following result is useful in the sequel.

Proposition 6.3.9 Let X and Y be two discrete random variables over (Ω, p)taking only nonnegative integer values. If X and Y are independent randomvariables then GX+Y (z) = GX(z)GY (z).

Proof: The probability generating function of X + Y is

GX+Y (z) =

+∞∑

k=0

PX+Y (k)zk.

Since X and Y are independent,

PX+Y (k) =k∑

i=0

PXY (i, k − i) =k∑

i=0

PX(i)PY (n− i)

for each k ∈ N0. Hence,

GX+Y (z) =

(+∞∑

k=0

PX(k)zk

)(+∞∑

k=0

PY (k)zk

)

= GX(z)GY (z).


QED

The above result can be extended to the sum of a finite number of ran-dom variables, that is, the equality GX1+X2+...+Xn

(z) = GX1(z)GX2(z) . . . GXn(z)

holds.

Average case analysis: unsuccessful search

In this section we return to the motivating example presented in section 6.1.1 andshow how to use probability generating functions in the average-case analysis ofthe key searching algorithm. The average case analysis depends on whether thekey has already been stored (successful search) or not (unsuccessful search). Wefirst consider unsuccessful search.

Assume we are searching for a key K and assume that n ∈ N keys have alreadybeen stored in table Key. For simplicity, we denote by Ki the key Key[i]. Supposewe have a hash function h : K → 1, 2, . . . , m, where m ∈ N and K is the keyspace, and therefore there are m different lists where to search for K.

Recall the function keySearch in Figure 6.1 to determine whether k has al-ready been stored. We want to determine the average number of comparisonskey[[j]]==k that are performed when searching for k. To this end some prob-abilistic hypothesis have to be considered. Since we are analyzing unsuccessfulsearches, the key K is not yet been stored. Hence, K 6= Ki for each 1 ≤ i ≤ n.

For illustration purposes suppose m = 2 that is, we have two lists L1 and L2,and suppose n = 3. There are 23 = 8 possible scenarios that can be sketched asfollows:

L1 L2

K1

K2

K3

(1)

L1 L2

K1

K2

K3

(2)

L1 L2

K1 K2

K3

(3)

L1 L2

K2 K1

K3

(4)

L1 L2

K3 K1

K2

(5)

L1 L2

K1 K3

K2

(6)

L1 L2

K1 K2

K3

(7)

L1 L2

K2 K1

K3

(8)

Observe that if h(K) = 1 then we have to perform

• 0 comparisons key[[j]]==k if scenario (2) is the case;

• 1 comparison key[[j]]==k if scenario (3), (4) or (5) is the case;


• 2 comparisons key[[j]]==k if scenario (6), (7) or (8) is the case;

• 3 comparisons key[[j]]==k if scenario (1) is the case.

Similarly, if h(K) = 2 then 0, 1, 2 or 3 comparisons have to be performed when wehave scenario (1), scenarios (6), (7) or (8), scenarios (3), (4) or (5), and scenario(2) respectively.

Note that in every scenario each stored key Ki is compared to K at most onceand it is only compared to K whenever h(K) = h(Ki). Furthermore, K is alwayscompared with all the keys in the list h(K).

Thus, with respect to the probability space involved, we can consider in thisexample 16 (=8×2) elementary events w corresponding to the 8 possible ways K1,K2 and K3 can be distributed by the 2 lists and the 2 possible values of h(K).We can use a tuple (r1, r2, r3, r4) ∈ 1, 24 to represent such an event lettingri = h(Ki) for 1 ≤ i ≤ 3 and r4 = h(K). In the uniform case, all the elementaryevents have the same probability and therefore p(w) = 1

16for each elementary

event w.To characterize the number of comparisons we first consider the random vari-

ables X1, X2 and X3 over (Ω, p), where each Xi is a random variable that onlytakes values in 0, 1, corresponding to the number of possible comparisons be-tween K and Ki. Then, the number of comparisons NC can be defined asNC = X1 + X2 + X3.

We now return to the general case with n ∈ N keys already stored and a hashfunction h : K → 1, 2, . . . , m with m ∈ N. To compute the average number ofthe intended comparisons we reason as follows.

(1) The discrete probability space is (Ω, p) where Ω = 1, . . . , mn+1. As de-scribed above, each elementary event represents one of the mn+1 possiblesituations corresponding to the mn possible ways the n keys can be dis-tributed by the m lists and the m possible values of h(K).

(2) For each 1 ≤ i ≤ n we have a random variable Xi over the probability space(Ω, p) such that Xi(Ω) = 0, 1 and

Xi(r1, . . . , rn+1) =

1 if rn+1 = ri

0 otherwise

The values of Xi correspond to number of possible comparisons between Kand Ki. Letting Ai = (r1, . . . , rn+1) ∈ Ω : ri = rn+1 for each 1 ≤ i ≤ n,

PXi(1) =

∑

w∈Ai

p(w) and PXi(0) = 1− PXi

(1).


The random variables X1, . . . , Xn are pairwise independent.

(3) Finally, we consider the discrete random variable

NC =

n∑

i=1

Xi

Note that NC(Ω) = 0, 1, . . . , n. The values of NC correspond to thepossible number of the intended comparisons. The average case number ofcomparisons is then E(NC), the expected value of NC.

(4) We use the probability generating function of NC to compute E(NC).Since X1, . . . , Xn are pairwise independent

GNC(z) =n∏

i=1

GXi(z)

and therefore we have to compute GXi(z) for each 1 ≤ i ≤ n. Since

GXi(z) =

+∞∑

r=0

PXi(r)zr = PXi

(0) + PXi(1)z

we get

GNC(z) =n∏

i=1

(PXi(0) + PXi

(1)z).

Then we just compute E(NC) = G′NC(1). We can also compute the vari-

ance V (NC) = G′′NC(1) + G′

NC(1)− (G′NC(1))2.

The following example illustrates the average case analysis assuming an uni-form distribution, that is, every elementary event is equally likely.

Example 6.3.10 Assuming an uniform distribution, that is, a situation whereevery elementary event w is equally likely:

• (Ω, p) is such that Ω = 1, . . . , mn+1 and p(w) = 1mn+1 for each w ∈ Ω,

where m, n ∈ N.

• for 1 ≤ i ≤ n

PXi(1) = m×mn−1 × 1

mn+1=

1

m


PXi(0) = 1− PXi

(1) =m− 1

m

GXi(z) =

m− 1

m+

1

mz

• GNC(z) =n∏

i=1

GXi(z) =

(m− 1

m+

1

mz

)n

• G′NC(z) =

n

m

(m− 1

m+

1

mz

)n−1

• G′′NC(z) =

n(n− 1)

m2

(m− 1

m+

1

mz

)n−2

The average case number of comparisons between the stored keys and thegiven key K is then

E(NC) = G′NC(1) =

n

m

The variance of NC is V (NC) = G′′NC(1) + G′

NC(1)− (G′NC(1))

2=

n(m− 1)

m2.

To end this section we return to the characterization of the random variablesXi for 1 ≤ i ≤ n. An alternative way of characterizing Xi without explicitlyinvolving the elementary events and their probabilities is as follows:

• Xi =

1 if h(K) = h(Ki)

0 otherwise

and

• PXi(1) =

m∑

r=1

P (h(K) = r e h(Ki) = r)

PXi(0) = 1− PXi

(1)

where P (h(K) = r e h(Ki) = r) ∈ [0, 1], for each 1 ≤ r ≤ m, is the probabilitythat both h(K) = r and h(Ki) = r. This probability depends on probabilityspace involved. In the uniform case described in Example 6.3.10,

P (h(K) = r e h(Ki) = r) = P (h(K) = r)× P (h(Ki) = r)

where P (h(K) = r) is the probability that h(K) = r and P (h(Ki) = r) is theprobability that h(Ki) = r, being both probabilities equal to 1

m. Hence, as ex-

pected

PXi(1) =

m∑

r=1

1

m× 1

m= m× 1

m2=

1

m.


Let us give a closer look to the relationship between the two definitions wehave presented for PXi

(1). Recall the set Ai defined above for each 1 ≤ i ≤ n. Wehave that Ai = A1

i ∪ . . . ∪ Ami where Ar

i = (r1, . . . , rn+1) ∈ Ω : ri = rn+1 = r foreach 1 ≤ r ≤ m. These sets are pairwise disjoint and therefore

PXi(1) =

∑

w∈Ai

p(w) =

m∑

r=1

p(Ari )

Note that p(Ari ) is the probability that h(K) = h(Ki) = r which just corresponds

to the probability P (h(K) = r e h(Ki) = r) mentioned above.

Average case analysis: successful search

We now address the successful search case. We again assume we are searching fora key K, that n ∈ N keys have already been stored, and that the hash functionis h : K → 1, 2, . . . , m, whit m ∈ N. Recall that Ki denotes the key Key[i] andthat K1 was the first key to be stored, K2 the second and so on. Since, we areanalyzing successful searches, the key K we are looking for is one of the keysalready stored.

This situation differs from the case of unsuccessful searches in several aspects.To begin with note that the minimum number of comparisons in the case of anunsuccessful search is 0 but it is 1 in the case of a successful search, since listh(K) is never empty. Moreover, in an unsuccessful search, K is compared withall the keys in the list h(K), whereas in a successful search this may not be thecase since we need no more comparisons once we find K. Observe also that ifK = Kj for some 1 ≤ j ≤ n then there are no comparisons between K and Kj′

for any j′ > j, and therefore the maximum number of comparisons is j. This isthe case because when j′ > j then Kj′ occurs after Kj in table Key. Finally, notethat in a successful search we have to take also into account the probability thatK = Kj for each 1 ≤ j ≤ n.

To compute the average number of the intended comparisons we reason asfollows.

(1) The discrete probability space is (Ω, p) where in this case

Ω = 1, . . . , mn × 1, . . . , n.

Each possible scenario consists of a particular distribution of the keys K1,K2, . . ., Kn over the m lists and K = Ki for some 1 ≤ i ≤ n. Thus, eachelementary event is a tuple (r1, r2, . . . , rn+1) ∈ Ω where ri = h(Ki) for each1 ≤ i ≤ n and rn+1 ∈ 1, . . . , n indicates which of the n keys is K.

(2) We consider a discrete random variable Y over the discrete probabilityspace (Ω, p) such that Y (Ω) = 1, . . . , n, with Y (r1, . . . , rn+1) = rn+1. The


values of Y correspond to the n possible values of K. Dropping the referenceto the elementary events, Y is defined just stating that

Y = j when K = Kj for 1 ≤ j ≤ n.

Letting Bj ⊆ Ω be the set of all elementary events whose last componentis j, we have PY (j) =

∑

w∈Bjp(w) for each 1 ≤ j ≤ n.

(3) For each 1 ≤ j ≤ n we also consider the discrete random variables

X1j, X2j , . . ., Xjj

over (Ω, p) such that Xij(Ω) = 0, 1 for each 1 ≤ i ≤ j. The values ofXij correspond to the possible number of comparisons between K and Ki,assuming that K is Kj . Recall that when K = Kj there are no comparisonsbetween K and Kj′ for any j′ > j, hence for each j we only need Xij foreach 1 ≤ i ≤ j. Moreover, there is only a comparison between Kj and Ki

whenever h(Ki) = h(Kj). Dropping the reference to the elementary eventsXij is such that

Xij =

1 if h(Ki) = h(Kj)

0 otherwise

Note that Xjj is always equal to 1. Thus,

PXij(1) =

1 when i = jm∑

r=1

P (h(ki) = r e h(kj) = r) when i 6= j

and

PXij(0) = 1− PXij

(1)

where P (h(K) = r e h(Ki) = r) ∈ [0, 1] is the probability that both h(K) = rand h(Ki) = r, for each 1 ≤ r ≤ m.

The random variables X1j , X2j , . . ., Xjj are pairwise independent.

(4) For each 1 ≤ j ≤ n we consider the random variable

NCj =

j∑

i=1

Xij

Note that NC(Ω) = 1, . . . , j. The values of NCj correspond to the pos-sible number of comparisons between the stored keys and K, assuming thatK is Kj.


(5) We then have the random variable NC over (Ω, p) such that

NC(Ω) = 1, . . . , n.

The values of NC correspond the possible number of comparisons betweenthe stored keys and K. For each 1 ≤ nc ≤ n we have that

PNC(nc) =

n∑

j=1

PY (j)PNCj(nc).

The average case number of comparisons is E(NC), the expected value ofNC.

(6) The probability generating function of NC is used to compute E(NC). Wecan prove that

GNC(z) =n∑

j=1

PY (j)GNCj(z).

For each 1 ≤ j ≤ n and 1 ≤ i ≤ j we have

GXij(z) =

+∞∑

r=0

PXij(r)zr = PXij

(0) + PXij(1)z

and, since X1j, X2j , . . ., Xjj are pairwise independent, we get

GNCj(z) =

j∏

i=0

GXij(z) =

j∏

i=0

(PXij(0) + PXij

(1)z).

Finally, the average number of comparisons is E(NC) = G′(1). The vari-ance V (NC) can also be computed as expected.

The following example illustrates the average case analysis assuming thatevery elementary event is equally likely.

Example 6.3.11 Assuming an uniform distribution, that is, a situation whereevery elementary event w is equally likely:

• (Ω, p) is such that Ω = 1, . . . , mn × 1, . . . , n and p(w) = 1mn×n

for eachw ∈ Ω, with m, n ∈ N.

• PY (j) = mn × 1

mn × n=

1

nfor each 1 ≤ j ≤ n

• for 1 ≤ j ≤ n and 1 ≤ i ≤ j


PXij(1) =

1 when i = jm∑

r=1

1

m× 1

mwhen i 6= j

that is,

PXij(1) =

1 when i = j

1

mwhen i 6= j

and therefore

PXij(0) = 1− PXij

(1) =

0 when i = j

m− 1

mwhen i 6= j

Moreover,

GXij(z) =

z when i = j

m− 1

m+

1

mz when i 6= j

• GNCj(z) = z

(m− 1

m+

1

mz

)j−1

for each 1 ≤ j ≤ n

• GNC(z) =

n∑

j=1

z

(1

n

)(m− 1

m+

1

m

)j−1

=z

n

n∑

j=1

(m− 1

m+

1

mz

)j−1

The average case number of comparisons between the stored keys and thegiven key K is then

E(NC) = G′NC(1) =

n− 1

2m+ 1.


We first discuss how generating functions can be used for solving recurrence re-lations. After presenting a first example, we consider the case of the Fibonaccisequence. We then return to the analysis of the Euclid’s algorithm already intro-duced in Section 6.1.2.


Solving recurrence relations: a first example

Consider the sequence t = tnn∈N0 such that

t0 = 0 and tn = 3tn−1 − 2 for n ≥ 1. (6.20)

We will use the generating function

G(z) =+∞∑

i=0

tizi

for t to solve the recurrence relation 6.20, in order to get an expression for tnthat does not depend on other elements of the sequence. In the sequel, recall thenotations and closed forms presented in Section 6.2. Let us proceed as follows.

(i) First of all note that

G(z) =

+∞∑

i=0

tizi =

+∞∑

i=1

tizi

=+∞∑

i=1

(3ti−1 − 2)zi

= 3

+∞∑

i=1

ti−1zi − 2

+∞∑

i=1

zi

= 3+∞∑

i=0

tizi+1 − 2(−1 +

+∞∑

i=0

zi)

= 3z

+∞∑

i=0

tizi − 2(−1 +

1

1− z)

= 3z+∞∑

i=0

tizi − 2z

1− z

= 3zG(z)− 2z

1− z

(ii) Solving

G(z) = 3zG(z) − 2z

1− z

for G(z) we get

G(z) =−2z

(1− z)(1− 3z)(6.21)


(iii) Now, the goal is to expand the right hand side of (6.23) into a power series∑+∞

i=0 aizi, since then the coefficient ai is an expression for ti, for each i ∈ N0.

(iii.1) On one hand, from

−2z

(1− z)(1− 3z)=

A

1− z+

B

1− 3z

we get A(1− 3z) + B(1− z) = −2z. For z = 0 and z = −1 we get A + B = 0and 4A + 2B = 2, respectively, and therefore A = 1 and B = −1. Thus,

−2z

(1− z)(1− 3z)=

1

1− z+−1

1− 3z.

(iii.2) On the other hand,

1

1− z+−1

1− 3z=

+∞∑

i=0

zi −+∞∑

i=0

3izi

=+∞∑

i=0

(−3i + 1)zi

(iv) From (ii) and (iii) we get

G(z) =+∞∑

i=0

(−3i + 1)zi

and therefore t = tnn∈N0 is such that tn = −3n + 1 for each n ∈ N0.

Solving the Fibonacci sequence

The Fibonacci numbers are present in several different situations, from Biology(eg the arrangement of leaves on a stem and the patterns of some pine cones) toEconomics (eg trading algorithms and strategies in financial markets). Fibonaccinumbers are used in some pseudorandom generators and, as we have alreadymentioned in Section 6.1.2, in the analysis of the Euclid’s algorithm. Recall thatthe Fibonacci sequence is the sequence s = snn∈N0 such that

s0 = 0 s1 = 1 and sn = sn−1 + sn−2 for n ≥ 2. (6.22)


Using the generating function

G(z) =+∞∑

i=0

sizi

of the Fibonacci sequence we are able to solve the recurrence relation 6.22, thusobtaining an expression for sn that does not depend on other elements of thesequence. The following steps are similar to the ones presented in the previousexample.

(i) Note that

+∞∑

i=0

sizi = z +

+∞∑

i=2

sizi

= z +

+∞∑

i=2

(si−1 + si−2)zi

= z ++∞∑

i=2

si−1zi +

+∞∑

i=2

si−2zi

= z +

+∞∑

i=0

sizi+1 +

+∞∑

i=0

sizi+2

= z + z+∞∑

i=0

sizi + z2

+∞∑

i=0

sizi.

Hence, G(z) = z + zG(z) + z2G(z).

(ii) Solving G(z) = z + zG(z) + z2G(z) for G(z) we get

G(z) =z

1− z − z2(6.23)

(iii) The goal is to expand the right hand side of (6.23) into a power series∑+∞

i=0 aizi, since then the coefficient ai is the Fibonacci number si, for each

i ∈ N0.

(iii.1) To make things easier we first rewrite it as follows. Note that the roots

of 1− z − z2 are β1 = −1+√

52

. and β2 = −1−√

52

and note that β1β2 = −1.Hence,


1− z − z2 = −(z − β1)(z − β2)

= −β1β2

(z

β1

− 1

)(z

β2

− 1

)

=

(

1− z

β1

)(

1− z

β2

)

and therefore we get

z

1− z − z2=

z

(1− α1z)(1− α2z)

where α1 = 1β1

= 2−1+

√5

= 1+√

52

and α2 = 1β2

= 2−1−

√5

= 1−√

52

.

Fromz

(1− α1z)(1− α2z)=

A

1− α1z+

B

1− α2z

we get the equation A(1− α2z) + B(1− α1z) = z. For z = 0 and z = 1 thisequation yields A + B = 0 and A(1− α2) + B(1− α1) = 1 respectively andtherefore A = 1√

5and B = − 1√

5. As a consequence,

z

(1− α1z)(1− α2z)=

1√5

(1

1− α1z− 1

1− α2z

)

(iii.2) We now expand the right hand side of (6.23) into a power series∑+∞

i=0 aizi as follows

z

1− z − z2=

1√5

(1

1− α1z− 1

1− α2z

)

=1√5

(+∞∑

i=0

(α1z)i −+∞∑

i=0

(α2z)i

)

=+∞∑

i=0

(αi

1 − αi2√

5

)

zi

(iv) From (ii) and (iii) we conclude that the generating function for the Fi-bonacci sequence is

G(z) =

+∞∑

i=0

(αi

1 − αi2√

5

)

zi

and therefore the Fibonacci sequence s = snn∈N0 is such that


sn =αn

1 − αn2√

5=

1√5

((

1 +√

5

2

)n

−(

1−√

5

2

)n)

for all n ∈ N0.

Many recurrence relations can also be solved reasoning as above for the Fi-bonacci sequence. We do not further develop this subject herein. The interestedreader is referred, for instance, to [31]. We just note that step (iii) may be notso easy as in the case of the Fibonacci sequence and the following theorem, therational expansion theorem for distinct roots, may be helpful. For simplicity, inthe sequel we also use p to denote the polynomial function associated with apolynomial p in R[z].

Proposition 6.3.12 Let p =∑n

i=0 pizi and q =

∑m

i=0 qizi be polynomials in

R[z] such that deg(p) < deg(q) and

q = q0(1− ρ1z) . . . (1− ρmz)

where ρi 6= 0 and ρi 6= ρj for all 1 ≤ i, j ≤ m with i 6= j. The function r =p

qis

such that

r(z) =

+∞∑

i=0

aizi

where, for each i ∈ N0

ai =m∑

j=1

bjρij

with

bj =−ρjp(1/ρj)

q′(1/ρj)

for each 1 ≤ j ≤ m.

To find the real numbers ρ1, . . . , ρm referred to in Proposition 6.3.12 it is oftenuseful to use the following property.

Proposition 6.3.13 Let q =∑m

i=0 qizi be a polynomial of degree m in R[z] such

that q0 6= 0. Let qr =∑m

i=0 qm−izi. Then q = q0(1− ρ1z) . . . (1− ρmz) if and only

if qr = q0(z − ρ1) . . . (z − ρm).

The following proposition states that all sequences involving a linear recur-rence relation have indeed generating functions given by the quotient of twopolynomial functions.


Proposition 6.3.14 Let s = snn∈N0 be a sequence given by a linear recurrencerelation of order k ∈ N, that is, the real numbers s0, . . . , sk−1 are explicitly givenand the other terms are defined by the recurrence relation

sn+k = c1sn+k−1 + c2sn+k−2 + . . . + cksn

where c1, . . . , ck ∈ R. Then

Gs(z) =

+∞∑

i=0

sizi =

p

q

where p and q are polynomials in R[z] such that deg(p) = k and deg(p) ≤ k.

Euclid’s algorithm analysis

We now return to the analysis of the Euclid’s algorithm (see Figure Figure 1.4)introduced in Section 6.1.2. The analysis of this algorithm involves countingthe number of recursive calls performed when euclid[m,n] is evaluated. Asexplained in Section 6.1.2 it is enough to study the case where m < n.

Proposition 6.3.15 If when evaluating euclid[m,n] with n > m ≥ 1 there arek ∈ N recursive calls then n ≥ sk+2 and m ≥ sk+1 where s = snn∈N0 is the Fi-bonacci sequence.

Proof: We use induction on k.

Basis: If k = 1 then m ≥ 1 = s2. Since n > m, we have n ≥ 2 = s3.

Step: We have to prove that n ≥ sk+3 and m ≥ sk+2 when euclid[m,n] evalua-tion involves k + 1 ≥ 2 recursive calls.

Since there are at least two recursive calls, then m ≥ 1. Therefore, the firstrecursive call is euclid[Mod[n,m],m]. Moreover, there k > 1 recursive calls in-volved in the evaluation of euclid[Mod[n,m],m]. Hence, by the induction hy-pothesis, m ≥ sk+2 and Mod[n,m] ≥ sk+1. But

n =⌊ n

m

⌋

m + Mod[n,m]

and, since m < n, the inequality ⌊ nm⌋ ≥ 1 holds and, as consequence, ⌊ n

m⌋m ≥ m.

We can then conclude that n ≥ m + Mod[n,m] ≥ sk+2 + sk+1 = sk+3. QED

The following theorem is known as the Lame theorem. It establishes anupper bound for the number of recursive calls that take place when evaluatingeuclid[m,n]. It easily follows from Proposition 6.3.15.

Theorem 6.3.16 For each k, m, n ∈ N, if m < sk+1, where s = snn∈N0 is theFibonacci sequence, then the evaluation of euclid[m,n] involves less than krecursive calls.


Proof: If there were k or more recursive calls, by Proposition 6.3.15, we wouldhave m ≥ sk+1. QED

The nonrecursive definition of the Fibonacci sequence s is relevant herein. Re-

call that sn = 1√5

((1+

√5

2

)n

−(

1−√

52

)n)

for all n ∈ N0. Hence, we can conclude

that k ∈ O(loge m) since sk is approximately 1√5

(1+

√5

2

)k

for large integers k,

given that |1−√

52| < 1. We can prove that the function Mod involves a O(log2

e(m))number of bit operations and therefore there are a O(log3

3(m)) number of bitoperations.

The Lame’s theorem establishes an upper bound for the number of recursivecalls that take place when evaluating euclid[m,n], but we do not know how closethe real number of calls is to this upper bound. The following proposition statesthat if the conditions of the theorem hold, the worst case number of recursive callsis indeed close to k when evaluating euclid[sk,sk+1], that is, when computingthe greatest common divisor of two consecutive Fibonacci numbers. In this casethere are k − 1 recursive calls.

Proposition 6.3.17 Let s = snn∈N0 be the Fibonacci sequence. For eachk ∈ N0 such that k ≥ 2 the evaluation of euclid[sk,sk+1] involves exactly k−1recursive calls.

Proof: We use induction on k ≥ 2.

Basis: Let k = 2. Since s2 = 1 and s3 = 2, the evaluation of euclid[s2,s3]

involves exactly one recursive call, that is k − 1 recursive calls.Step: Let k > 2. Then sk−1 < sk and therefore ⌊sk+sk−1

sk⌋ = 1. Hence,

mod(sk+1, sk) = sk+1 − sk

⌊sk+1

sk

⌋

= (sk + sk−1)− sk

⌊sk + sk−1

sk

⌋

= sk−1

The evaluation of euclid[sk,sk+1] yelds a call to euclid[Mod[sk+1,sk],sk],that is, to euclid[sk−1,sk]. By the induction hypothesis, the evaluation ofeuclid[sk−1,sk] involves exactly k − 2 recursive calls and therefore the evalua-tion of euclid[sk,sk+1] involves k − 1 recursive calls. QED

6.4 Exercises

1. Consider the sequences s = snn∈N0 and t = tnn∈N0. Prove that

(a) aGs(z) + bGt(z) =∑∞

i=0(asi + bti)zi.

(b) 11−z

Gs(z) =∑∞

i=0(∑i

j=0 sj)zi.

6.4. EXERCISES 213

2. Find a closed form for the generating function for the sequence s = snn∈N0

where

(a) sn = 1 if n is a multiple of 3 and sn = 0 otherwise.

(b) sn = 2 if n is odd and sn = 1 otherwise.

(c) sn = 2n + 1.

(d) sn = 5n + 5.

(e) sn = 3n + 2n.

(f) sn = 32

if n ≥ m and sn = 0 otherwise, for some m ∈ N0.

(g) sn = 3n if n ≥ m and sn = 0 otherwise, for some m ∈ N0.

3. Let (Ω, p) be a discrete probability space. Find a closed form for the prob-ability generating function of the random variable X : Ω→ N0 such that

(a) PX(k) = 16

if k < 6 and PX(k) = 0 otherwise.

(b) PX(k) = 1n

if k < n and PX(k) = 0 otherwise, for some n ∈ N.

(c) PX(k) =(

15

)k.

(d) PX(k) = 23

(13

)k.

(e) PX(k) =(

23

)k+1.

4. Find a closed form for the probability generating function of the randomvariable X that describes a loaded dice such that the probability of seeingan even number of spots in a roll is half the probability of seeing an oddnumber.

5. Find the mean and variance of the random variables X in Exercise 3.

6. Find the mean and variance of the random variable X in Exercise 4.

7. Consider the hashing technique for storing and retrieving information. Leth be the hashing function and let NC be the random variable correspondingto the number of comparisons between keys involved in a unsuccessful searchof a key K. Compute the mean and the variance of NC when

(a) there are 2 lists and the probability that h(K) = 1 is 14.

(b) there are 3 lists and the probability of that h(K) = 1 and that h(K) =2 is p1 and p2 respectively.

8. Consider the hashing technique for storing and retrieving information. Leth be the hashing function and let NC be the random variable correspondingto the number of comparisons between keys involved in a successful search


of a key K. Compute the mean and the variance of NC when there are3 lists and (i) the probability that h(K) = 1 is 1

3and the probability that

h(K) = 2 is 12; (ii) the probability that K is the first key inserted is 3

4and

it is uniformly distributed otherwise.

9. Consider the hashing technique for storing and retrieving information. Leth be the hashing function and let NC be the random variable correspondingto the number of comparisons between keys involved in a successful searchof a key K. Compute the mean and the variance of NC when there are 2lists and the probability that h(K) = 1 is p1.

10. Solve the following recurrence relations using generating functions

(a) s = snn∈N0 where s0 = 0 and sn = 3sn−1 − 2 for n ≥ 1.

(b) s = snn∈N0 where s0 = 2 and sn = 6sn−1 − 5 for n ≥ 1.

(c) s = snn∈N0 where s0 = 0, s1 = 1 and sn = 3sn−1 + 4sn−2 for n ≥ 2.

Bibliography

[1] W. Adams and P. Loustaunau. An Introduction to Grobner Bases, volume 3of Graduate Studies in Mathematics. American Mathematical Society, Prov-idence, RI, 1994.

[2] A. V. Aho, J. E. Hopcroft, and J. D. Ullman. Data Structures and Algo-rithms. Addison-Wesley Series in Computer Science and Information Pro-cessing. Addison-Wesley, 1983.

[3] J. Baillieul and D. P. Martin. Resolution of kinematic redundancy. InRobotics, volume 41 of Proc. Sympos. Appl. Math., pages 49–89. Amer. Math.Soc., Providence, RI, 1990.

[4] L. Blum, M. Blum, and M. Shub. A simple unpredictable random numbergenerator. SIAM Journal on Computing, 15:364–383, 1986.

[5] M. Borges-Quintana, M. A. Borges-Trenard, P. Fitzpatrick, and E. Martınez-Moro. Grobner bases and combinatorics for binary codes. Appl. AlgebraEngrg. Comm. Comput., 19(5):393–411, 2008.

[6] M. Brickenstein, A. Dreyer, A. G.-M. Greuel, M. Wedler, and O. Wienand.New developments in the theory of Grobner bases and applications to formalverification. J. Pure Appl. Algebra, 213(8):1612–1635, 2009.

[7] B. Buchberger. Ein Algorithmus zum Auffinden der Basiselemente des Restk-lassenringes nach einem nulldimensionalen Polynomideal. PhD thesis, 1965.University of Innsbruck.

[8] B. Buchberger. Ein algorithmisches kriterium fur die losbarkeit eines alge-braischen gleichungssystems. Aequationes mathematicae, 4(3):374–383, 1970.

[9] R. Cori and D. Lascar. Mathematical logic. Oxford University Press, 2000.A course with exercises. Part I.

[10] D. Cox, J. Little, and D. O’Shea. Ideals, Varieties and Algorithms. Springer,third edition, 2007.

215

216 BIBLIOGRAPHY

[11] B. A. Davey and H. A. Priestley. Introduction to Lattices and Order. Cam-bridge University Press, second edition, 2002.

[12] R. David and H. Alla. Discrete, Continuous, and Hybrid Petri Nets.Springer, 2005.

[13] G. H. Golub and C. F. Van Loan. Matrix Computations. Johns HopkinsStudies in the Mathematical Sciences. Johns Hopkins University Press, Bal-timore, MD, third edition, 1996.

[14] P. Henrici. Applied and Computational Complex Analysis, Volume 1. JohnWiley & Sons, 1974.

[15] G. A. Jones and J. M. Jones. Elementary Number Theory. Springer Under-graduate Mathematics Series. Springer-Verlag London, London, 1998.

[16] H. Kautz and B. Selman. The state of SAT. Discrete Appl. Math.,155(12):1514–1524, 2007.

[17] D. Knuth. The Art of Computer Programming, Volume 2. Addison-Wesley,third edition, 1998.

[18] S. K. Lando. Lectures on Generating Functions, volume 23 of Student Math-ematical Library. American Mathematical Society, 2003.

[19] S. Lang. Complex Analysis. Springer, forth edition, 1999.

[20] S. Lang. Algebra, volume 211 of Graduate Texts in Mathematics. Springer-Verlag, New York, third edition, 2002.

[21] F. Maric. Formalization and implementation of modern SAT solvers. J.Automat. Reason., 43(1):81–119, 2009.

[22] E. W. Mayr and A. R. Meyer. The complexity of the word problems forcommutative semigroups and polynomial ideals. Advances in Mathematics,46:305–329, 1982.

[23] P. Molitor and J. Mohnke. Equivalence Checking of Digital Circuits: Fun-damentals, Principles, Methods. Springer, 2004.

[24] H. L. Montgomery and R. C. Vaughan. Multiplicative Number Theory. I.Classical Theory, volume 97 of Cambridge Studies in Advanced Mathematics.Cambridge University Press, Cambridge, 2007.

[25] R. P. Paul. Robot Manipulators: Mathematics, Programming, and Control.MIT Press, 1981.

BIBLIOGRAPHY 217

[26] R. Rivest, A. Shamir, and L. Adleman. A method for obtaining digitalsignatures and public key cryptosystems. Commununications of the ACM,(21):120–126, 1978.

[27] M. Sala, T. Mora, L. Perret, S. Sakata, and C. Traverso. Grobner Bases,Coding, and Cryptography. Springer, 2009.

[28] P. Shor. Algorithms for quantum computation: Discrete logarithms andfactoring. In IEEE Symposium on Foundations of Computer Science, 1994.

[29] G. Strang. Linear Algebra and its Applications. Wellesley Cambridge Press,fourth edition, 2009.

[30] J. Tuomela. Kinematic analysis of multibody systems. BIT, 48(2):405–421,2008.

[31] H. S. Wilf. Generatingfunctionology. A. K. Peters, third edition, 2006.

[32] Z. Zhiping, L. Xu, and N. K. Bose. A tutorial on Grobner bases with appli-cations in signals and systems. IEEE Trans. Circuits Syst. I. Regul. Pap.,55(1):445–461, 2008.

218 BIBLIOGRAPHY

Subject Index

additiveinverse, 30unity, 30

Bernoulli number, 142binomial coefficient, 142Buchberger

algorithm, 106polynomial, 97theorem, 98

Carmichael function, 29Chinese remainder theorem, 36, 42closed form

for generating function, 186for summation, 137

congruencemodulo n, 24relation, 26

coprime numbers, 22

degreeof monomial, 69of polynomial, 69of term, 70

discrete Fourier transform, 160inverse, 163

divisionof polynomials, 80of terms, 78

divisor, 11greatest common, 13

Euclidalgorithm, 14, 204extended algorithm, 17, 33

lemma, 20theorem, 22

Eulerphi function, 23theorem, 27, 42

Euler-Maclaurin formula, 142evaluation of polynomial, 71

fast Fourier transform, 165Fibonacci

number, 180sequence, 180, 201

field, 31fundamental theorem of arithmetic,

20

Gaussian elimination technique, 144analysis, 144

generating function, 181closed form, 186derivative, 188integral, 188product, 183ring, 184sum, 183

greatest common divisor, 13Grobner basis, 95, 101

application toinverse kinematics, 68, 121SAT, 66, 118

reduced, 96, 105

harmonic number, 150Horner’s rule, 164

ideal, 91

219

220 SUBJECT INDEX

basis of, 92finitely generated, 92Grobner basis, 95proper, 92set of generators, 92

insertion sort, 147average-case analysis, 150worst-case analysis, 148

integer division, 11isomorphism of rings, 36

linear congruential sequence, 54increment, 54modulus, 54seed, 54

modular congruenceapplication to

pseudo-random numbers, 54public key cryptography, 10

monicpolynomial, 77term, 70

monomial, 69degree of, 69graded lexicographic order, 75lexicographic order, 73order, 73product, 72

multiple, 11multiplicative

inverse, 31unity, 31

multiplicative order, 28

numberBernoulli, 142coprime to, 22harmonic, 150prime, 20pseudo-random, 51

ordergraded lexicographic, 75

lexicographic, 73

perturbation technique, 140polynomial, 69

Buchberger, 97coefficent of, 69degree of, 69division, 80evaluation of, 71leading term, 77monic, 77multivariate, 70point-value representation, 170product, 72, 157reduction, 87reduction in one step, 84ring, 73sum, 72symmetric, 72term of, 70univariate, 70zero polynomial, 69

prime number, 20factorization, 21

probability generating function, 191product

of monomials, 72product of rings, 34

quadratic residue modulo n, 29quotient, 12

reduction of polynomials, 87remainder, 11ring, 30

additive inverse, 30additive unity, 30homomorphism, 35isomorphism, 36multiplicative unity, 31multiplivative inverse, 31of generating functions, 184of polynomials, 73product, 34

SUBJECT INDEX 221

unitary, 31roots of unity, 159RSA cryptosystem, 10, 42

summation, 127, 130additivity of indices, 134application to

analysis of Gaussian eliminationtechnique, 144

analysis of insertion sort algo-rithm, 147

associativity, 134change of variable, 134closed form, 137constant, 134distributivity, 134members of arithmetic progression,

138members of geometric progression,

139perturbation technique, 140

systems of linear congruences, 38

term, 70coefficient of, 70degree of, 70division, 78leading term, 77least common multiple, 97monic, 70monomial of, 70zero term, 70

theoremBuchberger, 98Chinese remainder theorem, 36,

42Euclid, 22Euler, 27, 42fundamental theorem of arithmetic,

20

Vandermonde matrix, 161

222 SUBJECT INDEX

Table of Symbols

+n, 27, 30−n, 27, 30=n, 24>glx, 75>lx, 73B(p1, p2), 97Bk, 142Gs(z), 181Mx1,...,xn

, 69[a]n, 27Buch(G), 105deg(p), 69

b∑

k=a

f(k), 127

d2∑

k=d1

e, 130

∑

k∈A

f(k), 127

gcd(m, n), 13lcm(t1, t2), 97⌊x⌋, 12lt(p), 77Zn, 26, 30, 32euclid(m, n), 14exteuclid(m, n), 17DFTn(a0, a1, . . . , an−1), 160DFT−1

n (a0, a1, . . . , an−1), 161FFT, 165×n, 27, 30m | n, 11

pD−→ p′, 87

pd−→ p′, 84

zkn, 159

0, 70C[x1, . . . , xn], 69mod(n,m), 12

223

discrete mathematics

Documents

motivating examples

discrete fourier transform1635

discrete fourier transform1615

modular congruences

discrete mathematicsincomplete

rings of polynomials

linear congruential

modular arithmetic241