algorithms for lattice transforms and

Algorithms for Lattice Transforms andLattice Transforms for

Algorithms Thore Husfeldt

Lund University and IT University of Copenhagen2

3

23 24 39

234239248

23482349

This Talk in 60 Seconds

x1

x2

x4

x3

y1

y2

y3

y4

In Out “Prefix sum”

y

i

= Âji

x

j


5

32

4

1

5

32+5=

37

32+5+1=

38

32+5+1+4=

42


y

i

= Âji

x

j

Time: Quadratic


5

32

4

1

5

32+5=

37

32+5+1=

38

32+5+1+4=

42


y

i

= Âji

x

j

5

32+5=

37

37+1=

38

38+4=

42

Out

Time: Quadratic

But: can do better

In This Talk

Nice combinatoricsNice algebra

Recent developments in algorithms for NP-hard problems

Our SODA result from last week

Fast Algorithms for computing the zeta transform on various lattices

with applications

Collaborators

Andreas Björklund (Lund), Petteri Kaski (Helsinki),Mikko Koivisto (Helsinki), Jesper Nederlof (Bergen),Pekka Parviainen (Helsinki)

Lattices & Partial Orders

Circuits

Zeta TransformYates’s Algorithm

Möbius InversionGraph colouring

What?Why?How?

Partial Orders & Lattices


ordered by size

1

2

3


ordered by size

1

2

3

1

2

3

ordered by divisibility


ordered by size

1

2

3

1

2

3


1

2

3

6

1

2

3

12

18

36

1

2

3

4

6

8

9

12

18

24

36

72


ordered by size

1

2

3

1

2

3


1

2

3

6

1

2

3

12

18

36

1

2

3

4

6

8

9

12

18

24

36

72

1

23

1

23

6

1

2 3

12 18

36

9 6 4

8

1

3 2

1218

36 24

72


ordered by size

1

2

3

1

2

3


1

2

3

6

1

2

3

12

18

36

1

2

3

4

6

8

9

12

18

24

36

72

1

23

1

23

6

1

2 3

12 18

36

9 6 4

8

1

3 2

1218

36 24

72join of 2 and 3


ordered by size

1

2

3

1

2

3


1

2

3

6

1

2

3

12

18

36

1

2

3

4

6

8

9

12

18

24

36

72

1

23

1

23

6

1

2 3

12 18

36

9 6 4

8

1

3 2

1218

36 24

72join of 2 and 3

lattice


ordered by size

1

2

3

1

2

3


1

2

3

6

1

2

3

12

18

36

1

2

3

4

6

8

9

12

18

24

36

72

1

23

1

23

6

1

2 3

12 18

36

9 6 4

8

1

3 2

1218

36 24

72join of 2 and 3

latticezero

join-irreducible

Zeta Transform

f: L → Rf z(x) = Â

yx

f (y)

Zeta Transform

f: L → Rf z(x) = Â

yx

f (y)

4

3

4

7

Zeta Transform

f: L → Rf z(x) = Â

yx

f (y)

4

3

4

7

4

3

16

4

14

510

Zeta Transform (linear algebra view)

f: L → Rf z(x) = Â

yx

f (y)

4

3

16

4

14

510

f =

2

664

3614

3

775


f: L → Rf z(x) = Â

yx

f (y)

4

3

16

4

14

510

f =

2

664

3614

3

775 z =

2

664

1 1 1 10 1 0 10 0 1 10 0 0 1

3

775


f: L → Rf z(x) = Â

yx

f (y)

4

3

16

4

14

510

f =

2

664

3614

3

775 z =

2

664

1 1 1 10 1 0 10 0 1 10 0 0 1

3

775

Note: zeta transform has inverse, called the Möbius transform

Zeta Transform (categoric view)

f: L → Rf z(x) = Â

yx

f (y)

4

3

16

4

14

510

2 FREL : L ! Linduces a map

Partial order on finite sets (here, a lattice)

RL ! RL

Zeta Transform (categoric view)

f: L → Rf z(x) = Â

yx

f (y)

4

3

16

4

14

510

2 FREL : L ! Linduces a map

Partial order on finite sets (here, a lattice)

RL ! RL

≤

S

T

RS

RT

for i ∈ {1,…,N}g(i) = 0for j ∈ {1,…,N}

if j ≤ i: g(i) = g(i) + f(j)

for i ∈ {1,…,N}g(i) = 0for j ∈ {1,…,N}

if j ≤ i: g(i) = g(i) + f(j)

4 132 5

0 5 37 38 42 0 5 37 38 38 38 38 38 0 5 37 37 37 37 37 37 37 37 370 5 5 5 5 5 5 5 5 5 5 5 5

f g

time = # steps = # gates

f fζ

?


f fζ

?

v2 gates


f fζ

?

v2 gates v gates

Which lattices allow optimal circuits for the

zeta transform?

What?Why?How?

Yates’s Algorithm

[Frank Yates, The Design and analysis of factorial experiments, Imperial Bureau of Soil Science Tech. Comm. 35, 1937.]

“The Fast Zeta Transform”, FZT

The subset lattice

{}

{2}{1}

{1,2}

Circuits for Zeta Transform

f: L → R

f z(x) = Âyx

f (y) 4

3

16

4

14

510

1 1 1

0

00 0

0

0 1 1

0 0 0

0

1

1 1 1

0

11 0

0

1 1 1

1 1 0

0

11 1 1

0

00 0

0

0 1 1

0 0 0

0

1

1 1 1

0

11 0

0

1 1 1

1 1 0

0

11 1 1

0

00 0

0

0 1 1

0 0 0

0

1

1 1 1

0

12 1

1

1 2 1

2 1 1

1

1

1 1 1

0

11 0

0

1 1 1

1 1 0

0

11 1 1

0

00 0

0

0 1 1

0 0 0

0

1

1 1 1

0

12 1

1

1 2 1

2 1 1

1

11 1 1

0

22 2

3

1 2 2

2 2 3

3

1

1 1 1

0

11 0

0

1 1 1

1 1 0

0

11 1 1

0

00 0

0

0 1 1

0 0 0

0

1

1 1 1

0

12 1

1

1 2 1

2 1 1

1

11 1 1

0

22 2

3

1 2 2

2 2 3

3

11 1 1

0

22 2

3

1 1 1

0

11 0

0

1 1 1

1 1 0

0

11 1 1

0

00 0

0

0 1 1

0 0 0

0

1

1 1 1

0

12 1

1

1 2 1

2 1 1

1

11 1 1

0

22 2

3

1 2 2

2 2 3

3

1

1 1 1

0

22 2

3

1 1 1

0

11 0

0

1 1 1

1 1 0

0

11 1 1

0

00 0

0

0 1 1

0 0 0

0

1

1 1 1

0

12 1

1

1 2 1

2 1 1

1

11 1 1

0

22 2

3

1 2 2

2 2 3

3

1

1 1 1

0

11 0

0

1 1 1

1 1 0

0

11 1 1

0

00 0

0

0 1 1

0 0 0

0

1

1 1 1

0

12 1

1

1 2 1

2 1 1

1

11 1 1

0

22 2

3

1 2 2

2 2 3

3

1

1 1 1

0

22 2

3

2 3 3

4 4 5

6

1

Fast Zeta Transform

AB

C

DG

0

1 1 1 1

0 0 0 0 1

BD

1 CD

0 0 0 0

0g0 = f

0

1 1 1 1

1 1 0 1 1 1

0 1 1 0

0g1

0

1 1 1 1

2 1 1 1 2 1

1 2 1 1

1g2

0

1 1 1 1

2 2 2 1 2 2

3 2 2 3

3g3

0

1 1 1 1

2 2 2 2 3 3

3 4 4 5

6g = g4

Fig. 7. Yates’s algorithm on the indicator function of the nonempty independent sets ofthe graph G. Arrows indicate how the value of gi(S) for i ⇤ S is computed by addinggi�1(S \ {i}) to the ‘previous’ value gi�1(S).

In particular, gn(S) = �R⇥S f (R). Correctness of (14) is established by a straight-forward but tedious induction argument for (15). The base case g0 = f is im-mediate. For the inductive step, adopt the notation S(i) for S ⌅ {i + 1, . . . , n}.Then the right hand side of (15) can be written as

�R⇥S

[S(i) = R(i)] f (R)

= �R⇥Si⇤R

[S(i) = R(i)] f (R) + �R⇥Si/⇤R

[S(i) = R(i)] f (R) .

If i /⇤ S then the first sum vanishes and the second sum simplifies to

�R⇥S

[S(i� 1) = R(i� 1)] f (R) = gi�1(S)

by induction. If i ⇤ S then we can rewrite both sums to

�R⇥S

[S(i� 1) = R(i� 1)] f (R) + �R⇥Si/⇤S

[S(i� 1) = R(i� 1)] f (R)

= gi�1(S) + gi�1(S \ {i})

by induction. Finally, by (14) the entire expression equals gi(S). �

Perspective. As before, our approach is basically dynamic programming for adecrease-and-conquer recurrence. The time and space requirements are within

time = # lines (“edges”) in Hasse diagram

e = v log v < v2

#edges#lattice points

(vertices)

Fast Zeta Transform for v = 24

a linear factor of the ones given in §2. However, the expression (14) is moregeneral and does not depend on the structure of independent sets. It appliesto any function f : 2N ⇤ R from subsets to a ring, extending the algorithm tomany other covering problems than graph colouring.

Yates’s algorithm has much in common with the fast Fourier transform. Wecan illustrate its operation using a butterfly-like network, see Fig. 8.

?AB

ABCACBC

ABCD

ADBD

ABDCDACDBCD

ABCD

11

1

1

1

1

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

g0 g1 g2 g3 g4

0122122312343456

Fig. 8. Yates’s algorithm for the zeta transform.

Here we used Yates’s algorithm to compute (13), but the method is moregeneral than that. For example, it computes the Möbius transform, see (17) below,and many others. A classical treatment of the algorithm appears in [13], recentapplications and modifications are in [7] and the forthcoming journal versionof [3].

11 Möbius inversion. Let f : 2N ⇤ {0, 1} be a function of subsets of N to{0, 1} (indeed, any ring would do). To connect to the graph colouring examplefrom §2, think of f as the indicator function of the nonempty independent setsin a graph. The zeta transform of f is the function ( f ⇥) : 2N ⇤ {0, 1} definedpoint-wise by

( f ⇥)(T) = �S⇥T

f (S) . (16)

The brackets around ( f ⇥) are usually omitted. The Möbius transform of f is thefunction ( f µ) : 2N ⇤ {0, 1} defined point-wise by

( f µ)(T) = �S⇥T

(�1)|T\S| f (S), (17)

What?

How?Why?

Subset lattice in time O(v⋅log v) [Yates]

What we know

v = |L| = # vertices in Hasse diagram

e = # “edges” in Hasse diagram

Every lattice in time O(v2) [Brute force]Most lattices not in time o(v3/2/log v) [Count]

Subset lattice in time O(e) [Yates]No lattice in time o(e) [Kennes]

Most lattices for the zeta transform need

v3/2/log v gates

There are exp(Θ(v3/2)) lattices on v points

1

0

(v/2)–1 “atoms”

(v/2)–1 “hyperatoms”


1

0



possible edges2Q(v2)


1

0



possible edges2Q(v2)

Doesn’t quite work (not a lattice, isomorphisms,…), but OK with more care.

[Klotz und Lucht, Endliche Verbände, J. Reine Angew. Math., 247:58–68, 1971]

There are exp(O(s log s)) circuits on s gates

gate input 1 input 2

1 v1 v22 v3 v4

3 v5 v6

4 v7 v8

5 g1 g2

6 v2 v4

7 g3 g4

8 v6 v8

9 g5 g7

10 g6 g8

11 g2 g4

12 v4 v8

output

g9g10

g11

g12

g7

g8

g4

v8

O(log s) bits

There are exp(O(s log s)) circuits on s gates


Thus, most lattices need exp(Θ(v3/2/log v)) gates

Monotone circuits for the zeta transform need

O(e) gates

f z(x) = Âyx

f (y)

Need to argue: some gate handles this contribution only.

f z(x) = Âyx

f (y)

Need to argue: some gate handles this contribution only.

f z(x) = Âyx

f (y)

[Kennes, Computational aspects of the Möbius transform of graphs, IEEE Transactions on Systems, Man and Cybernetics, 22(2):201–223, 1992.]

Subset lattice in time O(v⋅log v) [Yates]

What we know

v = |L| = # vertices in Hasse diagram

e = # “edges” in Hasse diagram

Every lattice in time O(v2) [Brute force]Most lattices not in time o(v3/2/log v) [Count]

Subset lattice in time O(e) [Yates]No lattice in time o(e) [Kennes]

What?

No, really: Why?How?

Why?

Invitation to Algorithmic Uses ofInclusion–Exclusion

Thore Husfeldt

IT University of Copenhagen, DenmarkLund University, Sweden

Abstract. I give an introduction to algorithmic uses of the principle ofinclusion–exclusion. The presentation is intended to be be concrete andaccessible, at the expense of generality and comprehensiveness.

R

T1 The principle of inclusion–exclusion. There are asmany odd-sized as even-sized subsets sandwiched be-tween two different sets: For R ⇤ T,

�R⇤S⇤T

(�1)|T\S| = [R = T] . (1)

We use Iverson notation [P] for proposition P, mean-ing [P] = 1 if P and [P] = 0 otherwise.

Proof of (1). If R = T then there is exactly one sandwiched set, namely S = T.Otherwise we set up a bijection between the odd- and even-sized subsets asfollows. Fix t ⌅ T \ R. For every odd-sized subset S1 with R ⇤ S1 ⇤ T letS0 = S1 ⇥ {t} denote the symmetric difference of S1 with {t}. Note that thesize of S0 is even and that S0 contains R. Furthermore, S1 can be recovered fromS0 as S1 = S0 ⇥ {t}. ⌦

Perspective. We will see the (perhaps more familiar) formulation of the principleof inclusion–exclusion in terms of intersecting sets in §6, and another equivalentformulation in §11.

AB

C

D

I1

I2

I32 Graph colouring. A k-colouring of a graph G = (N, E) onn = |N| nodes assigns one of k colours to every node such thatneighbouring nodes have different colours. In any such colour-ing, the nodes of the same colour form a nonempty independentset, a set of nodes none of which are neighbours.

Let g(S) denote the number of nonempty independent sub-sets in S ⇤ N. Then G can be k-coloured if and only if

�S⇤N

(�1)n�|S|�g(S)⇥k

> 0 . (2)

Base revision d124530. . ., Tue Jul 5 09:02:55 2011 +0200, Thore Husfeldt.

Graph Colouring

# colourings =# covers with independent sets

artistic freedom; should say “partitions”

0

1 1 1 1

0 0 0 0 1 1

0 0 0 0

0f

0

1 1 1 1

2 2 2 2 3 3

3 4 4 5

6f ⇥

0

1 1 1 1

4 4 4 4 9 9

9 16 16 25

36( f ⇥)2

0

1 1 1 1

8 8 8 8 27 27

27 64 64 125

216( f ⇥)3

0

1 1 1 1

2 2 2 2 7 7

0 2 2 6

0( f ⇥)2µ

0

1 1 1 1

6 6 6 6 25 25

6 24 24 66

18( f ⇥)3µ

⇥

µ

µ

x ⌅⇤ x2

x ⌅⇤ x3

Fig. 10. Covering by Möbius inversion for k = 2 and k = 3.

To see this, for every T ⇥ V, view g⇥(T) and�

f ⇥(T)⇥k as two different ways of

counting the number of ways so select k nonempty independent subsets of T.Now, by Möbius inversion (18), we have

g = ( f ⇥)kµ ,

which is the left hand side of (2). In fact, we can now rewrite and understandthe left hand side of (2) as

�S⇥N

(�1)|N\S|⇤

�R⇥S

f (R)⌅ k

µ ⇥

function in the original domainoperation in the transformed domain


Thore Husfeldt



R


�R⇤S⇤T

(�1)|T\S| = [R = T] . (1)




AB

C

D

I1

I2



�S⇤N

(�1)n�|S|�g(S)⇥k

> 0 . (2)


Graph 3-Colouring

g

0

1 1 1 1

0 0 0 0 1 1

0 0 0 0

0f

0

1 1 1 1

2 2 2 2 3 3

3 4 4 5

6f ⇥

0

1 1 1 1

4 4 4 4 9 9

9 16 16 25

36( f ⇥)2

0

1 1 1 1

8 8 8 8 27 27

27 64 64 125

216( f ⇥)3

0

1 1 1 1

2 2 2 2 7 7

0 2 2 6

0( f ⇥)2µ

0

1 1 1 1

6 6 6 6 25 25

6 24 24 66

18( f ⇥)3µ

⇥

µ

µ

x ⌅⇤ x2

x ⌅⇤ x3





g = ( f ⇥)kµ ,


�S⇥N

(�1)|N\S|⇤

�R⇥S

f (R)⌅ k

µ ⇥



Thore Husfeldt



R


�R⇤S⇤T

(�1)|T\S| = [R = T] . (1)




AB

C

D

I1

I2



�S⇤N

(�1)n�|S|�g(S)⇥k

> 0 . (2)


Graph 3-Colouring

f(S) = 1 iff S is independent

g(S) = # triplets of independent subsets

covering S

g

0

1 1 1 1

0 0 0 0 1 1

0 0 0 0

0f

0

1 1 1 1

2 2 2 2 3 3

3 4 4 5

6f ⇥

0

1 1 1 1

4 4 4 4 9 9

9 16 16 25

36( f ⇥)2

0

1 1 1 1

8 8 8 8 27 27

27 64 64 125

216( f ⇥)3

0

1 1 1 1

2 2 2 2 7 7

0 2 2 6

0( f ⇥)2µ

0

1 1 1 1

6 6 6 6 25 25

6 24 24 66

18( f ⇥)3µ

⇥

µ

µ

x ⌅⇤ x2

x ⌅⇤ x3





g = ( f ⇥)kµ ,


�S⇥N

(�1)|N\S|⇤

�R⇥S

f (R)⌅ k

µ ⇥



Thore Husfeldt



R


�R⇤S⇤T

(�1)|T\S| = [R = T] . (1)




AB

C

D

I1

I2



�S⇤N

(�1)n�|S|�g(S)⇥k

> 0 . (2)


Graph 3-Colouring

f(S) = 1 iff S is independent

g(S) = # triplets of independent subsets

covering S

g

f z

gz

0

1 1 1 1

0 0 0 0 1 1

0 0 0 0

0f

0

1 1 1 1

2 2 2 2 3 3

3 4 4 5

6f ⇥

0

1 1 1 1

4 4 4 4 9 9

9 16 16 25

36( f ⇥)2

0

1 1 1 1

8 8 8 8 27 27

27 64 64 125

216( f ⇥)3

0

1 1 1 1

2 2 2 2 7 7

0 2 2 6

0( f ⇥)2µ

0

1 1 1 1

6 6 6 6 25 25

6 24 24 66

18( f ⇥)3µ

⇥

µ

µ

x ⌅⇤ x2

x ⌅⇤ x3





g = ( f ⇥)kµ ,


�S⇥N

(�1)|N\S|⇤

�R⇥S

f (R)⌅ k

µ ⇥


Graph k-colouring in time O*(2|V|).Trivial algorithm: time O*(k|V|).

[Björklund, H., Koivisto, Set partitioning via inclusion-exclusion. SIAM J. Comput., 39(2):546–563, 2009.]



Exciting new trend in algorithms for hard problems



Theory of NP-hardness

1960 1980 2000




1960 1980 2000

Let’s give up




1960 1980 2000

Let’s give up Approximation algorithms

Let’s find the wrong answer real

fast!




1960 1980 2000



fast!Exponential time (exact) algorithms

Let’s find the right answer faster.




1960 1980 2000



fast!Exponential time (exact) algorithms

Let’s find the right answer faster.

French connection: Kratsch, Liedloff,

Thomassé, Paul, Havet, Escoffier, Paschos,…

What?

No, really: Why?How?

Why?

How fast can we multiply?

Multiply polynomials

(1x0+1x1+3x2) ⋅ (1x0+2x1) = 1x0+3x1+5x2+6x3

Multiply polynomials

(1x0+1x1+3x2) ⋅ (1x0+2x1) = 1x0+3x1+5x2+6x3

elements in semigroup S (monomials)

mapping f from S to some field K

multiplication on S

Multiplication in semigroups

(1x0+1x1+3x2) ⋅ (1x0+2x1) = 1x0+3x1+5x2+6x3

( f _ g)(c) = Âa,b2L : a_b=c

f (a)g(b)

For lattices: join product

semigroup S monomial lattice point

field K of scalars coefficient f,g: S → K

product in the semigroup

algebra

multiplication of polynomials join product

Algebraic view

Fourier meets Möbius

p

q

F(p)

F(q)

˙ F�1polynomial

multiplication

join productf

g

⇣

⇣

_ µ

Zeta circuits for lattices with

n nonzero irreducibles of size O(vn)

C’EST FRAIS!

Zeta circuits for lattices with

n nonzero irreducibles of size O(vn)

[Björklund, H. Kaski, Koivisto, Nederlof, Parviainen, SODA, last week]

C’EST FRAIS!

What?Why?How?

9 6 4

8

1

3 2

1218

36 24

72

5 nonzero irreducibles

Divisors of 72

“6” is reducible (join of 2 and 3)

9 6 4

8

1

3 2

1218

36 24

72

prime powers

9 6 4

8

1

3 2

1218

36 24

72

prime powers ground set A

vn = e = 2|A||A|

9 6 4

8

1

3 2

1218

36 24

72


vn = e = 2|A||A|

Also works optimally forsublattices of the subset lattice that are

downward closedintersection closedunion closed

9 6 4

8

1

3 2

1218

36 24

72


vn = e = 2|A||A|

Also works optimally forsublattices of the subset lattice that are

downward closedintersection closedunion closed

e = Q(v)

What?Why?How?

9 6 4

8

1

3 2

1218

36 24

72

The Spectrum Map

9 6 4

8

1

3 2

1218

36 24

72

{3,9} {2,3} {2,4}

{2,4,8}

Ø

{3} {2}

{2,3,4}{2,3,9}

{2,3,4,9} {2,3,4,8}

{2,3,4,8,9}

The Spectrum Map

9 6 4

8

1

3 2

1218

36 24

72

{3,9} {2,3} {2,4}

{2,4,8}

Ø

{3} {2}

{2,3,4}{2,3,9}

{2,3,4,9} {2,3,4,8}

{2,3,4,8,9}

The Spectrum Map

23

23 24 39

234239248

23482349

Arbitrary finite lattice can be embedded into the subset lattice 2[n].Result is closed under intersection.

n = # nonzero, join-irreducible elements.

23

23 24 39

234239248

23482349

The Spectrum Map

9 6 4

8

1

3 2

1218

36 24

72

23

23 24 39

234239248

23482349

What we can’t just do

Yates’s algorithm takes time 2nn.Exponential in # irreducibles

What we do: walks

S

T

Every pair S, T of elements is connected by a unique walk

23

23 24 39

234239248

23482349

392 2393 2394 23498 23499 2349

(Not necessarily walks in the Hasse diagram)

Ø

21

12

Ø

21

12Ø

1 {1}

2 {1,2}

Ø

1 {1}

2 {1,2}

From Ø to {1,2}


Ø

21

12

Ø

21

12Ø

1 {1}

2 {1,2}

Ø

1 {1}

2 {1,2}

From Ø to {1,2}

For i = 1,2,…,n walk from Bi–1 to Bi using

Bi = (Bi�1 [ {i})?


Ø

21

12

Ø

21

12Ø

1 {1}

2 {1,2}

Ø

1 {1}

2 {1,2}

From Ø to {1,2}


Bi = (Bi�1 [ {i})?

“bottom” above


Ø

21

12

Ø

21

12Ø

1 {1}

2 {1,2}

Ø

1 {1}

2 {1,2}

From Ø to {1,2}


Bi = (Bi�1 [ {i})? A? = minB◆AB2L

B

“bottom” above

23

23 24 39

234239248

23482349

Summary

Embed L in subset lattice using spectrum map.Define unique family of “bottom” walks.

Circuit sums over these.Time = # irreducibles ⋅ # elements = # covering pairs (Hasse edges)

Applications of FZToutside of exponential time algorithms and

Dempster–Schafer belief propagation?

Which other semigroups allow FZTs?

Faster? Lower bounds?

Killer application beyond graph colouring?

algorithms for lattice transforms and

Documents