winograd algorithm

WINOGRAD ALGORITHM FOR FAST CONVOLUTION

04/08/23 1

Overview• Introduction• Chinese Remainder Theorem• CRT for Integers• CRT for Polynomials• Winograd Algorithm• Example• Modified Winograd Algorithm• Advantages• Applications• References

04/08/23 2

Introduction

• Fast Convolution: implementation of convolution algorithm

using fewer multiplication

• Algorithmic Strength Reduction: Number of strong

operations is reduced at the expense of an increase in the

number of weak operations.

• Example: Reducing the multiplication complexity in complex

number multiplication:

– Assume (a+jb)(c+dj)=e+jf which requires 4 multiplications

and 2 additions04/08/23 3

Introduction

•the number of multiplications can be reduced to 3 at the expense of 3 extra additions by using

04/08/23 4

Chinese Remainder Theorem(CRT)

• Winograd algorithm -Based on the CRT .

• It’s possible to uniquely determine a nonnegative

integer given only its remainder with respect to the

given moduli, provided that the moduli are relatively

prime and the integer is known to be smaller than the

product of the moduli

04/08/23 5

CRT for Integers

6

• Given ci= [c]

•Then c=( NiMi) modM

Where

04/08/23

CRT for Polynomials

Given

Then

where

04/08/23 7

Winograd Algorithm

• Consider the computation s(p) =h(p) x(p) mod m(p)

• Choose a polynomial m(p) with degree higher than the degree of h(p) x(p) and

factor it into k+1 relatively prime polynomials ie m(p)= m(0)(p) m(1)(p)........... m(k)(p)

• Let M(i)(p)= m(p)/m(i)(p).Use the Euclidean GCD algorithm to solve for N(i)(p)

• Compute: h(i)(p) = h(p) mod m(i)(p)

x(i)(p) = x(p) mod m(i)(p)

04/08/23 8

Winograd Algorithm

• Compute s(i)(p) = h(i)(p) x(i)(p)mod m(i)(p)

• Compute s(p) = (p) N(i)(p) M(i)(p)mod m(p)

• The number of multiplications in Winograd algorithm is

highly dependent on the degree of each m(p) .

• Degree of m(p) should be as small as possible.

04/08/23 9

Example• Consider a 2X3 linear convolution.Construct an efficient

realization using Winograd algorithm m(p)= p(p-1)(p2-1)

h(p)= h0+h1p ,x(p)= x0+x1p+x2p2

04/08/23 10

Example

04/08/23 11

Example

04/08/23 12

Example

• Requires 5 multiplications and 11 additions compared with 6 multiplications and 2 additions for direct implementation

04/08/23 13

Modified Winograd Algorithm

• More efficient formof the Winograd algorithm

• It can be obtained by letting deg[m(p)]=deg[s(p)]

• And applying the CRT to

s’(p) =s(p) - hN-1 xL-1 m(p)

04/08/23 14

Modified Winograd Algorithm

• Choose a polynomial m(p) with degree higher than the degree of h(p) x(p) and factor it into k+1 relatively prime polynomials with real

coefficients.ie m(p)= m(0)(p) m(1)(p)........... m(k)(p)

• Let M(i)(p)= m(p)/m(i)(p).Use the Euclidean GCD algorithm to solve N(i)(p)

• Compute: h(i)(p) = h(p) mod m(i)(p)

x(i)(p) = x(p) mod m(i)(p)

04/08/23 15

Modified Winograd Algorithm

• Compute s’(i)(p) = h(i)(p) x(i)(p)mod m(i)(p)

• Compute s’(p) = (p) N(i)(p) M(i)(p)mod m(p)

•Compute s(p) =s’(p)+ hN-1 xL-1 m(p)

•Conclusion: this algorithm requires 4 multiplications and 7

additions

1604/08/23

Advantages

• Fast Convolution with less number of multiplication.

• Less complexity.

04/08/23 17

Application

• Fast matrix multiplication.

• Fast Fourier Transforms

• Cryptography.

• Fast Discrete Fourier Transform

04/08/23 18

Conclusion

• Convolution algorithm can be implemented using

fewer multiplication.

• Reduced the multiplication complexity.

04/08/23 19

References

• Keshab K. Parhi,VLSI Digital Signal Processing Systems,2nd ed.,Chap 8,pp 237-244,Wiley India pvt.ltd,2008

• BRENT, Error analysis of algorithms for matrix multiplication and triangular decomposition using Winograd’s identity, 16,pp 145–156, Numerische Mathematik

• Demmel, J., Dongarra, J., Eijkhout, E., Fuentes, E., Petitet, E., Vuduc, V., Whaley, R., And Yelick, K. 2005. Self-Adapting linear algebra algorithms and software. Proceedings of the IEEE, special issue on ”Program Generation, Optimization, and Adaptation” 93, 2.

04/08/23 20

References

• C. K. Koc and B. Sunar, Low-complexity bit-parallel canonical and normal basis multipliers for a class of finite fields. IEEE Transactions on Computers, 47(3):353–356, March 1998.

• S. Winograd. Some bilinear forms whose multiplicative complexity depends on the field of constants. Mathematical Systems Theory, 10:pp169–180, 1977

04/08/23 21