Introduction: Efficient Algorithms for the Problem of Computing

Fibonocci Numbers

Prepared by

John Reif, Ph.D.

Analysis of Algorithms

Readings

•  Main Reading Selection: – CLR, Chapter 1

Goal

•  Devise Algorithms to solve problems on a TI calculator

•  assume each step (a multiplication, addition, or control branch) takes 1 µ sec = 10-6 sec on 16 bit words

Time Efficiency

Fibonocci Sequence

•  0, 1, 1, 2, 3, 5, 8, 13…

•  Recursive Definition •  Fn = if n < 1 then n

else Fn-1 + Fn-2

•  Problem •  Compute Fn for n=109 fast.

Fibonacci Growth •  Can show as n à ∞

•  Golden ratio

•  So Fn ~ .45 ∙ 2 .7n is .7n bit number grows exponentially!

nn

n 5)( ~ F−−− φφ

...62.12

51 =

+=φ

Obvious Method

•  Fn =

> .01 n2 steps > 1016 µ sec for n = 109

= 1010 sec ~ 317 years!

ncosts at least adds of 2

.7n .35n bit numbers2=

n if n < 1

Fn-1 + Fn-2 if n > 1

⎟⎠

⎞⎜⎝

⎛⎟⎠

⎞⎜⎝

⎛≥bits 61bitsn 35.adds

2n TotalCost

Wanted!

•  An efficient algorithm for Fn

•  Weapons: •  Special properties of computational

problems

= combinatorics (in this case)

Theorem:

•  Proof by Induction – Basis Step

•  Holds by definition for n=1 –  Inductive Step

•  Assume holds for some n>0

⎟⎟⎠

⎞⎜⎜⎝

⎛=⎟⎟

⎠

⎞⎜⎜⎝

⎛

−

+

1nn

n1nn

FFFF

1011

Inductive Step

⎟⎟⎠

⎞⎜⎜⎝

⎛=

⎟⎟⎠

⎞⎜⎜⎝

⎛ ++=

⎟⎟⎠

⎞⎜⎜⎝

⎛⎟⎟⎠

⎞⎜⎜⎝

⎛=⎟⎟

⎠

⎞⎜⎜⎝

⎛

+

++

+

−+

−

+

+

n1n

1n2n

n1n

1nnn1n

1nn

n1n1n

FFFF

FFFFFF

FFFF

0111

1011

Powering Trick

•  Fix M =

•  To compute Mn when n is a power of 2 For i=1 to log n do

1 1

1 0

( ) ( )logn2

1-i1-ii

222

222

M ... ,M,M M, gives

MMM ×=

General Case of Powering •  Decompose n = 2 j1 + 2 j2 + ... + 2 jk

as sum of powers of 2 •  Compute:

•  Example:

23= 24 + 22 + 21 + 20 =16+ 4+ 2+1=10111 2

Mn = M2ji

i=1,...,k∏

M23 = M16+4+2+1 =M244 +22+21+20

= M24

M22

M21

M20

Algorithm

⎟⎟⎠

⎞⎜⎜⎝

⎛=1011

M

(1) compute M, M2, ..., M2 logn

by power method

(2) if n = 2 j1 + 2 j2 +...+ 2 jk then Mn = M2 ji

i=1,...,k∏

(3) output Fn = upper right of Mn

Bounding Mults

•  = 2 log n matrix products on symmetric matrices of size 2 x 2

•  Each matrix product costs 6 integer mults

•  Total Cost = 12 (log n) integer mults > 360 for n = 109 integer mults

Time Analysis

•  Fn ~ .45 2 .7n

•  So Fn is m = .7n bit integer

•  New method to compute Fn requires multiplying m bit number

•  BUT •  Grammar School Method for Mult

takes m2 bool ops = m2/16 steps ~ 1016 steps for n = 109 ~ 1016 µsec ~ 1010 seconds ~ 317 years!

Fast Multiplication of m bit integers a,b by •  “Divide and Conquer”

•  seems to take 4 multiplications, but…

Rk

L

Rk

L

b2bb

a2aa

+⋅=

+⋅=

aL aR

bL bR

RRk

LRRL2k

LL ba)2baba(2baba2m k where

+++=⋅

=

3 Mult Trick: Karatsubu Algorithm

LRRL

RLRL

RR

LL

baba y)(x)bb()aa( z )3(

bay )2(

ba x(1)

+=

+−+⋅+=

⋅=

⋅=

y2z2 x

ba2)bab(a2ba ba

bits-mon adds 6 and integersbit - 2mon

3only Requires

k2kRR

kLRRL

2kLL

+⋅+=

+++=⋅

mults

Recursive Mult Algorithm

•  Cost Cm bit ops C1 =1

Cm ≤ 3Cm2

+6m for m >1

≤32

⎛

⎝⎜⎞

⎠⎟

i=0

log m

∑i

6m

≤ 2 32

⎛

⎝⎜⎞

⎠⎟

log m

6m

≤12mlog2 3

~ 12m1.4 bit ops

~ 1013 bit ops if m = .7 109

~1016

13

steps

~.05 years

Schonhage-Strassen Integer Multiplication Algorithm

•  This is feasible to compute!

12 9

10

4

10m log m log log m bit ops~ .7 10 bit ops for m .7 10~ 4.3 10 steps ~4.3 10 sec. ~3 days

⋅

⋅ = ⋅

⋅

⋅

Improved Algorithm

•  Computed Fn for n = 109 using 360 = 12 log n integer mults each taking ~ 3 days

•  Total time to compute Fn •  ~ 3 years with 1 µsec/step

How Did We Get Even More Speed-Up?

1)  New trick •  a 2 log n mult algorithm

(rather than n adds) 2)  Old Trick

•  use known efficient algorithms for integer multiplication (rather than Grammar School mult)

3)  Also used careful analysis of efficiency of algorithms to compare methods

Problem (to be pondered later…)

•  How fast can 109 integers be sorted?

•  What algorithms would be used?

Introduction: Efficient Algorithms for the Problem of Computing

Fibonocci Numbers

Prepared by

John Reif, Ph.D.

Analysis of Algorithms