introduction to algorithms 6.046j/18.401j/sma5503 lecture 14 prof. charles e. leiserson

34
Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

Upload: dale-walker

Post on 12-Jan-2016

238 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

Introduction to Algorithms 6.046J/18.401J/SMA5503

Lecture 14

Prof. Charles E. Leiserson

Page 2: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

2

How large should a hash table be?

Goal: Make the table as small as possible, but large enough so that it won’t overflow (or otherwise become inefficient).

Problem: What if we don’t know the proper size

in advance?

Solution: Dynamic tables.

IDEA: Whenever the table overflows, “grow” it

by allocating (via malloc or new) a new, larger

table. Move all items from the old table into the

new one, and free the storage for the old table.

Page 3: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

3

Example of a dynamic table

1. INSERT

2. INSERT

Page 4: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

4

Example of a dynamic table

1. INSERT

2. INSERT

Page 5: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

5

Example of a dynamic table

1. INSERT

2. INSERT

Page 6: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

6

Example of a dynamic table

1. INSERT

2. INSERT

3. INSERT

Page 7: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

7

Example of a dynamic table

1. INSERT

2. INSERT

3. INSERT

Page 8: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

8

Example of a dynamic table

1. INSERT

2. INSERT

3. INSERT

Page 9: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

9

Example of a dynamic table

1. INSERT

2. INSERT

3. INSERT

4. INSERT

Page 10: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

10

Example of a dynamic table

1. INSERT

2. INSERT

3. INSERT

4. INSERT

5. INSERT

Page 11: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

11

Example of a dynamic table

1. INSERT

2. INSERT

3. INSERT

4. INSERT

5. INSERT

Page 12: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

12

Example of a dynamic table1. INSERT

2. INSERT

3. INSERT

4. INSERT

Page 13: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

13

Example of a dynamic table1. INSERT

2. INSERT

3. INSERT

4. INSERT

5. INSERT

6. INSERT

7. INSERT

5. INSERT

Page 14: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

14

Worst-case analysisConsider a sequence of n insertions. The

worst-case time to execute one insertion is

Θ(n). Therefore, the worst-case time for n

insertions is n · Θ(n) = Θ(n2).

WRONG! In fact, the worst-case cost for

n insertions is only Θ(n) 《 Θ (n2).

Let’s see why.

Page 15: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

15

Tighter analysis

Let ci = the cost of the i th insertion

= if i – 1 is an exact power of 2,

1 otherwise.

Page 16: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

16

Tighter analysisLet ci = the cost of the i th insertion

= i if i – 1 is an exact power of 2,

1 otherwise.

Page 17: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

17

Cost of n insertions

Thus, the average cost of each dynamic-table operation is Θ(n)/n = Θ(1).

Tighter analysis (continued)

Page 18: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

18

Amortized analysisAn amortized analysis is any strategy for

analyzing a sequence of operations to

show that the average cost per operation is small, even though a single operation

within the sequence might be expensive.

Even though we’re taking averages, however,

probability is not involved!

• An amortized analysis guarantees the

average performance of each operation in

the worst case.

Page 19: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

19

Types of amortized analysesThree common amortization arguments:

• the aggregate method,

• the accounting method,

• the potential method.

We’ve just seen an aggregate analysis.

The aggregate method, though simple, lacks the

precision of the other two methods. In particular,

the accounting and potential methods allow a

specific amortized cost to be allocated to each

operation.

Page 20: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

20

Accounting method• Charge i th operation a fictitious amortized cost

, where $1 pays for 1 unit of work (i.e., time).

• This fee is consumed to perform the operation.

• Any amount not immediately consumed is stored

in the bank for use by subsequent operations.

• The bank balance must not go negative! We

must ensure that

for all n.

• Thus, the total amortized costs provide an upper

bound on the total true costs.

Page 21: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

21

Accounting analysis ofdynamic tables

Charge an amortized cost of = $3 for the i th

insertion.

• $1 pays for the immediate insertion.

• $2 is stored for later table doubling.

Example:

When

recent item, and $1 pays to move an old item.

Page 22: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

22

Accounting analysis ofdynamic tablesCharge an amortized cost of = $3 for the i th insertion.• $1 pays for the immediate insertion.• $2 is stored for later table doubling.When the table doubles, $1 pays to move arecent item, and $1 pays to move an old item.Example:

Page 23: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

23

Accounting analysis ofdynamic tables

Charge an amortized cost of = $3 for the i th

insertion.

• $1 pays for the immediate insertion.

• $2 is stored for later table doubling.

When the table doubles, $1 pays to move a

recent item, and $1 pays to move an old item.

Example:

Page 24: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

24

Accounting analysis(continued)

Key invariant: Bank balance never drops below 0.Thus, the sum of the amortized costs provides anupper bound on the sum of the true costs.

*Okay, so I lied. The first operation costs only $2, not $3.

Page 25: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

25

Potential methodIDEA: View the bank account as the potential

energy (à la physics) of the dynamic set.

Framework:

• Start with an initial data structure D0.

• Operation i transforms D i –1 to D i.

• The cost of operation i is c i .

• Define a potential function Φ : {D i } → RR,

such that Φ(D0 ) = 0 and Φ(D i ) ≧ 0 for all i.

• The amortized cost with respect to Φ is defined to be = c i + Φ(D i ) – Φ(D i –1).

Page 26: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

26

Understanding potentials

Page 27: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

27

The amortized costs boundthe true costs

The total amortized cost of n operations is

Summing both sides.

Page 28: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

28

The amortized costs boundthe true costsThe total amortized cost of n operations is

The series telescopes.

Page 29: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

29

The amortized costs boundthe true costs

The total amortized cost of n operations is

Page 30: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

30

Potential analysis of tabledoubling

Define the potential of the table after the ith

insertion by (Assume that

)

Note:

• Φ(D0 ) = 0,

• Φ(D i ) . 0 for all i.

Example:

Page 31: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

31

Calculation of amortized costsThe amortized cost of the i th insertion is

Page 32: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

32

Calculation (Case 1)Case 1: i – 1 is an exact power of 2.

Page 33: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

33

Calculation (Case 2)Case 2: i – 1 is not an exact power of 2.

Therefore, n insertions cost Θ(n) in the worst case.

Exercise: Fix the bug in this analysis to show that

the amortized cost of the first insertion is only 2.

Page 34: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson

Introduction to Algorithms -Day 26 L14

34

Conclusions• Amortized costs can provide a clean abstraction of

data-structure performance.• Any of the analysis methods can be used when an

amortized analysis is called for, but each method has some situations where it is arguably the simplest.

• Different schemes may work for assigning amortized costs in the accounting method, or potentials in the potential method, sometimes yielding radically different bounds.