greedy algorithms. greedy algorithm greedy algorithm - makes locally optimal choice at each stage....

S

Greedy Algorithms

Greedy Algorithm

Greedy Algorithm

- Makes locally optimal choice at each stage.

- For optimization problems.

If the local optimum is a part of the global optimum, we get the global optimum.

Greedy Algorithm vs Dynamic Programming

DynamicProgramming

GreedyAlgorithm

Knapsack Problem

n items a thief’s knapsack of

size W

Knapsack Problem

0-1 knapsack problem

- Each item must be either taken or left behind.

Fractional knapsack problem

- The thief can take fractions of items.

Knapsack Problem

453

0201

0$60 $10

0$120

$135

knapsack($6/

unit)($5/unit)

($4/unit)

($3/unit)

n = 4,W = 50

50

Fractional Knapsack Problem

5045

30

20 1

0$60

$100

$120$13

5 knapsack

($6/unit)

($5/unit)

($4/unit)($3/

unit)

10

20

2030

Greedy algorithm:greatest value per unit

50

$240

0-1 Knapsack Problem

45

30

20

10

$60

$100

$120

$135 optima

l

($6/unit)

($5/unit)

($4/unit)

($3/unit)

50 50

value

50

valueper unit

50

10

20

45

30

20

$160

$135

$220

0-1 Knapsack Problem

optimal

50 50

value

50

valueper unit

50

10

20

45

30

20

$160

$135

$220

Difficult to get the optimal solution with a greedy strategy.

Dynamic Programming :

Optimal Substructure vsSubproblem Solution

48

5 7

4

1

3

2

T1,5 T1,6

T2,5 T2,6

Ti,j : the solution of a subproblem

A subproblem solution?

A local optimum?

Greedy Algorithm vs Dynamic Programming

Dynamic Programming

Greedy Algorithm

Computes all subproblems

Find a local optimum

Always finds the optimal solution

May not be able to find the optimal solution

Less efficient More efficient

Optimal Substructure vsSubproblem Solution

Subproblem solution

- From all subproblem solutions

Optimal substructure

- With only constant number of parameters

- Without subproblems or future choices

Usuallytop-down

UsuallyBottom-up

Huffman Codes

A lossless data compression algorithm.

It uses variable-length code

Variable-Length Code

Six characters : a, b, c, d, e, f

How can we represent them with binary strings?

a b c d e f

Fixed-length

000 001 010 011 100 101




a b c d e f

Variable-length

0 1 00 01 10 11

What 0010 menas?

0 0 1 0 = a a b a0 01 0 = a d a00 10 = c e…

Prefix Code



a b c d e f

Variable-length

0 101 100 111 1101 1100

No codeword is a prefix of another codeword.


a b c d e f Total

Frequency 45 13 12 16 9 5 100

Fixed-length 000 001 010 011 100 101 300

Variable-length

0 101 100 111 1101

1100

224

Is it the optimal way?


a b c d e f Total

Frequency 45 13 12 16 9 5 100

Fixed-length 000 001 010 011 100 101 300

Variable-length

0 101 100 111 1101

1100

224

Alternative? 0 1100

1101

10 1110

1111

231

Huffman Tree

100cha

rcode

freq

a 000 45

b 001 13

c 010 12

d 011 16

e 100 9

f 101 5

Fixed-length code

86 14

58 1428

a: 45

b: 13

c: 12

d: 16

e: 9

f: 5

0 1

0 1 0

0 1 0 1 0 1

Huffman Tree

char

code

freq

a 0 45

b 101 13

c 100 12

d 111 16

e 1101

9

f 1100

5

Variable-length code

100

55

25

14

30

a: 45

b: 13c: 12 d: 16

f: 5 e: 9

0 1

0 1

0 1

10

0 1

Huffman’s Algorithm

A

B

0

0

Every non-leaf node has two children.

B

0

Observation 1The longest code : at least 2 characters


C H

I J

G10

A10

F10

Observation 1

Observation 2

The longest code : at least 2 charactersThe longest 2 codes : the least frequent two characters

D E

B10


Observation 1

Observation 2

Observation 3

The longest code : at least 2 charactersThe longest 2 codes : the least frequent two charactersA non-leaf node : handled like a leaf node

d: 1614

30 x: 30T T

’


d: 1614

30 x: 30

A non-leaf node : handled like a leaf node

Observation 3

e: 9 f: 5

10

T T ’

10

Total length in T = Total length in T ‘ + 16 * 1 + (9 + 5) * 2


Observation 1

Observation 2

Observation 3

The longest code : at least 2 charactersThe longest 2 codes : the least frequent two charactersA non-leaf node : handled like a leaf node

d: 1614

30 x: 30T T

’


a: 45 b: 13 c: 12 d: 16 e: 9 f: 5

Merging two least frequent nodes.


a: 45 b: 13 c: 12 d: 16

e: 9 f: 5


1410


a: 45

c: 12 b: 13

d: 16

e: 9 f: 5


1410

2510


a: 45

c: 12 b: 13 d: 16

e: 9 f: 5


1410

2510

3010


a: 45

c: 12 b: 13 d: 16

e: 9 f: 5


1410

2510

3010

550 1


a: 45

c: 12 b: 13 d: 16

e: 9 f: 5


1410

2510

3010

550 1

1000 1 cha

rcode

a 0

b 101

c 100

d 111

e 1101

f 1100

Greedy Algorithm

Optimization Algorithms : finds a proper local optimum.

Not as powerful as Dynamic Programming, but simpler.

Greedy algorithms

- Knapsack Problem

- Huffman Code

S

Graphs

What is a graph?

1 2

3 4

5

V = { 1, 2, 3, 4, 5 }E = { {1,2}, {1,3}, {2,3}, {2,4}, {2,5}, {3,4} }

G = ( V, E )

Directed and Undirected

1 2

3 4

5

Directed graph

1 2

3 4

5

Undirected graph

Representations of Graphs

1 2

3 4

5

Directed graph

1

2

3

4

5

2 /

3

1

2

2 /

4 /

3 /

3 /

Adjacency-list


1 2

3 4

5

Directed graph Adjacency-Matrix1 2 3 4

5

5

3

4

2

1


Adjacency-Matrix

Adjacency-list

Adjacency List Adjacency Matrix

space |V| + |E| |V|2

Finding all edges

|V| + |E| |V|2

Finding one edge

num of edges 1

Adjacency Matrix of Graphs

1 2

3 4

5

Directed graph Adjacency-Matrix1 2 3 4

5

5

3

4

2

1

Adjacency Matrix of Graphs

A = AT =

Weighted Graphs

1 2

3 4

5

1.7

0.42.0

-0.3

3.1

-0.23.6-

2.1

greedy algorithms. greedy algorithm greedy algorithm - makes locally optimal choice at each stage....

Documents