2is80 fundamentals of informatics fall 2015 lecture 7: sorting and directed graphs

2IS80 Fundamentals of Informatics

Fall 2015

Lecture 7: Sorting and Directed Graphs

Sorting algorithmsInput: a sequence of n numbers ‹a1, a2, …, an›Output: a permutation of the input such that ‹ai1 ≤ … ≤ ain›

Important properties of sorting algorithms:

running time: how fast is the algorithm in the worst case

in place: only a constant number of input elements are ever stored outside the input array

Sorting algorithmsInput: a sequence of n numbers ‹a1, a2, …, an›Output: a permutation of the input such that ‹ai1 ≤ … ≤ ain›

Important properties of sorting algorithms:

running time: how fast is the algorithm in the worst case

in place: only a constant number of input elements are ever stored outside the input array

Θ(n2)yes

Θ(n log n) no

worst case running time in place

Selection Sort

Insertion Sort

Merge Sort

Θ(n2)yes

QuickSort

One more sorting algorithm …

worst case running time in place

Selection Sort Θ(n2) yes

Insertion Sort Θ(n2) yes

Merge Sort Θ(n log n) no

Quick Sort Θ(n2) yes

QuickSort

Why QuickSort?

1. Expected running time: Θ(n log n) (randomized QuickSort)

2. Constants hidden in Θ(n log n) are small

3. using linear time median finding to guarantee good pivot gives worst case Θ(n log n)

QuickSort QuickSort is a divide-and-conquer algorithm

To sort the subarray A[p..r]:

DividePartition A[p..r] into two subarrays A[p..q-1] and A[q+1..r], such that each element in A[p..q-1] is ≤ A[q] and A[q] is < each element in A[q+1..r].

ConquerSort the two subarrays by recursive calls to QuickSort

CombineNo work is needed to combine the subarrays, since they are sorted in place.

Divide using a procedure Partition which returns q.

QuickSortQuickSort(A, p, r) 1. if p < r2. then q = Partition(A, p, r)3. QuickSort(A, p, q-1)4. QuickSort(A, q+1, r)

Partition(A, p, r)5. x = A[r]6. i = p-17. for j = p to r-18. do if A[j] ≤ x9. then i = i+110. exchange A[i] ↔ A[j]11. exchange A[i+1] ↔ A[r]12. return i+1

Initial call: QuickSort(A, 1, n)

Partition always selects A[r] as the pivot (the element around which to partition)

Partition As Partition executes, the array

is partitioned into four regions (some may be empty)

Loop invariant1. all entries in A[p..i] are ≤ pivot2. all entries in A[i+1..j-1] are > pivot3. A[r] = pivot

Partition(A, p, r)1. x = A[r]2. i = p-13. for j = p to r-14. do if A[j] ≤ x5. then i = i+16. exchange A[i] ↔ A[j]7. exchange A[i+1] ↔ A[r]8. return i+1

x

p i j r

≤ x > x ???

Partition

xp i j r

≤ x > x ???

Partition(A, p, r)1. x = A[r]2. i = p-13. for j = p to r-14. do if A[j] ≤ x5. then i = i+16. exchange A[i] ↔ A[j]7. exchange A[i+1] ↔ A[r]8. return i+18 1 6 4 0 3 9 5

pi j r

8 1 6 4 0 3 9 5pi j r

1 8 6 4 0 3 9 5pi j r

1 4 6 8 0 3 9 5p i j r

1 8 6 4 0 3 9 5pi j r

1 4 0 8 6 3 9 5p i j r

1 4 0 3 6 8 9 5p i j r

1 4 0 3 6 8 9 5p i r

1 4 0 3 5 8 9 6p i r


Initializationbefore the loop starts, all conditions are satisfied, since r is the pivot and the two subarrays A[p..i] and A[i+1..j-1] are empty

Maintenancewhile the loop is running, if A[j] ≤ pivot, then A[j] and A[i+1] are swapped and then i and j are incremented ➨ 1. and 2. hold.If A[j] > pivot, then increment only j ➨ 1. and 2. hold.

Partition - CorrectnessPartition(A, p, r)1. x = A[r]2. i = p-13. for j = p to r-14. do if A[j] ≤ x5. then i = i+16. exchange A[i] ↔ A[j]7. exchange A[i+1] ↔ A[r]8. return i+1

xp i j

≤ x > x ???

r


Terminationwhen the loop terminates, j = r, so all elements in A are partitioned into one of three cases:

A[p..i] ≤ pivot, A[i+1..r-1] > pivot, and A[r] = pivot

Lines 7 and 8 move the pivot between the two subarrays

Running time:

Partition - CorrectnessPartition(A, p, r)1. x = A[r]2. i = p-13. for j = p to r-14. do if A[j] ≤ x5. then i = i+16. exchange A[i] ↔ A[j]7. exchange A[i+1] ↔ A[r]8. return i+1

xp i j

≤ x > x ???

r

Θ(n) for an n-element subarray

QuickSort: running timeQuickSort(A, p, r) 1. if p < r2. then q = Partition(A, p, r)3. QuickSort(A, p, q-1)4. QuickSort(A, q+1, r)

Running time depends on partitioning of subarrays: if they are balanced, then QuickSort is as fast as MergeSort if they are unbalanced, then QuickSort can be as slow as

InsertionSort

Worst case

subarrays completely unbalanced: 0 elements in one, n-1 in the other

T(n) = T(n-1) + T(0) + Θ(n) = T(n-1) + Θ(n) = Θ(n2) input: sorted array

QuickSort: running timeQuickSort(A, p, r) 1. if p < r2. then q = Partition(A, p, r)3. QuickSort(A, p, q-1)4. QuickSort(A, q+1, r)

Running time depends on partitioning of subarrays: if they are balanced, then QuickSort is as fast as MergeSort if they are unbalanced, then QuickSort can be as slow as

InsertionSort

Best case

subarrays completely balanced: each has ≤ n/2 elements T(n) = 2T(n/2) + Θ(n) = Θ(n log n)

Average? Much closer to best case than worst case …

Randomized QuickSort pick pivot at random

RandomizedPartition(A, p, r) 1. i = Random(p, r)2. exchange A[r] ↔A[i]3. return Partition(A, p, r)

random pivot results in reasonably balanced split on average ➨ expected running time Θ(n log n)

alternative: use linear time median finding to find a good pivot ➨ worst case running time Θ(n log n)

price to pay: added complexity

Graphs

Networks and other graphsroad network computer network

execution order for processes

process 1

process 2process 3

process 5

process 4process 6

Graphs: Basic definitions and terminology A graph G is a pair G = (V, E)

V is the set of nodes or vertices of G E ⊂ VxV is the set of edges or arcs of G

If (u, v) E then vertex v is ∈ adjacent to vertex u

directed graph (u, v) is an ordered pair

➨ (u, v) ≠ (v, u) self-loops possible

undirected graph (u, v) is an unordered pair

➨ (u, v) = (v, u) self-loops forbidden

Some special graphsTree

connected, undirected, acyclic graph

Every tree with n vertices has exactly n-1 edges

DAGdirected, acyclic graph

in-degreenumber of incoming edges

out-degreenumber of outgoing edges

every DAG has a vertex with in-degree 0 and with out-degree 0 …

Topological sort

Topological sortInput: directed, acyclic graph (DAG) G = (V, E)Output: a linear ordering of v1 ,v2 ,…, vn of the vertices such that

if (vi ,vj ) ∈ E then i < j

DAGs are useful for modeling processes and structures that have a partial order

Partial order a > b and b > c ➨ a > c but may have a and b such that neither a > b nor b > a

execution order for processes

process 1

process 2process 3

process 5

process 4process 6

Topological sortInput: directed, acyclic graph (DAG) G = (V, E)Output: a linear ordering of v1 ,v2 ,…, vn of the vertices such that


DAGs are useful for modeling processes and structures that have a partial order

Partial order a > b and b > c ➨ a > c but may have a and b such that neither a > b nor b > a

a partial order can always be turned into a total order(either a > b or b > a for all a ≠ b)

(that’s what a topological sort does …)

Input: directed, acyclic graph (DAG) G = (V, E)Output: a linear ordering of v1 ,v2 ,…, vn of the vertices such that


Every directed, acyclic graph has a topological order

Topological sort

v6v5v2 v3 v4v1

Topological sortunderwear

pants

belt

socks

shoes

shirt

tie

jacket

watch

watch socks underwear pants shoes shirt tie belt jacket


pants

belt

socks

shoes

shirt

tie

jacket

watch

watchsocksunderwear pants shoes shirt tie belt jacket


pants

belt

socks

shoes

shirt

tie

jacket

watch

watchsocksunderwear pants shoes shirt tiebelt jacket

Topological sortTopological-Sort(G)

Input: a DAG G with vertices 1 .. nOutput: A linear order of the vertices, such that u appears before v if (u, v) in G

1. Let in-degree[1..n] be a new array2. Set all values in in-degree to 03. For each vertex u

A. For each vertex v adjacent to u:i. Increment in-degree[v]

4. Make a list next consisting of all vertices u such that in-degree[u] = 05. While next is not empty do the following:

A. Delete a vertex from next and call it uB. Add u to the end of the linear orderC. For each vertex v adjacent to u:

i. Decrement in-degree[v]ii. If in-degree[v] = 0 then insert v into next

6. Return the linear order

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

1 2 3 4 5 6 7 8 9





4. Make a list next consisting of all vertices u such that in-degree[u] = 0

5. While next is not empty do the following:A. Delete a vertex from next and call it uB. Add u to the end of the linear orderC. For each vertex v adjacent to u:

i. Decrement in-degree[v]ii. If in-degree[v] = 0 then insert v

into next6. Return the linear order

1 2

3 4

6

5

7

8

9

in-degree

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 0 0 0 0 0 0 0

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 1 0 1 0 0 0 0

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 2 0 1 0 0 0 0

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 3 0 1 1 0 0 0

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 3 0 1 1 0 0 1

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 3 0 1 2 0 1 1

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 3 0 1 2 0 1 2

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 3 0 1 2 0 1 2

1 2 4 7

7

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 3 0 1 1 0 0 2

1 2 4 7

7

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 3 0 1 1 0 0 2

1 2 4 8

7 8

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 3 0 1 1 0 0 1

1 2 4 8

7 8

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 3 0 1 1 0 0 1

1 2 4

7 8 4 2

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 2 0 1 1 0 0 1

1 2

7 8 4 2

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 2 0 1 1 0 0 1

1

7 8 4 2 1

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 1 0 0 1 0 0 1

1

7 8 4 2 1

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 1 0 0 1 0 0 1

5

7 8 4 2 1 5

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 0 0 0 0 0 0 1

5

7 8 4 2 1 5

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 0 0 0 0 0 0 1

3 6

7 8 4 2 1 5 6

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1 2

3 4

6

5

7

8

9

in-degree 0 0 0 0 0 0 0 0 0

3 6

7 8 4 2 1 5 6

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

1 2 3 4 5 6 7 8 9









1

in-degree 0 0 0 0 0 0 0 0 0

3 9

7 8 4 2 1 5 6 9 3

2

3 4

6

5

7

8

9

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order


pants

belt

socks

shoes

shirt

tie

jacket

watch

watch socks underwear pants shoesshirt tie belt jacket

1 2 3 4 5 6 7 8 9









1

in-degree 0 0 0 0 0 0 0 0 0

7 8 4 2 1 5 6 9 3

2

3 4

6

5

7

8

9

underwear

pants

belt

socks

shoes

shirt

tie

jacket

watch1 2

3 4

6

5

7

8

9

next

linear order

Running time?

Graph representationGraph G = (V, E)

1. Adjacency listsarray Adj of |V| lists, one per vertex

Adj[u] = linked list of all vertices v with (u, v) E ∈

(works for both directed and undirected graphs)

54 3

21

3

3

1 4

1

5

4

3

2

2

2

3

1

Graph G = (V, E)


Adj[u] = linked list of all vertices v with (u, v) E ∈

(works for both directed and undirected graphs)

Graph representation

54 3

21

1

1

5

4

3

2

2

2

3

Graph G = (V, E)


Adj[u] = linked list of all vertices v with (u, v) E∈

2. Adjacency matrix |V| x |V| matrix A = (aij)

(also works for both directed and undirected graphs)


aij = 1 if (i, j) E∈0 otherwise

12345

31 2 4 51

11

11

1 11

54 3

21

Graph G = (V, E)


Adj[u] = linked list of all vertices v with (u, v) E∈

2. Adjacency matrix |V| x |V| matrix A = (aij)

(also works for both directed and undirected graphs)


aij = 1 if (i, j) E∈0 otherwise

12345

31 2 4 51

1

1 1

54 3

21

Adjacency lists vs. adjacency matrix3

3

1 4

1

5

4

3

2

2

2

3

112345

31 2 4 51

11

11

1 11

Adjacency lists Adjacency matrix

Space

Timeto list all vertices adjacent to u

Timeto check if (u, v) E∈

Θ(V + E) Θ(V2)

Θ(degree(u)) Θ(V)

Θ(1)Θ(degree(u))better if the graph is sparse … use V for |V| and E for |E|







i. Decrement in-degree[v]ii. If in-degree[v] = 0 then insert

v into next6. Return the linear order

Running time?

DAG G represented in adjacency list

next is a linked list G has n vertices and m edges

1. Θ(1)2. Θ(n)3. Θ(n+m)4. Θ(n)5. Θ(n+m)6. Θ(1)

Θ(n+m)

2is80 fundamentals of informatics fall 2015 lecture 7: sorting and directed graphs

Documents