Computability

Start complexity. Motivation by thinking about sorting.

Homework: Finish examples.

Introduction

• Say you have a way to compute a problem– There exists a TM decider.

• But how effective / efficient / good is the algorithm?– maybe it takes a really long time….– maybe it takes a lot of space…

– Other measures to be explored later.

Introduction, cont.

• Challenge is to measure the time or the space or some other resource to solve problems– To use a particular approach/algorithm to solving

problems

• Usual procedure is to produce the answer as an expression in terms of the size of the problem and

• Put the answer as a limit…• What happens as the problem gets bigger?

Note

• It is possible to talk about complexity for a problem that doesn't have a size, but generally it is done for problems such as– sorting– searching– finding best X … such as shortest route for a traveling

salesman – encoding and decoding codes– encoding and decoding images– others

Time measures

• Sometimes restrict / focus on just one operation– compares for sorts or searches

• Upper bounds (will give formal definition)

• Total maximum number of steps.

• Worst case

• Average case: average over all inputs.

Sorting

• How would you sort a list of N numbers, ending up with lowest to highest?

Ways of evaluating sorts

• Time taken (sometimes just focus on compares and swaps)

• Space• stable or not: if two values in the set are equal,

and one before the other, after the sort, they will remain the first before the second– This is important for sorting records on multiple keys

• adaptive: takes less time if list already nearly in order

Bubble sort idea• Start at beginning. Compare value to its

neighbor. If >, swap. Go on to next position. Keep going to the end. This is one pass– Effect of first pass is to bring highest value to the top.

• Do passes. Don't have to go to the end—can stop earlier and earlier.

• Only do another pass if a swap took place.• Values bubble up.

Bubble sort

for (e = n;e>1;e--) { swapped = false; for (j = 1;j<=(e-1);j++)

if (a[j] < a[j+1]) { swap (j,j+1) ; //separate function swapped = true ;}

// invariant: a[1..i] in final position if (!swapped) break;}

Bubble sort

• Could take up to n * (n-1) comparisons. This will be termed O(n2) when we get to defining it.

• Takes only a small amount of extra space: and this is not dependent on size of data.

• stable

• adaptive

Merge idea• Pass

– divide array in half– sort each half– combine two parts in order

• Combining two sorted sets is easier (quicker) than sorting a set.

• Recursive• Claim: combine easier (smaller) sorting tasks with the

easier merging task makes for a quicker algorithm.– Note: every item is not compared with every other item.

Conditional operatorRecall 3 operand operator x = (condition) ? y : z;

if (condition) then y else zRecall j++ returns the current value of j and THEN

increments j.

In merge, to merge two sorted parts:• a[k++] = (a[j] < b[i]) ? a[j++] : b[i++];• This decides which is less, if it is the jth element in a,

assigns it to be the next and increments j. Otherwise, assigns the ith element from b and increments i.

Merge sort//a is an array, sort(a, 1, m) sorts the array 1 through m// split in halfm = n / 2;// recursive sortssort (a,1,m);sort (a,m+1,n);// merge sorted sub-arrays using temp arrayb = copy (a,1,m);i = 1; j = m+1; k = 1;while(( i <= m) &&( j <= n)) {

a[k++] = (a[j] < b[i]) ? a[j++] : b[i++]; } // invariant: a[1..k] in final position

while (i <= m) { //any leftovers in first parta[k++] = b[i++] }

// invariant: a[1..k] in final position

Merge sort

• Does fewer compares, bounded by n*ln(n).– The merge operation is more efficient.– Not comparing everything to everything else.

• Requires extra space: bounded by n.

• stable

• NOT adaptive: always does the same thing even if original list sorted.

There is a difference!

• Visualization:

• http://www.sorting-algorithms.com/

Other sorts

• insertion

• heap

• quicksort

• ?

Homework

Preparation for next time

• Examine other sorts.

• Report back.