efficient algorithms lecture 1

Efficient AlgorithmsLecture 1

Contents of today’s lecture:• Introduction to the lecturers• The contents of the course• The process from Problem to Algorithm• Algorithm Examples• Introduction to efficency of algorithms

• Lecture: 2 x 35 minutes, followed by exercises

Lecturers• Assistant Professor Jens Myrup Pedersen

• NetSec, Aalborg University• jens@es.aau.dk

Associate Professor Volker Krüger• Media, Aalborg University Copenhagen• vok@cvmi.aau.dk

Course contents• Introduction to Analysis and Design of Algorithms• Recursive Algorithms and Recurrences• Greedy algorithms• Backtracking• Counting, Probabilities • Randomized algorithms• Sorting algorithms: Heap sort, quick sort• Hashing, hash tables• Binary search trees, red-black trees• Graph and graph algorithms• Examples from security and network planning points of view

Litterature etc.• Homepage: www.control.aau.dk/~jens/teaching/EA_2009

• Updated info on lecture contents and proposals for litterature • Lecture notes, ppt’s, additional material

• Book:• Cormen et al., Introduction to Algorithms, 2nd edition, MIT Press, 2003.• We suggest that you also look for additional litterature yourself.

From problem to algorithm• What is an algorithm – how would you define it?• What is required from an algorithm?

• Correctness – Efficiency• Consider the whole problem to be solved – usually a combination of algorithms and data structures are required for efficient solutions.

• Examples of algorithms (blackboard)• Integer sorting (Insertion sort, Merge sort)• Travelling salesman• Towers of Hanoi

Insertion sort

Merge sort

Analysis of an algorithm• Example: Insertion sort.• Two important properties: Correctness and Efficiency.

Correctness

• To verify correctness we need to get hold on loop invariants....• But first: How to verify intuitively that it holds?

Loop invarianter• Similar to induction – as you probably know from the Math lectures.

• Loop invariant: Describes the properties, for example of a variable, an array, etc. Loop invariants must fulfill 3 properties:

• Initialisation: It must be true before the first iteration of the loop.

• Maintenance: If it is true before an iteration – it is also true afterwards.

• Termination: When the loop terminates, the invariant must have a useful properti that helps showing the correctness of the algorithm.

• Example from previous slide: Insertion sort.

Efficiency of an algorithm

• How can we use these calculations – and what is the price of a ”step”?

A few words on efficiency• We want efficient algorithms - BUT • We also want algorithms that are easy to understand and maintain.• We also want algorithms, which can be reused (for example in other programs).• There is an important trade-off between development times and efficiency.

• Conclusion: Algorithm-design is always dependend on how the algorithm is to be used.

Analysis of efficiency• Ultimate goal: Determining calculation time for the algorithms design (well, that is if we don’t look at usage of other ressources such as memory, network, other kinds of I/O). • Challenge: Even in this case the calculation time depends on factors such as hardware, OS, set of instructions – and inputs for the algorithm. • In any case we would like to make an estimate of runtimes.

• Usual method: Analyse the worst-case complexity using Big-O notation (asymptotic notation).

• This is often fine, but be aware that in some situations best-case or average-case analysis may be more informative.

Asymptotic notation• Big O-notation, the usual, fast and often fine method.• Variants: , O, - just a short explanation (next slide). The book is more formal.• Mini-exercise: How does Insertion sort behave?

• In the afternoon we go into more details with different kinds of algorithms, such as divide-and-conquer and merge sort.

Asymptotic notation (blackboard)• Big O-notation: Can be considered an upper limit for calculation time.

• Variant: Little o-notation (not asymptotically tight).

• Big Theta () – There exists both c1 and c2 which (given n0) makes upper and lower bounds for growth.

• Big Omega () – Only lower boundary.

ProblemsFrom the book: • 1.1-1, 1.1-2, 1.1-3, 1.1-4, 1.1-5, 1.2-2, 1.2-3, 3.1-2, 3.1-3.

Project related:1. Identify and list the most important algorithms in your project (as

well as you can at this stage)2. For each of these algorithms, discuss if they are critical with

respect to design - and what are the critical factors? (e.g. time, correctness, computational power, memory/storage).

3. Select one or two of the algorithms, and try to make an estimate of complexity using Big O-notation.

4. Discuss what data structures could be used..