algorithms and data structures - david vernon · (e.g.!avl!trees,!red_black!trees)!...
TRANSCRIPT
Algorithms and Data Structures 1 Copyright © D. Vernon 2014
Algorithms and Data Structures CS-CO-412
David Vernon Professor of Informatics
University of Skövde Sweden
[email protected] www.vernon.eu
Algorithms and Data Structures 2 Copyright © D. Vernon 2014
Course Outline
• Motivation & Preview – The importance of algorithms & data structures
• Complexity of algorithms – Performance of algorithms – Time and space tradeoff – Worst case and average case performance – Big O notation – Recurrence relationships – Analysis of complexity of iterative and recursive algorithms – Tractable and intractable algorithmic complexity
Algorithms and Data Structures 3 Copyright © D. Vernon 2014
Course Outline
• Simple searching algorithms
– Linear search – Binary search
• Simple sorting algorithms
– Bubblesort – Quicksort
• Abstract Data Types (ADTs)
Algorithms and Data Structures 4 Copyright © D. Vernon 2014
Course Outline
• Lists, stacks, and queues – ADT specification – Array implementation – Linked-list implementations
• Trees – Binary trees – Binary search trees – Depth-first traversals – Applications of trees (e.g. Huffman coding) – Height-balanced trees (e.g. AVL Trees, Red-Black Trees)
Algorithms and Data Structures 5 Copyright © D. Vernon 2014
Course Outline
• Priority queues – Priority queue data structure – Binary heap – Heapsort
• Graphs – Graph data structure – Breadth-first search traversal – Depth-first search traversal – Minimum spanning tree (e.g. Prim’s and Kruskal’s algorithms) – Shortest-path algorithms (e.g. Dijkstra’s and Floyd’s algorithms) – Topological sort
Algorithms and Data Structures 6 Copyright © D. Vernon 2014
Course Outline
• Algorithmic strategies
– Brute-force – Divide-and-conquer – Greedy – Dynamic programming – Combinatorial Search & Backtracking
• Combinatorics: subsets and permutations • All paths in a graphs
– Branch-and-bound
Algorithms and Data Structures 7 Copyright © D. Vernon 2014
Motivation & Preview
The Importance of Algorithms & Data Structures
Algorithms and Data Structures 8 Copyright © D. Vernon 2014
Muḥammad ibn Mūsā al-Khwārizmī محمد بن موسى اخلوارزمي
Born approximately 780, died between 835 and 850 Persian mathematician and astronomer
from the Khorasan province of present-day Uzbekistan
The word algorithm is derived from his name
Algorithms and Data Structures 9 Copyright © D. Vernon 2014
Algorithms and Data Structures 10 Copyright © D. Vernon 2014
Algorithms + Data Structures = Programs
Inventor of Pascal and Modula
Niklaus Wirth, 1976
Inventor of Pascal and Modula programming languages Winner of Turing Award 1984
1969
Algorithms and Data Structures 11 Copyright © D. Vernon 2014
Algorithms + Data Structures = Programs
Information Processing: Representation & Transformation
Input Output
Algorithms and Data Structures 12 Copyright © D. Vernon 2014
The So6ware Development Process
System Specifica=on
Algorithms & Representa=ons
Requirements Elicia=on
So6ware
Problem Iden=fica=on
Computational Modelling
Problem Modelling
Validation
Software Design
Coding Testing: Verification &
Evaluation
Waterfall Model Software development life-cycle
Algorithms and Data Structures 13 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Goal, logic, strategy, model
I/O representa)on, transforma)on algorithm
Physical realiza)on
Loose coupling
Loose coupling
Marr’s Hierarchy of Abstrac)on / Levels of Understanding Framework
Algorithms and Data Structures 14 Copyright © D. Vernon 2014
Marr’s Hierarchy of Abstrac)on / Levels of Understanding Framework
“Trying to understand percep)on by studying only neurons is like trying to understand bird flight by studying only feathers: it just cannot be done. In order to understand bird flight, we have to understand aerodynamics; only then do the structure of feathers and the different shapes of birds’ wings make sense”
Marr, D. Vision, Freeman, 1982.
Algorithms and Data Structures 15 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Sorting a List
Given a sequence of n keys a1, … , an
Find the permuta)on (reordering) such that ai ≤ aj 1 ≤ i, j ≤ n
Algorithms and Data Structures 16 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Sorting a List
Bubble Sort
Insertion Sort
Quick Sort
Merge Sort, …
Key point: different computational efficiency
Algorithms and Data Structures 17 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Sorting a List
Algorithms and Data Structures 18 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Sorting a List
Algorithms and Data Structures 19 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Fourier Transform
Algorithms and Data Structures 20 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Fourier Transform
DFT: Discrete Fourier Transform
FFT: Fast Fourier Transform
FFTW: Fasted Fourier Transform in the West
Key point: different computational efficiency
Algorithms and Data Structures 21 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Fourier Transform
Algorithms and Data Structures 22 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Fourier Transform
Algorithms and Data Structures 23 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Marr’s Levels of Understanding Framework updated 2012 by T. Poggio
Learning & Development Calibra)ng & improving the model
CS-‐AI-‐421 Ar)ficial Cogni)ve Systems CS-‐AI-‐422 Machine Learning
Algorithms and Data Structures 24 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Marr’s Levels of Understanding Framework updated 2012 by T. Poggio
Evolu)on
Learning & Development Calibra)ng & improving the model
Genera)ng new models
Algorithms and Data Structures 25 Copyright © D. Vernon 2014
Algorithms and Data-‐Structures Strategy
Time
Diffi
culty
Sample of the core ADS topics from 2-‐3 standard undergraduate Algorithms and Data-‐Structures courses
Gradually increase the slope of the learning curve
Algorithms and Data Structures 26 Copyright © D. Vernon 2014
Algorithms and Data-‐Structures Topics
Time
Diffi
culty
Complexity Searching Sorting ADTs Lists, Stacks, Queues
Binary search trees
Height-balanced trees
Priority queues
Binary heaps
Heapsort
Graphs
Greedy
Divide & Conquer
Backtracking
Branch & Bound
Sample of the core ADS topics from 2-‐3 standard undergraduate Algorithms and Data-‐Structures courses
Gradually increase the slope of the learning curve
Algorithms and Data Structures 27 Copyright © D. Vernon 2014
Complexity of algorithms:
• Performance of algorithms • Big O nota)on • Recurrence rela)onships • Analysis of complexity • Itera)ve and recursive algorithms • Tractable, intractable complexity
Algorithms and Data-‐Structures Topics
Time
Diffi
culty
Complexity
Algorithms and Data Structures 28 Copyright © D. Vernon 2014
Use Big O notation to determine an upper bound function g(n) that characterizes how the complexity grows with the size of the data set … O(g(n)) e.g. O(n log2n)
Algorithms and Data Structures 29 Copyright © D. Vernon 2014
Algorithms and Data Structures 30 Copyright © D. Vernon 2014
function/ n 10 20 50 100 300
n2 1/10,000 second
1/2,500 second
1/400 second
1/100 second
9/100 second
n5 1/10 second
3.2 seconds
5.2 minutes
2.8 hours
28.1 days
2n 1/1000 second
1 second
35.7 years
400 trillion centuries
a 75 digit-number of centuries
nn 2.8 hours
3.3 trillion years
a 70 digit-number of centuries
a 185 digit-number of centuries
a 728 digit-number of centuries
Expo
nent
ial
Poly
nom
ial
Algorithms and Data Structures 31 Copyright © D. Vernon 2014
Algorithms and Data Structures 32 Copyright © D. Vernon 2014
Simple searching algorithms
• linear search O(n) • binary search O(log2n)
Algorithms and Data-‐Structures Topics
Time
Diffi
culty
Complexity Searching
Algorithms and Data Structures 33 Copyright © D. Vernon 2014
A B D F G J K M O P R
Algorithms and Data Structures 34 Copyright © D. Vernon 2014
A B D F G J K M O P R
Algorithms and Data Structures 35 Copyright © D. Vernon 2014
A B D F G J K M O P R
Algorithms and Data Structures 36 Copyright © D. Vernon 2014
A B D F G J K M O P R
Algorithms and Data Structures 37 Copyright © D. Vernon 2014
Simple sor)ng algorithms:
• Bubblesort (Itera)ve O(n2) ) • Quicksort (Recursive O(n log2n) )
Algorithms and Data-‐Structures Topics
Time
Diffi
culty
Complexity Searching Sorting
Algorithms and Data Structures 38 Copyright © D. Vernon 2014
/* Quicksort: assume A[R] contains a sentinel */
void quicksort (int A[], int L, int R) { int i, j, pivot; if ( R > L) {
i = L; j = R; pivot = A[i]; do { while (A[i] <= pivot) i=i+1; while ((A[j] >= pivot) && (j>l)) j=j-1; if (i < j) { exchange(A[i],A[j]); /*between partitions*/ i = i+1; j = j-1; } } while (i <= j); exchange(A[L], A[j]); /* reposition pivot */ quicksort(A, L, j);
quicksort(A, i, R); /*includes sentinel*/ } }
Algorithms and Data Structures 39 Copyright © D. Vernon 2014
4 9 8 10 11 99
Algorithms and Data Structures 40 Copyright © D. Vernon 2014
Abstract Data Types (ADTs)
• Informa)on hiding • Encapsula)on • Data-‐hiding • Basis for object-‐orienta)on
Algorithms and Data-‐Structures Topics
Time
Diffi
culty
Complexity Searching Sorting ADTs
Algorithms and Data Structures 41 Copyright © D. Vernon 2014
David Parnas Advisory Board Innopolis University
Algorithms and Data-‐Structures Topics
Time
Diffi
culty
Complexity Searching Sorting ADTs
Algorithms and Data Structures 42 Copyright © D. Vernon 2014
Lists, Stacks, and Queues
• ADT specifica)on • Array implementa)on • Linked-‐list implementa)on
Algorithms and Data-‐Structures Topics
Time
Diffi
culty
Complexity Searching Sorting ADTs Lists, Stacks, Queues
Algorithms and Data Structures 43 Copyright © D. Vernon 2014
• Example of List manipula6on
w = Next(Last(L),L)
Insert(e,w,L)
w = Previous(w,L)
Delete(w,L)
Algorithms and Data Structures 44 Copyright © D. Vernon 2014
List
Header node
element pointer NULL pointer
Algorithms and Data Structures 45 Copyright © D. Vernon 2014
List
To delete a node at this window position we re-arrange the links and free the node
window
temp
Algorithms and Data Structures 46 Copyright © D. Vernon 2014
Algorithms and Data Structures 47 Copyright © D. Vernon 2014
Head Tail Head Tail
Algorithms and Data Structures 48 Copyright © D. Vernon 2014
Trees
• Binary trees • Binary search trees • Depth-‐first traversal • Breadth-‐first traversals • Applica)ons of trees
(e.g. Huffman coding) • Height-‐balanced trees
(e.g. AVL Trees, Red-‐Black Trees)
Algorithms and Data-‐Structures Topics
Time
Diffi
culty
Complexity Searching Sorting ADTs Lists, Stacks, Queues
Binary search trees
Height-balanced trees
Algorithms and Data Structures 49 Copyright © D. Vernon 2014
Algorithms and Data Structures 50 Copyright © D. Vernon 2014
Binary Search Tree
Sun
Mon Tue
Fri Sat Thur Wed
Algorithms and Data Structures 51 Copyright © D. Vernon 2014
4
5
6
1
2
Pointer Implementation
Algorithms and Data Structures 52 Copyright © D. Vernon 2014
×
× +
- - +
A B D E F G
C
Inorder Traversal
Algorithms and Data Structures 53 Copyright © D. Vernon 2014
+2
A
+1
B
BL BR
AR
0
B
0
A BL
BR AR
AVL Trees – Height Balancing
Algorithms and Data Structures 54 Copyright © D. Vernon 2014
AVL Trees - RL rotation
-‐2
A
-‐1
C
CL CR
BL
+1
B AL
+1
A
0
C
CL CR BL
0
B
AL
Algorithms and Data Structures 55 Copyright © D. Vernon 2014
Red-Black Height-balanced Trees
Algorithms and Data Structures 56 Copyright © D. Vernon 2014
Priority queues, heaps, graphs:
• Topological sort • Minimum spanning tree
(e.g. Prim’s and Kruskal’s) • Shortest-‐path algorithms
(e.g. Dijkstra’s & Floyd’s)
Algorithms and Data-‐Structures Topics
Time
Diffi
culty
Complexity Searching Sorting ADTs Lists, Stacks, Queues
Binary search trees
Height-balanced trees
Priority queues
Binary heaps
Heapsort
Graphs
Algorithms and Data Structures 57 Copyright © D. Vernon 2014
Adjacency matrix and list implementa)ons of a graph
Algorithms and Data Structures 58 Copyright © D. Vernon 2014
Algorithms and Data Structures 59 Copyright © D. Vernon 2014
Minimum Spanning Trees
Tree formed from subset of graph edges that connects all ver)ces and minimizes the total path length
Algorithms and Data Structures 60 Copyright © D. Vernon 2014
Minimum Spanning Tree Shortest Path from Centre Spanning Tree
Algorithms and Data Structures 61 Copyright © D. Vernon 2014
Algorithms and Data Structures 62 Copyright © D. Vernon 2014
Algorithmic strategies
• Brute-‐force • Divide-‐and-‐conquer • Greedy algorithms • Combinatorial Search &
Backtracking • Branch-‐and-‐bound
Algorithms and Data-‐Structures Topics
Time
Diffi
culty
Complexity Searching Sorting ADTs Lists, Stacks, Queues
Binary search trees
Height-balanced trees
Priority queues
Greedy
Divide & Conquer
Backtracking
Branch & Bound
Binary heaps
Heapsort
Graphs
Algorithms and Data Structures 63 Copyright © D. Vernon 2014
Combinatorial Search and Heuris)c Methods
Algorithms and Data Structures 64 Copyright © D. Vernon 2014
Typical Input for HCP Hamiltonian cycle for the graph Another Hamiltonian cycle for the same graph in
A Hamiltonian cycle for a given graph G=(V, E) consists on finding an ordering of the ver)ces of the graph G such that each vertex is visited exactly once
Algorithms and Data Structures 65 Copyright © D. Vernon 2014
Travelling Salesman Problem … or, a variant, the vehicle rou)ng problem
Algorithms and Data Structures 66 Copyright © D. Vernon 2014
See www.algorist.com for online resources
Algorithms and Data Structures 67 Copyright © D. Vernon 2014
• `
Algorithms and Data-Structures The foundation of all solutions to computational information processing problems
Often unseen, but always there hpp://www.wired.com/wiredscience/2011/08/iceberg-‐towing-‐drinking-‐water/
Algorithms and Data Structures 68 Copyright © D. Vernon 2014
Computa)onal Theory
Representa)on & Algorithm
Hardware/So<ware Implementa)on
Goal, logic, strategy, model
I/O representa)on, transforma)on algorithm
Physical realiza)on
Marr’s Hierarchy of Abstrac)on / Levels of Understanding Framework