19 algorithms-and-complexity-110627100203-phpapp02
Post on 25-May-2015
207 Views
Preview:
TRANSCRIPT
Algorithms Algorithms Complexity and Data Complexity and Data Structures EfficiencyStructures Efficiency
Computational Complexity, Choosing Computational Complexity, Choosing Data StructuresData Structures
Svetlin NakovSvetlin NakovTelerik Telerik
CorporationCorporationwww.telerik.com
Table of ContentsTable of Contents
1.1. Algorithms Complexity and Algorithms Complexity and Asymptotic NotationAsymptotic Notation Time and Memory ComplexityTime and Memory Complexity Mean, Average and Worst CaseMean, Average and Worst Case
2.2. Fundamental Data Structures – Fundamental Data Structures – ComparisonComparison Arrays vs. Lists vs. Trees vs. Hash-Arrays vs. Lists vs. Trees vs. Hash-
TablesTables
3.3. Choosing Proper Data StructureChoosing Proper Data Structure
2
Why Data Structures Why Data Structures are Important?are Important?
Data structuresData structures and and algorithmsalgorithms are the are the foundation of computer programmingfoundation of computer programming
Algorithmic thinking, problem solving Algorithmic thinking, problem solving and data structures are vital for and data structures are vital for software engineerssoftware engineers All .NET developers should know when to All .NET developers should know when to
use use T[]T[], , LinkedList<T>LinkedList<T>, , List<T>List<T>, , Stack<T>Stack<T>, , Queue<T>Queue<T>, , Dictionary<K,T>Dictionary<K,T>, , HashSet<T>HashSet<T>, , SortedDictionary<K,T>SortedDictionary<K,T> and and SortedSet<T>SortedSet<T>
Computational complexity is important Computational complexity is important for algorithm design and efficient for algorithm design and efficient programmingprogramming
3
Algorithms Algorithms ComplexityComplexityAsymtotic NotationAsymtotic Notation
Algorithm AnalysisAlgorithm Analysis Why we should analyze algorithms?Why we should analyze algorithms?
Predict the resources that the Predict the resources that the algorithm requiresalgorithm requires Computational time (CPU consumption)Computational time (CPU consumption)
Memory space (RAM consumption)Memory space (RAM consumption)
Communication bandwidth consumptionCommunication bandwidth consumption
The The running timerunning time of an algorithm is: of an algorithm is: The total number of primitive operations The total number of primitive operations
executed (machine independent steps)executed (machine independent steps)
Also known as Also known as algorithm complexityalgorithm complexity
5
Algorithmic ComplexityAlgorithmic Complexity What to measure?What to measure?
MemoryMemory
TimeTime
Number of stepsNumber of steps
Number of particular operationsNumber of particular operations
Number of disk operationsNumber of disk operations
Number of network packetsNumber of network packets
Asymptotic complexityAsymptotic complexity
6
Time ComplexityTime Complexity Worst-caseWorst-case
An upper bound on the running time An upper bound on the running time for any input of given sizefor any input of given size
Average-caseAverage-case Assume all inputs of a given size Assume all inputs of a given size
are equally likelyare equally likely Best-caseBest-case
The lower bound on the running The lower bound on the running timetime
7
Time Complexity – Time Complexity – ExampleExample
Sequential search in a list of size nSequential search in a list of size n Worst-case:Worst-case:
nn comparisons comparisons
Best-case:Best-case: 11 comparison comparison
Average-case:Average-case: n/2n/2 comparisons comparisons
The algorithm runs in The algorithm runs in linear timelinear time Linear number of operationsLinear number of operations
…… …… …… …… …… …… ……
nn
8
Algorithms ComplexityAlgorithms Complexity Algorithm complexityAlgorithm complexity is rough estimation of is rough estimation of
the number of steps performed by given the number of steps performed by given computation depending on the size of the computation depending on the size of the input datainput data
Measured through asymptotic notationMeasured through asymptotic notation
O(g)O(g) where where gg is a function of the input data size is a function of the input data size
Examples:Examples:
Linear complexity Linear complexity O(n)O(n) – all elements are – all elements are processed once (or constant number of processed once (or constant number of times)times)
Quadratic complexity Quadratic complexity O(nO(n22)) – each of the – each of the elements is processed elements is processed nn times times
9
Asymptotic Notation: Asymptotic Notation: DefinitionDefinition
Asymptotic upper boundAsymptotic upper bound O-notation (Big O notation)O-notation (Big O notation)
For given function For given function g(n)g(n), we denote by , we denote by O(g(n))O(g(n)) the set of functions that are the set of functions that are different than different than g(n)g(n) by a constant by a constant
Examples:Examples: 33 ** nn22 ++ n/2n/2 ++ 12 12 ∈∈ O(nO(n22)) 4*n*log4*n*log22(3*n+1)(3*n+1) ++ 2*n-1 2*n-1 ∈∈ O(nO(n ** loglog n)n)
O(g(n))O(g(n)) == { {f(n)f(n): there exist positive : there exist positive constants constants cc and and nn00 such that such that f(n)f(n) <=<= c*g(n)c*g(n) for all for all nn >=>= nn00}}
10
Typical ComplexitiesTypical Complexities
11
ComplexComplexityity
NotatiNotationon DescriptionDescription
constantconstant O(1)O(1)
Constant number of Constant number of operations, not operations, not depending on the depending on the input data size, e.g.input data size, e.g.n = 1 000 000 n = 1 000 000 1-2 1-2 operationsoperations
logarithlogarithmicmic
O(logO(log n)n)
Number of operations Number of operations propor-tional propor-tional of logof log22(n) (n) where n is the size of where n is the size of the input data, e.g. n the input data, e.g. n = 1 000 000 000 = 1 000 000 000 30 30 operationsoperations
linearlinear O(n)O(n)
Number of operations Number of operations proportional to the proportional to the input data size, e.g. n input data size, e.g. n = 10 000 = 10 000 5 000 5 000 operationsoperations
Typical Complexities (2)Typical Complexities (2)
12
ComplexiComplexityty
NotatiNotationon DescriptionDescription
quadraticquadratic O(nO(n22))
Number of operations Number of operations proportional to the proportional to the square of the size of square of the size of the input data, e.g. n the input data, e.g. n = 500 = 500 250 000 250 000 operationsoperations
cubiccubic O(nO(n33))
Number of operations Number of operations propor-tionalpropor-tional to the to the cube of the size of the cube of the size of the input data, e.g. n =input data, e.g. n =200 200 8 000 000 8 000 000 operationsoperations
exponentexponentialial
O(2O(2nn)),,O(O(kknn)),,O(n!)O(n!)
Exponential number of Exponential number of operations, fast operations, fast growing, e.g. n = 20 growing, e.g. n = 20 1 048 576 operations1 048 576 operations
Time Complexity and Time Complexity and SpeedSpeed
13
ComplexComplexityity 1010 2020 5050 100100 11
0000001010 000000
100100 000000
O(1)O(1) < < 11 ss< < 11 ss
< < 11 ss < < 11 ss < < 11 ss < < 11 ss < < 11 ss
O(log(n))O(log(n)) < < 11 ss< < 11 ss
< < 11 ss < < 11 ss < < 11 ss < < 11 ss < < 11 ss
O(n)O(n) < < 11 ss< < 11 ss
< < 11 ss < < 11 ss < < 11 ss < < 11 ss < < 11 ss
O(n*log(nO(n*log(n))))
< < 11 ss< < 11 ss
< < 11 ss < < 11 ss < < 11 ss < < 11 ss < < 11 ss
O(nO(n22)) < < 11 ss< < 11 ss
< < 11 ss < < 11 ss < < 11 ss 22 ss33--44 minmin
O(nO(n33)) < < 11 ss< < 11 ss
< < 11 ss < < 11 ss 2020 s s55
hourshours231231
daysdays
O(2O(2nn)) < < 11 ss< < 11 ss
260260 daysdays
hanhangsgs
hanhangsgs
hanghangss
hangshangs
O(n!)O(n!) < < 11 sshanhangsgs
hanhangsgs
hanhangsgs
hanhangsgs
hanghangss
hangshangs
O(nO(nnn))33--44 minmin
hanhangsgs
hanhangsgs
hanhangsgs
hanhangsgs
hanghangss
hangshangs
Time and Memory Time and Memory ComplexityComplexity
Complexity can be expressed as formula Complexity can be expressed as formula on multiple variables, e.g.on multiple variables, e.g.
Algorithm filling a matrix of size Algorithm filling a matrix of size nn * * mm with with natural numbers natural numbers 11, , 22, … will run in , … will run in O(n*m)O(n*m)
DFS traversal of graph with DFS traversal of graph with nn vertices vertices and and mm edges will run in edges will run in O(nO(n ++ m)m)
Memory consumption should also be Memory consumption should also be considered, for example:considered, for example:
Running time Running time O(n)O(n), memory requirement , memory requirement O(nO(n22))
n = 50 000 n = 50 000 OutOfMemoryExceptionOutOfMemoryException14
Polynomial AlgorithmsPolynomial Algorithms A A polynomial-time algorithmpolynomial-time algorithm is one is one
whose worst-case time complexity is whose worst-case time complexity is bounded above by a polynomial function bounded above by a polynomial function of its input sizeof its input size
Example of worst-case time complexityExample of worst-case time complexity Polynomial-time:Polynomial-time: log log nn, , 2n2n, , 3n3n33 ++ 4n4n, , 22 * * nn
log log nn Non polynomial-time : Non polynomial-time : 22nn, , 33nn,, nnkk, , n!n!
Non-polynomial algorithms don't work Non-polynomial algorithms don't work for large input data setsfor large input data sets
W(n) W(n) ∈ O(p(n)) O(p(n))
15
Analyzing Analyzing Complexity of Complexity of
AlgorithmsAlgorithmsExamplesExamples
Complexity ExamplesComplexity Examples
Runs in Runs in O(n)O(n) where where nn is the size of the is the size of the arrayarray
The number of elementary steps is The number of elementary steps is ~~ nn
int FindMaxElement(int[] array)int FindMaxElement(int[] array){{ int max = array[0];int max = array[0]; for (int i=0; i<array.length; i++)for (int i=0; i<array.length; i++) {{ if (array[i] > max)if (array[i] > max) {{ max = array[i];max = array[i]; }} }} return max;return max;}}
Complexity Examples Complexity Examples (2)(2)
Runs in Runs in O(nO(n22)) where where nn is the size of is the size of the arraythe array
The number of elementary steps is The number of elementary steps is ~~ n*(n+1)n*(n+1) // 22
long FindInversions(int[] array)long FindInversions(int[] array){{ long inversions = 0;long inversions = 0; for (int i=0; i<array.Length; i++)for (int i=0; i<array.Length; i++) for (int j = i+1; j<array.Length; i+for (int j = i+1; j<array.Length; i++)+) if (array[i] > array[j])if (array[i] > array[j]) inversions++;inversions++; return inversions;return inversions;}}
Complexity Examples Complexity Examples (3)(3)
Runs in cubic time Runs in cubic time O(nO(n33)) The number of elementary steps is The number of elementary steps is ~~ nn33
decimal Sum3(int n)decimal Sum3(int n){{ decimal sum = 0;decimal sum = 0; for (int a=0; a<n; a++)for (int a=0; a<n; a++) for (int b=0; b<n; b++)for (int b=0; b<n; b++) for (int c=0; c<n; c++)for (int c=0; c<n; c++) sum += a*b*c;sum += a*b*c; return sum;return sum;}}
Complexity Examples Complexity Examples (4)(4)
Runs in quadratic time Runs in quadratic time O(n*m)O(n*m) The number of elementary steps is The number of elementary steps is ~~ n*mn*m
long SumMN(int n, int m)long SumMN(int n, int m){{ long sum = 0;long sum = 0; for (int x=0; x<n; x++)for (int x=0; x<n; x++) for (int y=0; y<m; y++)for (int y=0; y<m; y++) sum += x*y;sum += x*y; return sum;return sum;}}
Complexity Examples Complexity Examples (5)(5)
Runs in quadratic time Runs in quadratic time O(n*m)O(n*m) The number of elementary steps is The number of elementary steps is
~~ n*mn*m ++ min(m,n)*nmin(m,n)*n
long SumMN(int n, int m)long SumMN(int n, int m){{ long sum = 0;long sum = 0; for (int x=0; x<n; x++)for (int x=0; x<n; x++) for (int y=0; y<m; y++)for (int y=0; y<m; y++) if (x==y)if (x==y) for (int i=0; i<n; i++)for (int i=0; i<n; i++) sum += i*x*y;sum += i*x*y; return sum;return sum;}}
Complexity Examples Complexity Examples (6)(6)
Runs in exponential time Runs in exponential time O(2O(2nn)) The number of elementary steps is The number of elementary steps is ~~ 22nn
decimal Calculation(int n)decimal Calculation(int n){{ decimal result = 0;decimal result = 0; for (int i = 0; i < (1<<n); i++)for (int i = 0; i < (1<<n); i++) result += i;result += i; return result;return result;}}
Complexity Examples Complexity Examples (7)(7)
Runs in linear time Runs in linear time O(n)O(n) The number of elementary steps is The number of elementary steps is ~~ nn
decimal Factorial(int n)decimal Factorial(int n){{ if (n==0)if (n==0) return 1;return 1; elseelse return n * Factorial(n-1);return n * Factorial(n-1);}}
Complexity Examples Complexity Examples (8)(8)
Runs in Runs in exponential time exponential time O(2O(2nn)) The number of elementary steps is The number of elementary steps is
~~ Fib(n+1)Fib(n+1) w where here Fib(k)Fib(k) is the is the kk-th Fib-th Fiboonacci's numbernacci's number
decimal Fibonacci(int n)decimal Fibonacci(int n){{ if (n == 0)if (n == 0) return 1;return 1; else if (n == 1)else if (n == 1) return 1;return 1; elseelse return Fibonacci(n-1) + Fibonacci(n-2);return Fibonacci(n-1) + Fibonacci(n-2);}}
Comparing Data Comparing Data StructuresStructures
ExamplesExamples
Data Structures Data Structures EfficiencyEfficiency
26
Data Data StructureStructure AddAdd FinFin
ddDeletDelet
ee
Get-Get-by-by-
indexindexArray (Array (T[]T[])) O(n)O(n) O(n)O(n) O(n)O(n) O(1)O(1)
Linked list Linked list ((LinkedList<T>LinkedList<T>
))O(1)O(1) O(n)O(n) O(n)O(n) O(n)O(n)
Resizable Resizable array list array list ((List<T>List<T>))
O(1)O(1) O(n)O(n) O(n)O(n) O(1)O(1)
Stack Stack ((Stack<T>Stack<T>)) O(1)O(1) -- O(1)O(1) --
Queue Queue ((Queue<T>Queue<T>)) O(1)O(1) -- O(1)O(1) --
Data Structures Data Structures EfficiencyEfficiency (2) (2)
27
Data Data StructureStructure AddAdd FindFind DeletDelet
ee
Get-Get-by-by-
indexindexHash table Hash table
((Dictionary<K,TDictionary<K,T>>))
O(1)O(1) O(1)O(1) O(1)O(1) --
Tree-based Tree-based dictionary dictionary
((Sorted Sorted Dictionary<K,TDictionary<K,T
>>))
O(logO(log n)n)
O(logO(log n)n)
O(logO(log n)n) --
Hash table Hash table based set based set
((HashSet<T>HashSet<T>))O(1)O(1) O(1)O(1) O(1)O(1) --
Tree based set Tree based set ((SortedSet<T>SortedSet<T>))
O(logO(log n)n)
O(logO(log n)n)
O(logO(log n)n) --
Choosing Data Choosing Data StructureStructure
Arrays (Arrays (T[]T[])) Use when fixed number of elements Use when fixed number of elements
should be processed by indexshould be processed by index Resizable array lists (Resizable array lists (List<T>List<T>))
Use when elements should be added Use when elements should be added and processed by indexand processed by index
Linked lists (Linked lists (LinkedList<T>LinkedList<T>)) Use when elements should be added Use when elements should be added
at the both sides of the listat the both sides of the list Otherwise use resizable array list Otherwise use resizable array list
((List<T>List<T>)) 28
Choosing Data Choosing Data Structure (2)Structure (2)
Stacks (Stacks (Stack<T>Stack<T>)) Use to implement LIFO (last-in-first-out) Use to implement LIFO (last-in-first-out)
behaviorbehavior List<T>List<T> could also work well could also work well
Queues (Queues (Queue<T>Queue<T>)) Use to implement FIFO (first-in-first-out) Use to implement FIFO (first-in-first-out)
behaviorbehavior LinkedList<T>LinkedList<T> could also work well could also work well
Hash table based dictionary (Hash table based dictionary (Dictionary<K,T>Dictionary<K,T>)) Use when key-value pairs should be added fast Use when key-value pairs should be added fast
and searched fast by keyand searched fast by key Elements in a hash table have no particular Elements in a hash table have no particular
orderorder29
Choosing Data Choosing Data Structure (3)Structure (3)
Balanced search tree based dictionary Balanced search tree based dictionary ((SortedDictionary<K,T>SortedDictionary<K,T>)) Use when key-value pairs should be added Use when key-value pairs should be added
fast, searched fast by key and enumerated fast, searched fast by key and enumerated sorted by keysorted by key
Hash table based set (Hash table based set (HashSet<T>HashSet<T>)) Use to keep a group of unique values, to Use to keep a group of unique values, to
add and check belonging to the set fastadd and check belonging to the set fast Elements are in no particular orderElements are in no particular order
Search tree based set (Search tree based set (SortedSet<T>SortedSet<T>)) Use to keep a group of ordered unique Use to keep a group of ordered unique
valuesvalues30
SummarySummary Algorithm complexityAlgorithm complexity is rough estimation is rough estimation
of the number of steps performed by of the number of steps performed by given computationgiven computation
Complexity can be logarithmic, linear, n log Complexity can be logarithmic, linear, n log n, square, cubic, exponential, etc.n, square, cubic, exponential, etc.
Allows to estimating the speed of given Allows to estimating the speed of given code before its executioncode before its execution
Different data structures have different Different data structures have different efficiency on different operationsefficiency on different operations The fastest add / find / delete structure The fastest add / find / delete structure
is the hash table – is the hash table – O(1)O(1) for all these for all these operationsoperations
31
Algorithms Complexity Algorithms Complexity and Data Structures and Data Structures
EfficiencyEfficiency
Questions?Questions?
http://academy.telerik.com
ExercisesExercises
1.1. A text file A text file students.txtstudents.txt holds holds information about students and their information about students and their courses in the following format:courses in the following format:
Using Using SortedDictionary<K,T>SortedDictionary<K,T> print the print the courses in alphabetical order and for courses in alphabetical order and for each of them prints the students each of them prints the students ordered by family and then by name:ordered by family and then by name:
33
Kiril | Ivanov | C#Kiril | Ivanov | C#Stefka | Nikolova | SQLStefka | Nikolova | SQLStela | Mineva | JavaStela | Mineva | JavaMilena | Petrova | C#Milena | Petrova | C#Ivan | Grigorov | C#Ivan | Grigorov | C#Ivan | Kolev | SQLIvan | Kolev | SQL
C#: Ivan Grigorov, Kiril Ivanov, Milena PetrovaC#: Ivan Grigorov, Kiril Ivanov, Milena PetrovaJava: Stela MinevaJava: Stela MinevaSQL: Ivan Kolev, Stefka NikolovaSQL: Ivan Kolev, Stefka Nikolova
Exercises (2)Exercises (2)2.2. A large trade company has millions of articles, A large trade company has millions of articles,
each described by barcode, vendor, title and each described by barcode, vendor, title and price. Implement a data structure to store them price. Implement a data structure to store them that allows fast retrieval of all articles in given that allows fast retrieval of all articles in given price range price range [x…y][x…y]. Hint: use . Hint: use OrderedMultiDictionary<K,T>OrderedMultiDictionary<K,T> from from Wintellect's Power Collections for .NET.
3.3. Implement a data structure Implement a data structure PriorityQueue<T>PriorityQueue<T> that provides a fast way to execute the that provides a fast way to execute the following operations:following operations: add element; extract the add element; extract the smallest element.smallest element.
4.4. Implement a class Implement a class BiDictionary<K1,K2,T>BiDictionary<K1,K2,T> that that allows adding triples allows adding triples {key1,{key1, key2,key2, value}value} and and fast search by fast search by key1key1, , key2key2 or by both or by both key1key1 and and key2key2. Note: multiple values can be stored for . Note: multiple values can be stored for given key.given key.
34
Exercises (3)Exercises (3)5.5. A text file A text file phones.txtphones.txt holds information holds information
about people, their town and phone about people, their town and phone number:number:
Duplicates can occur in people names, Duplicates can occur in people names, towns and phone numbers. Write a towns and phone numbers. Write a program to execute a sequence of program to execute a sequence of commands from a file commands from a file commands.txtcommands.txt:: find(name)find(name) – display all matching records by – display all matching records by
given name (first, middle, last or nickname)given name (first, middle, last or nickname)
find(name,find(name, town)town) – display all matching – display all matching records by given name and townrecords by given name and town 35
Mimi Shmatkata | Plovdiv | 0888 12 34 56Mimi Shmatkata | Plovdiv | 0888 12 34 56Kireto | Varna | 052 23 45 67Kireto | Varna | 052 23 45 67Daniela Ivanova Petrova | Karnobat | 0899 999 888Daniela Ivanova Petrova | Karnobat | 0899 999 888Bat Gancho | Sofia | 02 946 946 946Bat Gancho | Sofia | 02 946 946 946
top related