Lecture # 1 Introduction Analysis of Algorithm by Qamar Abbas Analysis of Algorithms Class Policy Homework Assignments: 10 % Quizzes: 10 % Midterm : 30 % Final Exam : 50 % Analysis of Algorithm by Qamar Abbas Introduction to Algorithms Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford Stein Pre Requisite Analysis of Algorithm by Qamar Abbas Data Structures Some Programming like C++ etc Some Mathematics knowledge Basic Math skills Lots of Time What can you expect? Analysis of Algorithm by Qamar Abbas After the course expect to Know more about algorithms (of course) Think algorithmically Know how to solve real world algorithmic problems Both in theory (algorithm) and practice (code) Be better at applications that require algorithms: and apply algorithms to places you never imagined Expectations Analysis of Algorithm by Qamar Abbas Work hard and learn/understand the material well. Feel free to ask questions. Take help when required Course Load: 4-6 hours a week (assumes you are getting help). Why Study Algorithms? Necessary in any computer programming problem Improve algorithm efficiency: run faster, process more data, do something that would otherwise be impossible Solve problems of significantly large sizes Technology only improves things by a constant factor Analysis of Algorithm by Qamar Abbas Why Study Algorithms? Compare algorithms Algorithms as a field of study Learn about a standard set of algorithms New discoveries arise Numerous application areas Learn techniques of algorithm design and analysis Analysis of Algorithm by Qamar Abbas Applications Analysis of Algorithm by Qamar Abbas Multimedia MP3, JPG, DivX, HDTV, etc Internet Packet routing, data retrieval (Google) Communication Cell-phones, e-commerce Applications Analysis of Algorithm by Qamar Abbas Computers Circuit layout, file systems Transportation Airline crew scheduling, UPS deliveries Science Applications Some More Applications Analysis of Algorithm by Qamar Abbas Compression LZ, BW, PPM, JPEG, MPEG Cryptography ssh, RSA, Rijdael, kerberos, Error Correcting Codes Reed-Solomon, tornado-codes, CDs Graph Separators Solving linear systems, VLSI, Shortest paths, Linear and Integer Programming Interior point methods, airline crew scheduling, Web Algorithms Searching, Google pagerank, duplicate removal, Computational Biology/String Matching MED, Suffix Trees, BLAST Some More Applications Analysis of Algorithm by Qamar Abbas Face Recognition Scheduling Algorithm Shortest Route Algorithms Pattern Matching Indexing and Search Engines Positioning Algorithms Some More Applications Analysis of Algorithm by Qamar Abbas Clustering Graph search A* Backtracking best-first search Depth-first search Dijkstra's algorithm Clipping Line Drawing Machine learning Algorithms Apriori algorithm C 4.5 Algorithm Compression algorithms Data Structure Vs Algorithm Analysis of Algorithm by Qamar Abbas An algorithm defines a sequence of steps and decisions which can be employed to solve a problem An algorithm is a specific way of programming a task to make it work on the data you have. An algorithm is a specific way that the programmer writes a program to process the information. Algorithms are unique in the way that the program runs and can be measured by their efficiency in processing the data. Data Structure Vs Algorithm Analysis of Algorithm by Qamar Abbas A data structure is the way you define a certain object in a programming language. Data structures describe a collection of values, often with names and information about the hierarchical relationship of those values The data structure is defined by what you need the program to keep track of. Roadmap Analysis of Algorithm by Qamar Abbas Different problems Sorting Searching String processing Graph problems Geometric problems Numerical problems Different design paradigms Divide-and-conquer Incremental Dynamic programming Greedy algorithms Randomized/probabilistic Analyzing Algorithms Analysis of Algorithm by Qamar Abbas Predict the amount of resources required: Memory: how much space is needed? Computational time: how fast the algorithm runs? FACT: running time grows with the size of the input Input size (number of elements in the input) Size of an array, polynomial degree, # of elements in a matrix, # of bits in the binary representation of the input, vertices and edges in a graph Analyzing Algorithms Cont Analysis of Algorithm by Qamar Abbas Def: Running time = the number of primitive operations (steps) executed before termination Arithmetic operations (+, -, *), data movement, control, decision making (if, while), comparison Algorithm Analysis: Example Analysis of Algorithm by Qamar Abbas Alg.: MIN ( a[1], , a[n] ) m a[1]; for i 2 to n if a[i] < m then m a[i]; Algorithm Analysis: Example Analysis of Algorithm by Qamar Abbas Running time: the number of primitive operations (steps) executed before termination T(n) =1 [first step] + (n) [for loop] + (n-1) [if condition] + (n-1) [the assignment in then] = 3n - 1 Order (rate) of growth: The leading term of the formula Expresses the asymptotic behavior of the algorithm Typical Running Time Functions Analysis of Algorithm by Qamar Abbas 1 (Constant running time): Instructions are executed once or a few times logN (logarithmic) A big problem is solved by cutting the original problem in smaller sizes, by a constant fraction at each step N (linear) A small amount of processing is done on each input element N logN A problem is solved by dividing it into smaller problems, solving them independently and combining the solution Typical Running Time Functions Analysis of Algorithm by Qamar Abbas N 2 (quadratic) Typical for algorithms that process all pairs of data items (double nested loops) N 3 (cubic) Processing of triples of data (triple nested loops) N K (polynomial) 2 N (exponential) Few exponential algorithms are appropriate for practical use Recurrences Analysis of Algorithm by Qamar Abbas Def.: Recurrence = an equation or inequality that describes a function in terms of its value on smaller inputs, and one or more base cases E.g.: T(n) = T(n-1) + n Sorting Analysis of Algorithm by Qamar Abbas Iterative methods: Insertion sort Bubble sort Selection sort 2, 3, 4, 5, 6, 7, 8, 9, 10, J, Q, K, A Divide and conquer Merge sort Quicksort Non-comparison methods Counting sort Radix sort Bucket sort