templates, algorithms, big oh data structures and algorithms cs 244 brent m. dingle, ph.d....
TRANSCRIPT
Templates, Algorithms, Big Oh
Data Structures and Algorithms
CS 244
Brent M. Dingle, Ph.D.
Department of Mathematics, Statistics, and Computer Science
University of Wisconsin – Stout
Based on the book: Data Structures and Algorithms in C++ (Goodrich, Tamassia, Mount)Some content derived/taken from: http://www.stroustrup.com/Programming/ and some from Data Structures for Game Programmers (Penton)
In October of 1976 I observed that a certain algorithm – parallel reduction – was associated withmonoids: collections of elements with an associative operation.
That observation led me to believe that it is possible to associate every useful algorithm with a mathematical theory and that such association allows for both widest possible use and meaningful taxonomy.
As mathematicians learned to lift theorems into their most general settings, so I wanted to lift algorithms and data structures.
– Alex Stepanov, inventor of the STL. [Ste07]
Things to Note
• Homework 5 is Due Soon
• Homework 6 is Posted on D2L– Do NOT delay in starting it
• Do not forget to look at the Meta-Info files
From Last Time– UML Hierarchy
– C++ Inheritance
– Polymorphism
– Program Design
For Today
• Templates– General Introduction– Function Templates– Class Templates
Marker Slide
• Any General Questions ?
• Next up– Templates• Introduction• Function Templates• Class Templates
– Algorithms– Big-Oh
Template Description
• A template is– a pattern, or mold, which will
be reused over and over again.
• Example: Company manufactures figurines• First creates a mold of the figurine to produce• Then chooses what material they want the figurine
made out of– clay, lead, plastic, iron, pewter, gold, silver…
• Then uses the mold to create the figurine
Template in C++
• A template is a mold for an algorithm or class– Function Templates (algorithm)
– Class Templates (class)
• where the programmers decide what type of material they want to use with it
• Aside:– The standard library (Standard Template Library)
• Makes heavy use of Templates (surprised? =)– We may see much more of them in the future
Graded In-Class Activity: MySums• Write a program as follows. Name the file: MySums.cpp
• Write a function named SumInts(int* numList, int size) that sums an array of integers and returns the result as an integer
• Write a second function named SumFloats(float* numList, int size) that sums 5 floats and returns the result as a float
• Write a third function named SumDoubles(double* numList, int size) that sums 5 doubles and returns the result as a double
• Write a main() function that declares 5 variables of each type. Assign them whatever values you feel appropriate. Call the three functions above and output the results in a single cout line:– cout << SumInts(myInts, 5) << " "
<< SumFloats(myFloats, 5) << " " << SumDoubles(myDoubles, 5) << endl;
• When complete turn the source file into D2L in the appropriate dropbox– May work as groups, but each individual must turn something in
Template Motivation• As was illustrated by the previous class activity• If you want an algorithm to work with SIX different types of data.
– Need to write SIX different functions
• Templates can helpmitigate this
Marker Slide
• Any Questions On:– Templates• Introduction
• Next up– Templates• Function Templates• Class Templates
– Algorithms– Big-Oh
Function Templates
• A template function is – a function that can operate on a generic data type• i.e. you can use the same function with different data
types and the compiler “magically” makes it work for you
Syntax for Template Function• template < typename T >
returntype functionName( parameter list )
• template keyword indicates you are creating a template
• typename keyword indicates you are going to provide a name for the generic type to be used in the template– by convention T is commonly used
as the generic type name
• The rest is as a ‘normal’ function– you will likely use the generic type T somewhere, commonly:
• as the return type • or as one or more of the parameter types
Syntax for Template Function• template < typename T >
returntype functionName( parameter list )
• template keyword indicates you are creating a template
• typename keyword indicates you are going to provide a name for the generic type to be used in the template– by convention T is commonly used
as the generic type name
• The rest is as a ‘normal’ function– you will likely use the generic type T somewhere, commonly:
• as the return type • or as one or more of the parameter types
Older code (and compilers) re-used the keyword class instead of typename
This is a little confusing to humans.
Some of the example code likely uses it instead of typename.
However, typename is the more understandable to use
Sum Template Example// ---------------------------------------------------------------------------// Templated Sum Function// Sums a list of numbers of specified length// ---------------------------------------------------------------------------template <typename T>T SumThings(T* thingList, int size){ T sum = 0; for (int i=0; i < size; i++) { sum += thingList[i]; } return sum;}
Sum Template Example// ---------------------------------------------------------------------------// Templated Sum Function// Sums a list of numbers of specified length// ---------------------------------------------------------------------------template <typename T>T SumThings(T* thingList, int size){ T sum = 0; for (int i=0; i < size; i++) { sum += thingList[i]; } return sum;}
Declare a template
Generic Type Name is T
Sum Template Example// ---------------------------------------------------------------------------// Templated Sum Function// Sums a list of numbers of specified length// ---------------------------------------------------------------------------template <typename T>T SumThings(T* thingList, int size){ T sum = 0; for (int i=0; i < size; i++) { sum += thingList[i]; } return sum;}
Return type is Generic Type T
Function name is: SumThings
Sum Template Example// ---------------------------------------------------------------------------// Templated Sum Function// Sums a list of numbers of specified length// ---------------------------------------------------------------------------template <typename T>T SumThings(T* thingList, int size){ T sum = 0; for (int i=0; i < size; i++) { sum += thingList[i]; } return sum;}
First parameter is a pointer to type Generic Type T
to be used as an arrayas the second parameter is a “size”
Sum Template Example// ---------------------------------------------------------------------------// Templated Sum Function// Sums a list of numbers of specified length// ---------------------------------------------------------------------------template < typename T>T SumThings(T* thingList, int size){ T sum = 0; for (int i=0; i < size; i++) { sum += thingList[i]; } return sum;}
Local variable sum is type Generic Type T
Note there is an assumption type T supports the assignment operator and can be assigned the value zero
Sum Template Example// ---------------------------------------------------------------------------// Templated Sum Function// Sums a list of numbers of specified length// ---------------------------------------------------------------------------template < typename T>T SumThings(T* thingList, int size){ T sum = 0; for (int i=0; i < size; i++) { sum += thingList[i]; } return sum;}
Sum up things using a for-loop.
Another implicit assumption:type T supports the operator +=
Sum Template Example// ---------------------------------------------------------------------------// Templated Sum Function// Sums a list of numbers of specified length// ---------------------------------------------------------------------------template < typename T>T SumThings(T* thingList, int size){ T sum = 0; for (int i=0; i < size; i++) { sum += thingList[i]; } return sum;}
Return the calculated sumof generic type T
Magic of the Compiler• You can run the below and the compiler will
automatically create the appropriate function for you– Putting in the correct type for T each time
int main(){ int myInts[5] = { 4, 45, 65, 34, 12 }; float myFloats[5] = { 3.4, 0.5, 6.9, 43.2, -14.7 }; double myDoubles[5] = { 3.4, 10.5, 6.9, 43.2, -14.7 }; cout << SumThings(myInts, 5) << " " << SumThings(myFloats, 5) << " " << SumThings(myDoubles, 5) << endl; return 0;}
Magic of the Compiler• You can run the below and the compiler will
automatically create the appropriate function for you– Putting in the correct type for T each time
int main(){ int myInts[5] = { 4, 45, 65, 34, 12 }; float myFloats[5] = { 3.4, 0.5, 6.9, 43.2, -14.7 }; double myDoubles[5] = { 3.4, 10.5, 6.9, 43.2, -14.7 }; cout << SumThings(myInts, 5) << " " << SumThings(myFloats, 5) << " " << SumThings(myDoubles, 5) << endl; return 0;}
The type of variable sent to SumThings() determines what generic type T becomes
Magic of the Compiler• You can run the below and the compiler will
automatically create the appropriate function for you– Putting in the correct type for T each time
int main(){ int myInts[5] = { 4, 45, 65, 34, 12 }; float myFloats[5] = { 3.4, 0.5, 6.9, 43.2, -14.7 }; double myDoubles[5] = { 3.4, 10.5, 6.9, 43.2, -14.7 }; cout << SumThings(myInts, 5) << " " << SumThings(myFloats, 5) << " " << SumThings(myDoubles, 5) << endl; return 0;}
myInts is int * so T = int
Magic of the Compiler• You can run the below and the compiler will
automatically create the appropriate function for you– Putting in the correct type for T each time
int main(){ int myInts[5] = { 4, 45, 65, 34, 12 }; float myFloats[5] = { 3.4, 0.5, 6.9, 43.2, -14.7 }; double myDoubles[5] = { 3.4, 10.5, 6.9, 43.2, -14.7 }; cout << SumThings(myInts, 5) << " " << SumThings(myFloats, 5) << " " << SumThings(myDoubles, 5) << endl; return 0;}
myFloats is float* so T = float
Magic of the Compiler• You can run the below and the compiler will
automatically create the appropriate function for you– Putting in the correct type for T each time
int main(){ int myInts[5] = { 4, 45, 65, 34, 12 }; float myFloats[5] = { 3.4, 0.5, 6.9, 43.2, -14.7 }; double myDoubles[5] = { 3.4, 10.5, 6.9, 43.2, -14.7 }; cout << SumThings(myInts, 5) << " " << SumThings(myFloats, 5) << " " << SumThings(myDoubles, 5) << endl; return 0;}
myDoubles is double*
so T = double
Sum Template Int
• Template: no changes needed
// ---------------------------------------------------------------------------// Templated Sum Function// Sums a list of numbers of specified length// ---------------------------------------------------------------------------template < typename T>T SumThings(T* thingList, int size){ T sum = 0; for (int i=0; i < size; i++) { sum += thingList[i]; } return sum;}
// --------------------------------------------------------// SumInts// Sums a list of integers of specified length// --------------------------------------------------------
int SumInts(int* numList, int size){ int sum = 0; for (int i=0; i < size; i++) { sum += numList[i]; } return sum;}
Sum Template Float
• Template: no changes needed
// ---------------------------------------------------------------------------// Templated Sum Function// Sums a list of numbers of specified length// ---------------------------------------------------------------------------template < typename T>T SumThings(T* thingList, int size){ T sum = 0; for (int i=0; i < size; i++) { sum += thingList[i]; } return sum;}
// ------------------------------------------------------// SumFloats// Sums a list of floats of specified length// -----------------------------------------------------
float SumFloats(float* numList, int size){ float sum = 0; for (int i=0; i < size; i++) { sum += numList[i]; } return sum;}
Sum Template Double
• Template: no changes needed
// ---------------------------------------------------------------------------// Templated Sum Function// Sums a list of numbers of specified length// ---------------------------------------------------------------------------template < typename T>T SumThings(T* thingList, int size){ T sum = 0; for (int i=0; i < size; i++) { sum += thingList[i]; } return sum;}
// ------------------------------------------------------// SumDoubles// Sums a list of integers of specified length// -----------------------------------------------------
double SumDoubles(double* numList, int size){ double sum = 0; for (int i=0; i < size; i++) { sum += numList[i]; } return sum;}
Marker Slide
• Any Questions On:– Templates• Introduction• Function Templates
• Next up– Templates• Class Templates
– Algorithms– Big-Oh
Template Classes
• Same idea as template functions
• BUT the entire class operates on a generic type
Template Class – Motivation class IntAdder{ public: // constructor IntAdder() { m_sum = 0; } // add function void Add( int p_number ) { m_sum += p_number; } // get sum function. int Sum() { return m_sum; } private: // sum variable. int m_sum;};
class FloatAdder{ public: // constructor FloatAdder() { m_sum = 0.0f; } // add function void Add( float p_number ) { m_sum += p_number; } // get sum function. float Sum() { return m_sum; } private: // sum variable. float m_sum;};
Template Class – Motivation class IntAdder{ public: // constructor IntAdder() { m_sum = 0; } // add function void Add( int p_number ) { m_sum += p_number; } // get sum function. int Sum() { return m_sum; } private: // sum variable. int m_sum;};
class FloatAdder{ public: // constructor FloatAdder() { m_sum = 0.0f; } // add function void Add( float p_number ) { m_sum += p_number; } // get sum function. float Sum() { return m_sum; } private: // sum variable. float m_sum;};
int main(){ IntAdder iadder1; FloatAdder fadder1; int i; float f; for( i = 0, f = 0.0f; i < 10; i++, f += 1.1f ) { iadder1.Add( i ); fadder1.Add( f ); } cout << "The integer sum using an IntAdder: " << iadder1.Sum() << endl; cout << "The float sum using a FloatAdder: " << fadder1.Sum() << endl; return 0;}
Template Class – Motivation class IntAdder{ public: // constructor IntAdder() { m_sum = 0; } // add function void Add( int p_number ) { m_sum += p_number; } // get sum function. int Sum() { return m_sum; } private: // sum variable. int m_sum;};
class FloatAdder{ public: // constructor FloatAdder() { m_sum = 0.0f; } // add function void Add( float p_number ) { m_sum += p_number; } // get sum function. float Sum() { return m_sum; } private: // sum variable. float m_sum;};
int main(){ IntAdder iadder1; FloatAdder fadder1; int i; float f; for( i = 0, f = 0.0f; i < 10; i++, f += 1.1f ) { iadder1.Add( i ); fadder1.Add( f ); } cout << "The integer sum using an IntAdder: " << iadder1.Sum() << endl; cout << "The float sum using a FloatAdder: " << fadder1.Sum() << endl; return 0;}
Template Class – Template class Adder template< typename T >class Adder{ private: // sum variable. T m_sum; public: // constructor Adder() { m_sum = 0; } // add function void Add( T p_number ) { m_sum += p_number; } // get sum function. T Sum() { return m_sum; }};
Declare a template
Generic Type Name is T
Template Class – Template class Adder template< typename T >class Adder{ private: // sum variable. T m_sum; public: // constructor Adder() { m_sum = 0; } // add function void Add( T p_number ) { m_sum += p_number; } // get sum function. T Sum() { return m_sum; }};
class name Adder
Template Class – Template class Adder template< typename T >class Adder{ private: // sum variable. T m_sum; public: // constructor Adder() { m_sum = 0; } // add function void Add( T p_number ) { m_sum += p_number; } // get sum function. T Sum() { return m_sum; }};
private member variable of generic type T named m_sum
Template Class – Template class Adder template< typename T >class Adder{ private: // sum variable. T m_sum; public: // constructor Adder() { m_sum = 0; } // add function void Add( T p_number ) { m_sum += p_number; } // get sum function. T Sum() { return m_sum; }};
Implicit assumption generic type T supports the assignment operator=
and you can assign a zero to the type
Template Class – Template class Adder template< typename T >class Adder{ private: // sum variable. T m_sum; public: // constructor Adder() { m_sum = 0; } // add function void Add( T p_number ) { m_sum += p_number; } // get sum function. T Sum() { return m_sum; }};
Add function has parameter ofgeneric type Tnamed p_number
(p_ for parameter variable… not a common naming conventionas it conflicts with p_ for pointer)
Template Class – Template class Adder template< typename T >class Adder{ private: // sum variable. T m_sum; public: // constructor Adder() { m_sum = 0; } // add function void Add( T p_number ) { m_sum += p_number; } // get sum function. T Sum() { return m_sum; }};
Implicit assumption generic type T supports the assignment operator+=
Template Class – Template class Adder template< typename T >class Adder{ private: // sum variable. T m_sum; public: // constructor Adder() { m_sum = 0; } // add function void Add( T p_number ) { m_sum += p_number; } // get sum function. T Sum() { return m_sum; }};
Return value of Sum() function is generic type T
Template Class – Template class Adder int main(){ Adder<int> iadder2; Adder<float> fadder2; int i; float f; for( i = 0, f = 0.0f; i < 10; i++, f += 1.1f ) { iadder2.Add( i ); fadder2.Add( f ); } cout << "The integer sum using an Adder: " << iadder2.Sum() << endl; cout << "The float sum using an Adder: " << fadder2.Sum() << endl; return 0;}
template< typename T >class Adder{ private: // sum variable. T m_sum; public: // constructor Adder() { m_sum = 0; } // add function void Add( T p_number ) { m_sum += p_number; } // get sum function. T Sum() { return m_sum; }};
Declare variable named iadder2of type integer Adder
Note <int> specifies what T becomes
Template Class – Template class Adder int main(){ Adder<int> iadder2; Adder<float> fadder2; int i; float f; for( i = 0, f = 0.0f; i < 10; i++, f += 1.1f ) { iadder2.Add( i ); fadder2.Add( f ); } cout << "The integer sum using an Adder: " << iadder2.Sum() << endl; cout << "The float sum using an Adder: " << fadder2.Sum() << endl; return 0;}
template< typename T >class Adder{ private: // sum variable. T m_sum; public: // constructor Adder() { m_sum = 0; } // add function void Add( T p_number ) { m_sum += p_number; } // get sum function. T Sum() { return m_sum; }};
Declare variable named iadder2of type float Adder
Note <float> specifies what T becomes
Template Class – Template class Adder int main(){ Adder<int> iadder2; Adder<float> fadder2; int i; float f; for( i = 0, f = 0.0f; i < 10; i++, f += 1.1f ) { iadder2.Add( i ); fadder2.Add( f ); } cout << "The integer sum using an Adder: " << iadder2.Sum() << endl; cout << "The float sum using an Adder: " << fadder2.Sum() << endl; return 0;}
template< typename T >class Adder{ private: // sum variable. T m_sum; public: // constructor Adder() { m_sum = 0; } // add function void Add( T p_number ) { m_sum += p_number; } // get sum function. T Sum() { return m_sum; }};
Template Summary
• Templates allow the creation of– template functions– template classes
• Allowing the “same” code to be executed on different types of data
More Examples?
• ok… a couple more examples
Function Templates• Suppose we have a useful function, like the one below.
– But we really want it to work with numbers of any type• Solution: Create a template
int integerMin( int a, int b ){ return (a < b ? a : b);}
T T T
template <typename T>
The compiler looks at the argument types and determines which form of the function to instantiate.
cout << genericMin(3, 4) << ’ ’ // = genericMin<int>(3,4)
<< genericMin(1.1, 3.1) << ’ ’ // = genericMin<double>(1.1, 3.1)
<< genericMin(’t’, ’g’) << endl; // = genericMin<char>(’t’,’g’)
Class Templates• BasicVector class with its elements limited to being only of type double
class BasicVector // a simple vector class{public: BasicVector(int capac = 10); // constructor double& operator[](int i) // access element at index i { return a[i]; } // ... other public members omitted private: double* m_a; // array storing the elements int m_capacity; // length of array a };
• BasicVector class using templates to allow elements to be of any type
template <typename T>
T &
T *
Caution: Templates for Hats
• Hat templates – fail to work with some types of materials
Marker Slide
• Any Questions On:– Templates• Introduction• Function Templates• Class Templates
• Next up– Algorithms– Big-Oh
Serving Up Some Tools
• Algorithms Introduction
• Recipes as Algorithms
• Analyzing Computer Algorithms
Algorithm Introduction• An algorithm is a set of instructions used to solve
a specific problem.– A sequence of steps to solve the problem– A step by step set of instructions to solve the
problem
– Synonyms• Process• Method• Procedure• Routine• Recipe
Didn’t we just see some
algorithms?
Yeah, but I’m hungry. Let’s cook
something.
Properties of Algorithms• Input – Accurate Specification
– Under what conditions should the algorithm be expected to give the correct solution
• Definitive Instructions– Clear and Unambiguous
• Correctness– The algorithm must correctly solve the problem
• Result Specification– What the algorithm is intended to do should be clear
• Termination – Execution Time– The algorithm must eventually stop for all valid inputs
– i.e. complete in finite time
Recipes as Algorithms
• Lemon Soufflé– 1 Envelope unflavored
gelatin– ¼ Cup cold water– 6 Egg yolks– 1 Cup sugar– 2/3 Cup lemon juice– 1 Tablespoon grated
lemon rind– 4 Egg whites– 1 ½ Cup heavy cream
1. Soften the gelatin in water. Beat egg yokes and sugar until thick and light. Stir in lemonjuice and cook over low heat, beating steadilywith a whisk until thickened and hot but not boiling (10 – 15 minutes)
2. Pour mixture into large bowl, mix in gelatinuntil dissolved then add lemon rind. Stiroccasionally until cool.
3. Beat egg whites until stiff but not dry. Foldinto lemon mixture, then whip the creamand fold in. Pour into a two-quart souffledish (large bowl) and refrigerate at least12 hours.
Evaluate the Recipeas an Algorithm
• Input?
• Definitive Instructions?
• Correctness ?
• Result Specification?
• Termination ?
Ingredients are listed
What is “light?”How long should it be whipped?Refrigerated at what temperature?
The proof would be in the tasting… (subjective?)
What if stuff never boils? What if it never becomes light?
Name is pretty indicative of result: Lemon Souffle
Algorithms: Putting the Pieces Together• Input – Accurate Specification
– Under what conditions should the algorithm be expected to give the correct solution
• Definitive Instructions– Clear and Unambiguous
• Correctness– The algorithm must correctly solve the problem
• Result Specification– What the algorithm is intended to do should be clear
• Termination – Execution Time– The algorithm must eventually stop for all valid inputs
– i.e. complete in finite time
InputDefinition
Correctness
Specification
Time
Marker Slide
• Any Questions On:– Templates• Introduction• Function Templates• Class Templates
– Algorithms• General
• Next up– Algorithms• Execution Time
– Big-Oh
Algorithm Termination: Execution Time
• Two points to consider1. Does the algorithm ever terminate?
So the Second Point will be the one we want to focus on:
2. How long will it take to terminate?• Characterize it• In comparative
AND consistent way
Assume YES
Analyzing Running Time
• So…How might we determine how long an algorithm will take to execute?
• Suggestions?– no peaking on next slide
– [ pause here for student participation ]
Two Common Ways
• Physically Time the Implementation• Wall-clock time• Benchmarks• Experimental Studies
• Theoretical Analysis– Asymptotic Analysis• Use Math
Experimental Studies• Write a program
implementing the algorithm
• Run the program with inputs of varying size and composition
• Use a method like clock() to get an accurate measure of the actual running time
• Plot the results0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 50 100
Input Size
Tim
e (
ms)
Class Experiment• On D2L locate the file:
– EX083_clockTest.cpp• Content->Unit01
->In-Class Assignments->Examples
• Form into groups– Appoint someone to keep
track of times• Each team member compile
and run the program• Record the output execution time• Repeat 3 to 5 times
– Analyze the data• Group’s Mean Average,
Minimum, Maximum
n Group 1 Group 2
10,000 0 s 0.002 s
310,000 0.21 s 0.207 s
610,000 0.59 s 0.559 s
910,000 1.06 s 1.009 s
121,000 1.66 s 1.537 s
151,000 2.37 s 2.166 s
181,000 3.1 s 2.845 s
211,000 3.88 s …
241,000 4.67 s
271,000 5.51 sn Group 1 Group 2
620,000 0.04 s 0.176 s
182,000 0.11 s 0.291 s
302,000 0.17 s
422,000 0.25 s
542,000 0.31 s
Freqy Experiment Results
n Group 1 Group 2 Group 3 Group 4 Group 5 Group 6
10,000 0 s
310,000 0.21 s
610,000 0.59 s
910,000 1.06 s
121,000 1.66 s
151,000 2.37 s
181,000 3.1 s
211,000 3.88 s
241,000 4.67 s
271,000 5.51 s
Loopy Experiment Results
n Group 1 Group 2 Group 3 Group 4 Group 5 Group 6
620,000 0.04 s
182,000 0.11 s
302,000 0.17 s
422,000 0.25 s
542,000 0.31 s
Limitations of Experiments• It is necessary to implement the algorithm,
which may be difficult
• Results may not be indicative of the running time on other inputs not included in the experiment.
• In order to compare two algorithms, the same hardware and software environments must be used
The Pieces So Far• Input – Accurate Specification
– Under what conditions should the algorithm be expected to give the correct solution
• Definitive Instructions– Clear and Unambiguous
• Correctness– The algorithm must correctly solve the problem
• Result Specification– What the algorithm is intended to do should be clear
• Termination – Execution Time– The algorithm must eventually stop for all valid inputs
– i.e. complete in finite time
InputDefinition
Correctness
Specification
Time
All these can be done in ‘consistent’ fashion for any
algorithm
The Pieces So Far• Input – Accurate Specification
– Under what conditions should the algorithm be expected to give the correct solution
• Definitive Instructions– Clear and Unambiguous
• Correctness– The algorithm must correctly solve the problem
• Result Specification– What the algorithm is intended to do should be clear
• Termination – Execution Time– The algorithm must eventually stop for all valid inputs
– i.e. complete in finite time
InputDefinition
Correctness
Specification
Time
Physical Time
Has IssuesHighly Dependent on Factors of Implementation
May be difficult to compare one algorithm versus another
Marker Slide• Any Questions On:– Templates
• Introduction• Function Templates• Class Templates
– Algorithms• General• Execution Time: Physically Timing
• Next up– Algorithms
• Execution Time: Asymptotic Analysis: Big-Oh
– Big-Oh• Multiple Topics
Algorithms: How Big-Oh Fits• Input – Accurate Specification
– Under what conditions should the algorithm be expected to give the correct solution
• Definitive Instructions– Clear and Unambiguous
• Correctness– The algorithm must correctly solve the problem
• Result Specification– What the algorithm is intended to do should be clear
• Termination – Execution Time– The algorithm must eventually stop for all valid inputs
– i.e. complete in finite time
InputDefinition
Correctness
Specification
Time
Big-Oh
Termination of AlgorithmExecution Time
Analyzed Using Math
Math
The Problem to Solve
• Algorithms take time to run.
– We want to measure the time relative to data input size (i.e. n )
– Using a physical timer has lots of complications• Does not allow for easy comparison of algorithms
– Most interested in “worst case” time
for those thatmay have drifted off
Measuring Running Time• Running time of an algorithm grows with the input size
• Average case time is often difficult to determine
• We focus on the worst case running time– Crucial to applications such as games, finance and robotics– Easier to analyze
Run
ning
Tim
e
1ms
2ms
3ms
4ms
5ms
Input Instance
A B C D E F G
worst case
best case
emphasis point
Bounding Functions• Long ago in a galaxy called Calculus
– Functions were bounded• Not only could they be bound by constants
• But by other limiting functions
• It was a dark and mysterious place– Let’s go there =)
Big-Oh: Pre-math Concept
• Big-Oh of an algorithm is a function that estimates how the algorithm scales when it is used on different sized datasets– Big-Oh notation is written:• O( mathFunction ) = O( g(n) )
O( mathFunction )
• mathFunction, g(n), is usually a math formula based on the letters n and c– n is the input size (number of elements to be
processed by the algorithm)– c is some constant number
a function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large n.So, f(n) ≤ cg(n) for all n > no .
Math: Asymptotic Analysis
• Uses a high-level description of the algorithm instead of an implementation
• Characterizes running time as a function of the input size, n.
• Takes into account all possible inputs
• Allows us to evaluate the speed of an algorithm independent of the hardware/software environment
Marker Slide• Any Questions On:
– Templates• Introduction• Function Templates• Class Templates
– Algorithms• General• Execution Time: Physically Timing• Execution Time: Asymptotic Analysis: Big-Oh
• Next up– Big-Oh
• Linear Search• Seven Functions• Some rule details• Binary Search
Motivation: Linear Search
• Say you have 1,000 action figures– You keep them in an unorganized (but very cool)
looking pile on your otherwise unused bed
• A friend comes over and wants to see the super rare and totally awesome Boba Fett
• In the worst case how many action figures will you have to look at before you are guaranteed to have found the specified Boba Fett?
Motivation: Linear Search (cont)• Worst case: 1000
• If there were 10,000 action figures?– Worst case is 10,000
• If there were 100,000?– Worst case is 100,000
• So the input size (1000, 10000, 100000) is directly proportional to the worst case search time– This is described as O(n)
for k = 1 to 1000look at action figure number kstop if it is Boba Fett
for k = 1 to 10,000look at action figure number kstop if it is Boba Fett
for k = 1 to 100,000look at action figure number kstop if it is Boba Fett
for k = 1 to nlook at action figure number kstop if it is Boba Fett
Marker Slide• Any Questions On:
– Templates• Introduction• Function Templates• Class Templates
– Algorithms• General• Execution Time: Physically Timing• Execution Time: Asymptotic Analysis: Big-Oh
– Big-Oh• Linear Search
• Next up– Big-Oh
• Seven Functions• Some rule details• Binary Search
7 Functions
• Seven functions commonly used to examine the complexity of an algorithm are:– constant, log2n, n, n log2n, n2, n3, 2n
• These are listed in lowest complexity to highest complexity– More details follow
In this class the subscript 2 may be omitted:log2n log n but in the context of this class log n should be taken to mean log base 2 of n,unless explicitly stated otherwise.
You may also see log2n abbreviated lg nNote your calculator will not understand this
Base Conversion Reminder:
Common Big-Oh Functions• Seven functions that often
appear in algorithm analysis:– Constant 1– Logarithmic log n– Linear n– N-Log-N n log n– Quadratic n2
– Cubic n3
– Exponential 2n
1)(nLog
n
)(nLogn
Common Big-Oh Functions• Seven functions that often
appear in algorithm analysis:– Constant 1– Logarithmic log n– Linear n– N-Log-N n log n– Quadratic n2
– Cubic n3
– Exponential 2n
)(nLogn
2n
3nn2
Running Time Comparisons
* just too big of a number to calculate
Example: Linear Time O(n)• The linear search for action figures we already saw
was an example of an algorithm running in linear time– i.e. O(n)
• This run time is typical when using single for-loops: – for (i=1 to n) { do stuff }
• Further Example:– n = number of numbers to sum– Algorithm: sum=0; for k=1 to n; sum += num[k]
– Takes n time, every time = O(n)• for this example the ‘worst’ case occurs every time
Example: Quadratic Time O(n2)
• quadratic: O(n2)– Typically occurs when using nested for-loops:• for i=1 to n { for k=1 to n { do stuff } }
• Example, Create a Times Table– n = number of numbers to consider– for i=1 to n { for k=1 to n { product[i][k] =i*k } }
– Takes n2 time, every time = O(n2)• for this example the ‘worst’ case occurs every time
Example: Other Run Times
• As we go we will see more examples of algorithms with other run times
• So far O(n) and O(n^2) are enough for discussion of some rules
Marker Slide• Any Questions On:
– Templates• Introduction• Function Templates• Class Templates
– Algorithms• General• Execution Time: Physically Timing• Execution Time: Asymptotic Analysis: Big-Oh
– Big-Oh• Linear Search• Seven Functions
• Next up– Big-Oh
• Some rule details• Binary Search
Big-Oh: Some Details
• Consider the algorithm that has a nested for-loop on n items and then performs a single loop on the same number of items.
int main(){ double sum = 0.0; double product[5][5];
for (int i=0; i < 5; i++) { for (int k=0; k < 5; k++) { product[i][k] = i*k; }}
for (int k=0; k < 5; k++) { sum += k; }}
Big-Oh: Some Details
• What’s its Big Oh?– O(n2 + n) right?
int main(){ double sum = 0.0; double product[5][5];
for (int i=0; i < 5; i++) { for (int k=0; k < 5; k++) { product[i][k] = i*k; }}
for (int k=0; k < 5; k++) { sum += k; }}
n2
n
Big-Oh: Some Details
• What’s its Big Oh?– O(n2 + n) right?
– We only care about worst case
– So we throw out the lower complexity terms
int main(){ double sum = 0.0; double product[5][5];
for (int i=0; i < 5; i++) { for (int k=0; k < 5; k++) { product[i][k] = i*k; }}
for (int k=0; k < 5; k++) { sum += k; }}
n2
n
O(n2) is the correct answer
NO
[insert maniacal laugh]
Big-Oh: More Details• Dividing or multiplying by a
constant has no effect on the complexity of an algorithm.
– An algorithm consisting of a single for-loop that only processes half the items is NOT O(n/2).
– It is still O(n) because the growth of the algorithm is still linear• Doubling the number of items to
be processed still doubles the time required to complete the algorithm
int anAlg(int* numList, int size){ int sum = 0 for (int k=0; k < size / 2; k++) { sum += k; } return sum;}
n
Big-Oh: More Details• Dividing or multiplying by a
constant has no effect on the complexity of an algorithm.
– An algorithm consisting of a single for-loop that only processes half the items is NOT O(n/2).
– It is still O(n) because the growth of the algorithm is still linear• Doubling the number of items to
be processed still doubles the time required to complete the algorithm
int anAlg(int* numList, int size){ int sum = 0 for (int k=0; k < size / 2; k++) { sum += k; } return sum;}
n
Eat 1 of 2: Time = 30 seconds
Big-Oh: More Details• Dividing or multiplying by a
constant has no effect on the complexity of an algorithm.
– An algorithm consisting of a single for-loop that only processes half the items is NOT O(n/2).
– It is still O(n) because the growth of the algorithm is still linear• Doubling the number of items to
be processed still doubles the time required to complete the algorithm
int anAlg(int* numList, int size){ int sum = 0 for (int k=0; k < size / 2; k++) { sum += k; } return sum;}
n
Eat 1 of 2: Time = 30 secondsEat 2 of 4: Time = 60 seconds
Big-Oh: More Details• Dividing or multiplying by a
constant has no effect on the complexity of an algorithm.
– An algorithm consisting of a single for-loop that only processes half the items is NOT O(n/2).
– It is still O(n) because the growth of the algorithm is still linear• Doubling the number of items to
be processed still doubles the time required to complete the algorithm
int anAlg(int* numList, int size){ int sum = 0 for (int k=0; k < size / 2; k++) { sum += k; } return sum;}
n
Eat 1 of 2: Time = 30 secondsEat 2 of 4: Time = 60 secondsEat 4 of 8: Time = 120 seconds
Marker Slide• Any Questions On:
– Templates• Introduction• Function Templates• Class Templates
– Algorithms• General• Execution Time: Physically Timing• Execution Time: Asymptotic Analysis: Big-Oh
– Big-Oh• Linear Search• Seven Functions• Some rule details
• Next up– Big-Oh
• Binary Search
More Examples – More Motivation
• Why do we care how long an algorithm will take to run?– So we can• Generalize• Re-apply• Improve• Compare• Determine if we are using the RIGHT TOOL for the job
• Consider Dictionaries…
Dictionaries – Motivation
• Why are dictionaries alphabetized?– [ pause for student participation ]
Dictionaries – Motivation
• Why are dictionaries alphabetized?– To speed up looking for things.
Dictionaries – Motivation
• Why are dictionaries alphabetized?– To speed up looking for things.
• Why does alphabetizing them help?– [ Yet Another Pause ]• YAP…. YAP…. YAP
Dictionaries – Motivation
• Why are dictionaries alphabetized?– To speed up looking for things.
• Why does alphabetizing them help?– We can find things faster.– Using an Ordered List.
Dictionaries – Find Things Faster
• Using an ordered list helps us find things faster
– How does that work?
– [ YAP ] • Describe processes of using dictionary
How to Use a Dictionary• Keep cutting the search area in half
• Open dictionary to the middle– Then to ¼ or ¾ mark
• Then (1/8 or 3/8 mark) or (5/8 or 7/8)– Then …
Find Happiness
Discovered: Binary Search• By investigating dictionaries
– We discovered an orderingallows us to perform a binary search• We will see more on that search method later
– Which is “much faster” than searching an UN-ordered list
– By extrapolation• If given a list and asked to find many somethings• We may want to Sort it then Search it
• BUT… – How much faster is it really? How do we know?
• [see next slide]
Measuring How Much Faster?
• Asymptotic Execution Time• Big-Oh Notation
Asymptotic Execution Time• Suppose f(n) is a formula describing the exact
execution run time of some algorithm for an input size of n.
• That algorithm is then O( g(n) )• Big Oh of g of n
• If there exist constants c and n0 such that:• f(n) ≤ c*g(n), for all n > n0
Searching a dictionary is O( log n )Searching an unordered list is O( n )
For Example…
Big-Oh Notation• Given functions f(n) and g(n),
we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that
f(n) cg(n) for n n0
In other words:a function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large n.So, f(n) ≤ cg(n) for all n > no .
Big-Oh Notation• Given functions f(n) and g(n),
we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that
f(n) cg(n) for n n0
• Example: 2n + 10 is O(n)– 2n + 10 cn– (c 2) n 10– n 10/(c 2)– Pick c = 3 and n0 = 10 1
10
100
1,000
10,000
1 10 100 1,000n
3n
2n+10
n
Marker Slide• Any Questions on:
– Templates• Introduction• Function Templates• Class Templates
– Algorithms• General• Execution Time: Physically Timing• Execution Time: Asymptotic Analysis: Big-Oh
– Big-Oh• Linear Search• Seven Functions• Some rule details• Binary Search
• Next up– Free Play
Free Play – Things to Work On
• Graded In-Class Activity:• Homework 5• Homework 6
• Various In-Class Activities to revisit
The End
• Or is it?