the fundamentals: algorithms, integers, & matrices
DESCRIPTION
The Fundamentals: Algorithms, Integers, & Matrices. Chapter 2. 2.2 Growth of Functions. Big-O notation. A tool to analyze a program's efficiency. An approximation of work an algorithm performs as a function of size of input Work = O(input size). Work. A measure of - PowerPoint PPT PresentationTRANSCRIPT
The Fundamentals: Algorithms, Integers, & Matrices
Chapter 2
2.2 Growth of Functions
Big-O notation
A tool to analyze a program's efficiency.
An approximation of work an algorithm performs as a function of size of input
Work = O(input size)
Work
A measure of effort expended by computer in performing a computation
Commonly used Big-O values (orders of magnitude)
Big-O Name O(1) Constant time LowerO(log n) Logarithmic time O(n) Linear time O(n log n) O(n2) Quadratic time
O(n3) Cubic time O(nk) Polynomial time
O(2n) Exponential time HigherO(n!) Factorial time
Table of Common Running Times
N Linear N Log N Quadratic Cubic
10 10 10.0 100 100020 20 26.0 400 800030 30 44.3 900 2700040 40 64.0 1600 6400050 50 84.9 2500 12500060 60 106.7 3600 21600070 70 129.2 4900 34300080 80 152.2 6100 48800090 90 175.9 8100 729000
100 100 200.0 10000 1000000
Growth Rate of Some Functions
0100200300400500600700800900
1000
0 10 20 30 40 50 60 70 80 90 100
LinearnLognQuadraticCubic
Big-O
f(x) is O(g(x))if there are constants C and k such that
| f(x) | C | g(x) |
whenever x > k
Show f(x) = x2+2x+1 is O(x2)
Prove: |x2+2x+1| C | x2 |
• If we let k > 0, f(x) & g(x) are always positive, we can drop | |
Let C be the sum of f(x) coefficients x2+2x+1 1x2+2x2+1x2 = 4x2
» Note: x2 1x2 and 2x 2x2 and 11x2
» This inequality holds beginning at x=1: 44
Choosing k=1, C=4, x2+2x+1 is O(x2)
Show f(x) = 7x3 is O(x2)
Prove: |7x3| C | x2 |
Assuming k > 0, divide both sides by x2
7x C
» You choose a constant > any y on y=7x line» No such C exists, since x is arbitrarily large» (Choose any C, and there is an x > C)
7x3 is not O(x2)
Big-O Example algorithm
O(1) Assigning value to ith array element;
always the same number of steps
not necessarily short
a program which takes 320 steps,
reguardless of input values
O(log n) Binary search,
What power of 2 is greater than a input number?
A loop whose terminal value is being successively halved or doubled
Big-O Example algorithm
O(n) Printing all elements of an array; searching an unordered array; A loop which executes 1 to N times (where N is the input size, the number of data values being
processed)
O(n log n) Faster sorts; A loop whose index is being halved/doubled inside a loop executing from 1 to N
Big-O Example algorithm
O(n2) Slower sorts; A loop which executes from 1 to N times inside a loop executing from 1 to N
times
O(n3) Incrementing all the elements in a NxNxN array; A 1..N loop inside a 1..N loop inside a 1..N loop
O(nk) k levels of loops inside loops
Big-O Example algorithm
O(2n) List all the subsets of a set with n elements; practical only with small values of n; n=64 takes 5 years on a supercomputer
O(n!) List all the possible arrangements of a set with n elements
Big-O General Rules
a.) Multiplicative constants do not matter. O(3n) = O(n)
b.) Addition is done by taking the max. O(n2) + O(n) = O(max(n2, n)) = O(n2) O(n3) + O(log n) = O(max(n3, log n)) = O(n3)
c.) Multiplication remains multiplication. O(n) * O(log n) = O(n log n) O(n) * O(n2) = O(n3)
d.) Lower is always O(higher) for nbut closest is desired
f(n2) = O(n3)
How can a program with many control structures be categorized by only one?
A certain country taxes pet stores based only on the combined weight of all the pets.
Only two pets thrive in the country: parakeets and elephants
When reporting their taxes, pet shop owners generally only report the weight of the _____.
Example Complexities
4n3 + 20n + 30 = O(n3) n + 10000 = O(n) 4n4 + 20n + 30 = O(n4) 2n + n3 = O(2n) 200 = O(1)
Running Time of Statements Simple statements are: O(1)
initialization of variables; input/ouput
Loops are: O(g(n)f(n)) where g(n) is upper bound on number of loop iterations & f(n) is upper bound on the body of the loop If g(n) and f(n) are constant,
then this is constant time
Running Time of Statements (continued)
Conditional statements are
O(max(f(n),g(n))) where f(n) is upper bound on then part and g(n) is upper bound on else part
Blocks of statements with complexities:
f1(n), f2(n), ..,fk(n),
have complexity O(f1(n) + f2(n)+ ...+ fk(n))
Simple Analysis
cin >> n; // 1if (n > 20) // 1
cout << “Number is > 20” << endl; // 2else
cout << “Number is <= 20” << endl; // 2
T(n) = 2 + max (2,2) = 4 = O(1)
Example Analysis
cin >> n; // 1factorial = 1; // 1for (i = 2; i<=n; i++) // 1 initialization +1 test +
//(1 test + 1 inc) per iterationfactorial *= i; // 1
cout << factorial; // 1
T(n) = 5 + 3*n = O(n)
Another example
cin >> n; // 1if (n > 0) { // 1 factorial = 1; // 1 for (i = 2; i <=n; i++) // 1 initialization +1 test +
//(1 test + 1 inc) per iterationfactorial *= i; // 1
cout << factorial; // 1}else cout << “Can’t do factorial of” << n; // 2
T(n) = 2 + max(4+3*n, 2) = O(n)
Analysis of simple function callsint factorial(int n){ int fact=1,i; for (i = 2; i <=n; i++)
fact *= i; return fact;}
void main(){ int n; // 1
cin >> n; // 1 cout << factorial(n) << endl; //O(n)}
Main is O(n)
Analysis of Nested Loops
EXAMPLE 1:
for (int i = 0; i < n; i++) // n*O(n) = O(n2) for (int j=0; j<n;j++) // n*O(1) = O(n) x++; // O(1)
EXAMPLE 2:
for (int i = 0; i < n; i++) for (int j=i; j<n;j++) x++; // O(1)
T(n) = n+(n-1) + (n-2)+…+ 1 = n(n+1)/2 = O(n2)
Summing blocks of statements
for (int i = 0; i < n; i++) // O(n2) for (int j=0; j<n;j++) x++;
for (int i = 0; i < n; i++) // O(n) x++;
T(n) = O(n2) + O(n) = O(n2 + n) = O(n2)
Loops with different limits
for (int i = 0; i < n; i++) // n*O(m) = O(mn) for (int j=0; j<m; j++) // m*O(1) = O(m) x++; // O(1)
O(mn)
More complex loops
for (i = 1;i < n; i *=2) x++;
for (i=n; i > 0; i/=2) x++;
Repetitive halving or doubling results in logarithmic complexity. Both loops are O(log n)
The Integers and Division
2.4
Divides (|), Prime Number theory - the study of integers and their properties a | b (a divides b) - integer c where ac = b
3 | 12 since 12 = 3 * 4
Prime - an integer > 1 whose only factors are the number and 1 Composite - an integer > 1 that is not prime
Fundamental theorem of arithmetic Every positive int. can be written as a product of primes 100 = 2 * 2 * 5 * 5
If n is composite, it has a prime divisor n
Divisibility Tests
A number is divisible by 2 if its unit’s digit is divisible by 2. A number is divisible by 3 if the sum of the digits is divisible by 3. A number is divisible by 4 if the # made of the last two digits is div by 4. A number is divisible by 5 if its unit’s digit is divisible by 5. A number is divisible by 6 if its is divisible by 2 and 3. A number is divisible by 8 if the # made of the last 3 digits is div by 8. A number is divisible by 9 if the sum of the digits is divisible by 9.
The Division Algorithm
Given integer a and positive integer d. integers q,r with 0 rd, such that a = dq
+ r d is divisor, r remainder, q quotient, a dividend
quotient remainder divisor dividend
101 divided by 11 == quotient 9, remainder 2101 = 11* 9 + 2
-11 divided by 3 == quotient -4, remainder 1-11 = 3(-4) + 1
Greatest Common Divisor gcd(a,b) = largest integer d, such that d | a and d | b
gcd(24,36) = 12
Algorithm for finding gcd(a,b) Express a, b as powers of products of primes The products of lowest powers of all distinct primes is gcd
(intersection)
gcd(252,420) 252 = 22 32 7 420 = 22 3 5 7 gcd(252,420) = 22 3 7
= 84
Relatively Prime Two integers are relatively prime if their gcd is 1
17 and 22 are r.p. 6 and 22 are not r.p.
The integers a1, a2, …, an are pairwise relatively prime if
gcd(ai,aj) = 1, whenever 1 ijn 10, 17 and 21 are p.r.p. 10, 17 and 22 are not p.r.p. because gcd(10,22) = 2
Least Common Multiple lcm(a,b) = smallest positive integer that is divisible by both a and b
Finding lcm(a,b) Express a, b as powers of products of primes The products of highest powers of all distinct primes is lcm (union)
lcm(252,420) 252 = 22 32 7 420 = 22 3 5 7 gcd(252,420) = 22 32 5 7
= 1260
NOTE: ab = gcd(a,b) lcm(a,b)
Modular Arithmetic a mod m
the remainder when a is divided by m
17 mod 5 = 2
2001 mod 101 = 82 19 r 82
What about negative operands?-17 mod 5 The answer in C++ is –2 (-3 –2/5)
Text answer would be 3 (-4 + 3/5)
Mod 0 is not defined
Congruence a b (mod m) a Is Congruent to b modulo m
a mod m = b mod mOR
m divides a-b
17 5 (mod 6) is true6 divides (17 - 5) 17 mod 6 = 5 5 mod 6 = 5
24 14 (mod 6) is false6 does not divide (24 - 14) 24 mod 6 = 0 14 mod 6 = 2
Hashing
Applying a function to a key value mapping range of possible key values into a smaller range of addresses
h(k) = k mod m h(064212848) = 064212848 mod 111 = 14 h(037149212) = 037149212 mod 111 = 65
Cryptology
The study of secret messages
Caesar cipher f (p) = (p + 3) mod 26
“MEET YOU IN THE PARK” becomes“PHHW BRX LQ WKH SDUN”
Security Now Podcast
Leo LaPorte, twit.tv, leoville.com
Steve Gibson, grc.com
Decoder Ring (SN-31)
Basic Terminology• plaintext - the original message
• ciphertext - the coded message
• cipher - algorithm for transforming plaintext to ciphertext
• key - info used in cipher known only to sender/receiver
• encipher (encrypt) - converting plaintext to ciphertext
• decipher (decrypt) - recovering ciphertext from plaintext
• cryptography - study of encryption principles/methods
• cryptanalysis (codebreaking) - the study of principles/ methods of deciphering ciphertext without knowing key
• cryptology - the field of both cryptography and cryptanalysis
Cryptology
The study of secret messages
Caesar cipher f (p) = (p + 3) mod 26
“MEET YOU IN THE PARK” becomes“PHHW BRX LQ WKH SDUN”
English Letter Frequencies
Letter Pair Frequencies
For instance, given a section of English language, E tends to be very common, while X is very rare. Likewise, ST, NG, TH, and QU are common pairs of letters (termed bigrams or digraphs), while NZ and QJ are rare. The mnemonics phrase "ETAOIN SHRDLU" encodes the 12 most frequent letters in typical English language text.
EXCLUSIVE OR
The EXCLUSIVE OR is true when
exactly one of its operands is true.
p q p q
____________________
F F F
F T T
T F T
T T F
Reversible XOR
1100 plaintext
1010 encrypt
-----
0110 ciphertext
1010 decrypt
-----
1100 plaintext
Ch 7 / Foil 47
Random Numbers
Generating a sequence of random numbers is often useful In a game, it ensures that a player does not see
the same behavior each time In a simulation of a complex system,
random numbers can be used tohelp generate random events
– Car crash in a simulationof a highway system
– Likelihood of a gene in cell mutation– Weather simulation
Ch 7 / Foil 48
Uniform Random Numbers
Uniform random number sequence A sequence of random numbers where
– Each value in the sequence is drawn from the same range of numbers
– In each position of the sequence, any value in the number range is equally likely to occur
Ch 7 / Foil 49
Random Numbers
Examples Generate a uniform random
number sequence in the range1 to 6
– Use a fair six-sided dice– Each roll represents a new random number
Do two die produce uniform random numbers in the range 1 ... 12?
Generate a uniform random numbersequence in the range 1 to 2
– Use a fair coin Heads: 1, Tails: 2
Ch 7 / Foil 50
Random Numbers
We can write an algorithmfor generating what lookslike random numbers
Because it’s an algorithm,we know the rules for generating the next number The generated numbers are not really random
– They are properly called pseudorandom numbers
30 21 9 28 29 ...
Ch 7 / Foil 51
Stdlib Library
Provides in part an interface to functions that generate pseudorandom numbers rand() -- returns a uniform pseudorandom number (unsigned int)
from the inclusive interval 0 to RAND_MAX Consider rand.cpp
#include <iostream>#include <string>using namespace std;#include <stdlib.h>int main() { for (int i = 1; i <= 5; ++i)
cout << rand() << endl; return 0;
}
Ch 7 / Foil 52
Different Sequences
To produce a different sequence, invoke
void srand(unsigned int); Consider seed.cpp
int main() { cout << "Enter a seed: "; unsigned int Seed; cin >> Seed; srand(Seed); for (int i = 1; i <= 5; ++i) cout << rand() << endl; return 0;
}
Ch 7 / Foil 53
Different Sequences
To get a different sequence each time Need a method of setting the seed to a random value
– The standard method is to use the computer's clock as the value of the seed
– The function invocation time() can be used Returns an integral value of type time_t Invocation time(0) returns a suitable value for
generating a random sequence
Ch 7 / Foil 54
Randseed.cpp
#include <iostream>#include <string>using namespace std;#include <stdlib.h>#include <time.h>
int main() { srand((unsigned int) time(0)); for (int i = 1; i <= 5; ++i)
cout << rand() << endl; return 0;
}
Symmetric Key
Symmetric-key algorithms are a class of algorithms for cryptography that use trivially related, often identical, cryptographic keys for both decryption and encryption.
Diffie-Hellman Key Exchange
We both agree on a public key a I raise ab for some large integer b You raise ac for integer c We exchange values You raise ab to the c power I raise ac to the b power We both have abc for our session key (RNG seed)
No one eavesdropping can perform log ab or log ac for large values of a, b, c
Security Now 34, 7:30mins-11:30
RSA Encryption C = Me mod n
C = encrypted message M = plaintext message (converted to #s via ASCII) n is product of two primes pq e is exponent relatively prime to (p-1)(q-1) (public key)
The formula is invertible if p, q are known So, huge prime values are chosen for p,q (200 digits) n cannot be factored in a reasonable amount of time (2 billion years of computer time) Receiver of message is given decryption key: p & q
RSA Demo
RSA Encryption The RSA algorithm, invented in 1977 by Ron Rivest, Adi Shamir, and Leonard
Adleman, is used frequently for public-key encryption and decryption. It works as follows: Take two large prime numbers, p and q, and find their
product n = pq; n is called the modulus. Choose a number, e, less than n and relatively prime to (p1)(q1), and find d, an inverse of e modulo (p1)(q1), which means that ed 1 mod (p1)(q1);
e and d are called the public and private exponents, respectively. The public key is the pair (n,e); the private key is d. The factors p and q must be kept secret, or destroyed. It is difficult (presumably) to obtain the private key d from the public key (n,e). If one could factor n into p and q (which can be difficult for large values of p and q) however, then one could obtain the private key d.
RSA Demo
Matrices
2.6
Matrix
A rectangular array of numbers
m n matrix m rows n columns
If m=n, the matrix is square Two matrices are equal if
they have same number of rows the same number of columns the corresponding entries in every position are equal
A is an m n matrix
a a a
a a a
a a a
n
n
m m mn
11 12 1
21 22 2
1 2
..
..
.. .. .. ..
..
aij
The element of matrix A on the ith row the jth column
A = [a ij] shorthand for denoting matrix A
Sum of matrices
A + B = [aij + bij]
1 0 1
2 2 3
3 4 0
3 4 1
1 3 0
1 1 2
4 4 2
3 1 3
2 5 2
Matrix Multiplication A is a m k matrix, B is a k n matrix
The product of A and B the sum of products of corresponding elements from ith row of A and jth column of B
AB = [cij]
cij = ai1b1j + ai2b2j + ai3b3j + … + aikbkj a bi x x j
x
k
1
Multiplication Example
1 0 4
2 1 1
3 1 0
0 2 2
2 4
1 1
3 0
2 0 12 4 0 0
4 1 3 8 1 0
6 1 0 12 1 0
0 2 6 0 2 0
c11 c12
c41 c42
Is Matrix Multiplication Commutative?
Does AB = BA? A B
1 1
2 1
2 1
1 1
AB
1 1
2 1
2 1
1 1
2 1 1 1
4 1 2 1
3 2
5 3
BA
2 1
1 1
1 1
2 1
2 2 2 1
1 2 1 1
4 3
3 2
The Transpose of Matrix A
A is an m n matrix
AT is an n m matrix obtained by interchanging the rows and cols of A
AT = [bij], bij = aji
A AT
7 2 4
1 0 9
7 1
2 0
4 9
Symmetric Matrix
A square matrix A is symmetic if A=AT
aij = aji for all i,j
1 1 0
1 0 1
0 1 1
a1,2
a2,1
The Identity Matrix of Order n
An n n matrix main diagonal is all 1 every other element is 0
A In = ImA = A
I I2 3
1 0
0 1
1 0 0
0 1 0
0 0 1
Powers for square matrices
Given A is an n n matrix
A0 = In
Ar = A A A A
r times
Zero-One (Logical) Matrices
Elements are 0/1 representing False/True
The join of logical matrices A and B aij bij
The meet of logical matrices A and B aij bij
Join and Meet of A and B
A B
1 0 1
0 1 0
0 1 0
1 1 0
A B A B v ^
1 1 1
1 1 0
0 0 0
0 1 0
Boolean Product A is a m k logical matrix, B is a k n logical matrix
The boolean product of logical matrices A and B
A B = [cij]
cij = (ai1 b1j) (ai2 b2j) (ai3 b3j) … (aik bkj)
Boolean Product Example
A B
1 0
0 1
1 0
1 1 0
0 1 1
A B
1 1 0
0 1 1
1 1 0
Boolean Powers
Given A is an n n logical matrix
A[0] = In
A[r] = A A A A
r times
Given A, find A2, A3
A[2] = A A
A[3] = A[2] A
A
0 0 1
1 0 0
1 1 0 0 0 1
1 0 0
1 1 0
0 0 1
1 0 0
1 1 0
1 1 0
0 0 1
1 0 1
o
1 1 0
0 0 1
1 0 1
0 0 1
1 0 0
1 1 0
1 0 1
1 1 0
1 1 1
o
