data structures and algorithms introduction to algorithms m. b. fayek cufe 2006

Data Structures and Data Structures and Algorithms Algorithms

Introduction to Introduction to AlgorithmsAlgorithms

M. B. FayekM. B. Fayek

CUFE 2006CUFE 2006

AgendaAgenda

1.1. Introduction to AlgorithmsIntroduction to Algorithms

2.2. Algorithm Design BasicsAlgorithm Design Basics

3.3. Time Complexity of an Time Complexity of an AlgorithmAlgorithm

4.4. Analysis using Asymptotic Analysis using Asymptotic NotationNotation

5.5. Basic Efficiency ClassesBasic Efficiency Classes

1. Introduction to 1. Introduction to AlgorithmsAlgorithms

WhyWhy study Algorithms? study Algorithms? Heart of computer Heart of computer Promote analytical skillsPromote analytical skills Donald Knuth:Donald Knuth:

“ “ A person does not understand A person does not understand something until after teaching it to something until after teaching it to someone else.someone else.

Actually: A person does not understand Actually: A person does not understand something until after he teaches it to a something until after he teaches it to a …………COMPUTERCOMPUTER!”!”

1. Introduction to 1. Introduction to Algorithms Algorithms

WhatWhat is an algorithm? is an algorithm? ““A sequence of A sequence of unambiguous unambiguous

instructionsinstructions for solving a problem, for solving a problem, i.e., for obtaining a i.e., for obtaining a requiredrequired output output for any for any legitimate inputlegitimate input in a in a finite finite amount of timeamount of time.”.”

1. Introduction to 1. Introduction to Algorithms Algorithms

Algorithms are all around us in Algorithms are all around us in everyday life. everyday life.

In the recipe of a cook book. In the recipe of a cook book. In assembling a toy. In assembling a toy. In setting the table. In setting the table. In preparing a cup of tea. In preparing a cup of tea. In calling your friend on the phone.In calling your friend on the phone. ……. There are countless examples! . There are countless examples!

AgendaAgenda






2. Algorithm Design 2. Algorithm Design BasicsBasics

Guidelines for Algorithm Designing Guidelines for Algorithm Designing and Analysis:and Analysis:

1.1. Understand the ProblemUnderstand the Problem ((requirement analysis)requirement analysis)

2.2. Select Data structureSelect Data structure

3.3. Write Pseudo CodeWrite Pseudo Code

4.4. Analyze PerformanceAnalyze Performance

5.5. Implement using suitable Implement using suitable programming languageprogramming language

6.6. Test to resolve syntax and logic Test to resolve syntax and logic errorserrors



1.1. Understand the ProblemUnderstand the Problem ((requirement analysis)requirement analysis)

Gather dataGather data Ask usersAsk users Carefully review any written Carefully review any written

requirementsrequirements



2.2. Select Data structure: Select Data structure:

To verify the appropriateness of To verify the appropriateness of the selected data structure:the selected data structure:

Judge how well your data structure Judge how well your data structure responds to user requirements responds to user requirements (updates, questions)(updates, questions)

Modify design as necessaryModify design as necessary



3.3. Write Pseudo CodeWrite Pseudo Code Use pseudo code or flow chartUse pseudo code or flow chart level of details of the pseudo code level of details of the pseudo code

may varymay vary



4.4. Analyze PerformanceAnalyze Performance Determine the feasibility of solution Determine the feasibility of solution

w.r.t. memory requirements, w.r.t. memory requirements, performance constraints … etc.performance constraints … etc.

Manually review and validate pseudo Manually review and validate pseudo codecode

Analyze the complexity of the Analyze the complexity of the algorithm using the algorithm using the big Obig O notation to notation to determine the determine the complexity w.r.t. to complexity w.r.t. to time and storagetime and storage..

Other criteria include: Clarity, Other criteria include: Clarity, Maintainability, PortabilityMaintainability, Portability



5.5. Implement Implement using suitable using suitable programming language: programming language:

Is rather direct provided a good Is rather direct provided a good pseudo code is written, however pseudo code is written, however erroneous mapping may occur!erroneous mapping may occur!



6.6. TestTest to resolve syntax and logic to resolve syntax and logic errors.errors.

Testing is divided into 2 main Testing is divided into 2 main parts: parts:

Trying to break the function of the Trying to break the function of the program by entering unexpected data.program by entering unexpected data.

Debugging: It is concerned with Debugging: It is concerned with finding out what is what caused the finding out what is what caused the program to function in incorrectly. program to function in incorrectly.

AgendaAgenda






3. Time Complexity of an 3. Time Complexity of an AlgorithmAlgorithm

Time complexity is a main issue in Time complexity is a main issue in evaluating evaluating an algorithm. an algorithm.

It reflects It reflects how the algorithm how the algorithm responds to the increase in data responds to the increase in data size (n)size (n) it handles, by measuring the it handles, by measuring the corresponding increase in number of corresponding increase in number of instructions to be performed. instructions to be performed.

Time complexity is meant to Time complexity is meant to classify algorithmsclassify algorithms into categories. into categories.

3. Time Complexity of an 3. Time Complexity of an AlgorithmAlgorithm

Steps to determine time Steps to determine time complexity:complexity:

1.1. Identify Identify basic operationsbasic operations used for evaluating used for evaluating complexity.complexity.

2.2. Usually, Usually, loopsloops and nested loops are the and nested loops are the significant parts of a program. significant parts of a program.

3.3. One iteration of the loop is considered as a unit. One iteration of the loop is considered as a unit. It is then important to determine the It is then important to determine the order of order of magnitudemagnitude of run time involved based on the of run time involved based on the number of iterations. number of iterations.

4.4. Parts concerned with Parts concerned with initializationsinitializations and and reportingreporting summary results are of secondary summary results are of secondary importance. importance.

Example 1Example 1Example1: count number of different chars in a file:Example1: count number of different chars in a file:// Input: string of characters// Input: string of characters // Output: array of 256 integers containing the count of // Output: array of 256 integers containing the count of

the 256 charsthe 256 chars

For all 256 char doFor all 256 char doassign zero to counterassign zero to counter

end forend for

while there are more chars dowhile there are more chars doget next charget next charincrement count for this charincrement count for this char

end whileend while

N+1 Eof check

+ N count increment

257 assignment

+ 256 increment

+ 257 bound check

Bookkeeping!

Example 1Example 1

Consider basic operation BOP = 1 Consider basic operation BOP = 1 Inc + 1 check

for N= 500, integral BOP ≈ 500for N= 500, integral BOP ≈ 500

bookkeeping ≈ 50 %bookkeeping ≈ 50 % For N =10000, BOP ≈ 10000For N =10000, BOP ≈ 10000

bookkeeping = 7 %bookkeeping = 7 %

Total =

257 assign + 256 Inc + 258 check (bookkeeping)

+ N Inc + N+1 check (integral)

Time grows linearly with n!

Example 2Example 2

Summing each of the rows of an N x Summing each of the rows of an N x N two-dimensional array A, storing N two-dimensional array A, storing row sums in a one-dimensional array row sums in a one-dimensional array Sum and the overall in GrandTotal.Sum and the overall in GrandTotal.

// // InputInput: 2-dimensional array A: 2-dimensional array A

// // OutputOutput: 1- dimensional array Sum : 1- dimensional array Sum that contains the sum of 1that contains the sum of 1stst, 2, 2ndnd,… ,… row in 1row in 1stst, 2, 2ndnd, … element and , … element and GrandTotal that contains the total GrandTotal that contains the total sum sum

Example 2Example 2

First Algorithm : First Algorithm :

GrandTotal = 0GrandTotal = 0

For k = 1 to N doFor k = 1 to N do

Sum [K] = 0Sum [K] = 0

For J = 1 to N doFor J = 1 to N do

Sum [K] = Sum [K] + A Sum [K] = Sum [K] + A [K,J][K,J]

GrandTotal = GrandTotal + GrandTotal = GrandTotal + A [K,J]A [K,J]

EndForEndFor

EndForEndFor

► complexity = n2 !

Basic Operation

Example 2Example 2

Second Algorithm : Second Algorithm : GrandTotal = 0GrandTotal = 0For k = 1 to N doFor k = 1 to N do

Sum [K] = 0Sum [K] = 0For J = 1 to N doFor J = 1 to N do

Sum [K] = Sum [K] + A Sum [K] = Sum [K] + A [K,J][K,J]

EndForEndForGrandTotal = GrandTotal + A GrandTotal = GrandTotal + A

[K,J][K,J]EndForEndFor

► complexity = n2

Basic Operation 1

Basic Operation 2

► complexity = n

Example 2Example 2 Total complexity= nTotal complexity= n22 + n + n Hence Total ComplexityHence Total Complexity mainly mainly

depends on depends on nn22 ►►Note however that:Note however that: In the first approach the basic In the first approach the basic

operation includes 2 additionsoperation includes 2 additions In the second approach In the second approach eacheach basic basic

operation includes 1 additionoperation includes 1 addition Hence, BOTH approaches are of Hence, BOTH approaches are of

order norder n22, however, the second is , however, the second is better! better!

……. Why?. Why?

AgendaAgenda






4. Analysis using Asymptotic 4. Analysis using Asymptotic NotationNotation

Big Oh:Big Oh: IfIf

Then there exists c such thatThen there exists c such that

)( fg

)()( ncfng


Find Big OhFind Big Oh

by finding f(n) such thatby finding f(n) such that

)( fg

cnf

ngn

)(

)(lim

for some real value

*Rc


A Useful PropertyA Useful Property:: If If

andand

Then Then

))(()( 11 ngOnt

))(()( 22 ngOnt

)})(),((max{)()( 2122 ngngOntnt

AgendaAgenda






5. Basic Efficiency Classes5. Basic Efficiency Classes

Common complexities Common complexities (ordered in ascending order) (ordered in ascending order) are:are:

log nlog n nn n log nn log n nn22

nn33

22nn

Complexity

5. Basic Efficiency Classes5. Basic Efficiency Classes

Values of important asymptotic Values of important asymptotic functionsfunctionsn n2log nn 2logn 2n 3n n2 !n

Example 1Example 1

Find the number of binary digits in the Find the number of binary digits in the binary representation of a number binary representation of a number

// input: A is a positive number// input: A is a positive number

// output: number of bin digits that represent A// output: number of bin digits that represent A

count <-- 1count <-- 1

while n > 1 dowhile n > 1 do

count <-- count + 1count <-- count + 1

n <-- n/2n <-- n/2

return countreturn count

Example 2: Example 2: Element Element UniquenessUniqueness

Element Uniqueness Problem:Element Uniqueness Problem: //Check whether all elements of array are //Check whether all elements of array are

uniqueunique // input: array A[0, .. n-1]// input: array A[0, .. n-1] // output true or false// output true or false

for i <-- 0 to n-2 do for i <-- 0 to n-2 do for j <-- i+1 to n-1 dofor j <-- i+1 to n-1 do

if A[i] = A[j] return falseif A[i] = A[j] return falsereturn truereturn true

► ► Efficiency does not depend on n Efficiency does not depend on n only!only!

There is aThere is a best, worst and average best, worst and average case case efficiency! efficiency!

…… …… When do those cases When do those cases occur?occur?

Example 3: Example 3: Fibonacci Fibonacci NumbersNumbers

It is a series described by: It is a series described by:

FFii = F = Fi-1i-1+ F+ Fi-2i-2 and F(0) =0, F(1) =1 and F(0) =0, F(1) =1

// Input: A nonnegative integer n// Input: A nonnegative integer n

// Output: The n// Output: The nthth Fibonacci number Fibonacci number

F[0] <-- 0; F[1] <-- 1F[0] <-- 0; F[1] <-- 1

for i <-- 2 to n dofor i <-- 2 to n do

F[i] <-- F[i-1] + F[i-2]F[i] <-- F[i-1] + F[i-2]

return F[n]return F[n]

►►EXTRA ARRAY STORAGEEXTRA ARRAY STORAGE, necessary?, necessary?

ExercisesExercises

Write an implementation of the Write an implementation of the fibonacci example fibonacci example withoutwithout using and using and additional array!additional array!

Check if a number is prime or notCheck if a number is prime or not Find all prime numbers in a Find all prime numbers in a

specified rangespecified range Find the GCD of 2 numbersFind the GCD of 2 numbers

Website atWebsite at

http://http://elearning.eng.cu.edu.egelearning.eng.cu.edu.eg

data structures and algorithms introduction to algorithms m. b. fayek cufe 2006

Documents

questionsmodify design

algorithms algorithms

problem requirement

algorithms introduction

pseudo codeuse pseudo

algorithmtime complexity

selected data structure

unexpected data