Algorithms +

L. Grewe

Algorithms and Programs

Algorithm: a method or a process followed to solve a problem.

– A recipe.

An algorithm takes the input to a problem (function) and transforms it to the output.

– A mapping of input to output.

A problem can have many algorithms.

Algorithm Properties

An algorithm should (ideally) possess the following properties:

– It must be correct.

– It must be composed of a series of concrete steps.

– There can be no ambiguity as to which step will be performed next.

– It must be composed of a finite number of steps.

– It must terminate.

A computer program is an instance, or concrete representation, for an algorithm in some programming language.

Algorithm Efficiency

There are often many approaches (algorithms) to solve a problem. How do we choose between them?

At the heart of computer program design are two (sometimes conflicting) goals.

1. To design an algorithm that is easy to understand, code, debug.

2. To design an algorithm that makes efficient use of the computer’s resources.

5

Algorithms

Any problem can have a large number of algorithms that could be used to solve it.

……but there are some algorithms that are effecitve in solving many problems.

6

Common Algorithms / Algorithm Methods

•Greedy Algorithms

•Divide and Conquer

•Dynamic Programming

•Backtracking

•Branch and Bound

7

Other (common) Algorithms / Algorithm Methods

•Linear Programming

•Integer Programming

•Neural Networks

•Genetic Algorithms

•Simulated Annealing

Typically these are covered in application specific courses that use them (e.g. Artificial Intelligence)

8

Algorithms specific to a data structure

Also algorithms can be specifically designed for common operations on a particular data structure.

•Example - Graph Algorithms

•Graph matching

•Find shortest path(s)

9

Defining the Problem

Before even trying to design or reuse an existing algorithm, you must define your problem.

……many problems can be defined as an optimization problem.

10

Optimization ProblemProblem = Function to X + Constraints

Function to X

This is where you describe the problem as a formula/function. Example, “find the shortest path” can be stated as Sum (distances) is minimum. Here the X is to “minimize”. The Function is the “Sum(distances)”.

•Sometimes these are called “COST functions”

Constraints

In our shortest path problem this might be to never visit the same node in the path twice.

One Solution to any Problem…..The Brute

Force Algorithm

Exponential Time, because exponentially many

This is WHY we discuss algorithms!!!!

Try every solution!

•Instances: The possible inputs to the problem. •Solutions for Instance: Each instance has an exponentially large set of solutions. •Cost of Solution: Each solution has an easy to compute cost or value.

Specification•<preCond>: The input is one instance.•<postCond>: An valid solution with optimal cost. (minimum or maximum)

Optimization Problem Elements

13

•Class of algorithms that solve Optimization Problems in a “greedy way”

•Some greedy algorithms will generate an optimal solution, others only a “good (enough)” solution.

•Greedy Method = at each point in the algorithm a decision is make that is best at that point. Decisions made are not changed at a later point.

•Greedy Criterion = criterion used to make a greedy decision at each point.

Greedy Algorithms

14

Divide and Conquer

Problem = Set of Several Independent (smaller) sub-problems.

•Divide problem into several independent sub-problems and solve each sub-problem. Combine solutions to derive final solution.

•Can work well on parallel computers.

•Many times the sub-problems are the same problem and need to only develop one algorithm to solve them and then the algorithm to combine the results.

15

Divide and Conquer

Example:You are given a bag with 16 coins and told one is counterfeit and lighter than the others. Problem = determine if bag contains the counterfeit coin. You have a machine that compares the weight of two sets of coins and tells you which is lighter or if they are the same.

Starting Thoughts:You could start with comparing coin 1 and 2. If one is lighter you are done. You can then compare coin 3 and 4 and so on. If you have N coins this can take N/2 times.

The Divide and Conquer Way:If you have N coins divide into two N/2 groups. Weigh them. If one is lighter then we are done and the bag does contain a counterfeit coin. If they are the same, there is no counterfeit coin. This takes ONLY 1 operation.

What happens if you want to find the counterfeit coin?

16

Dynamic Programmingdynamic programming is a method of solving complex problems by breaking them down into simpler steps.

•Bottom-up dynamic programming simply means storing the results of certain calculations, which are then re-used later because the same calculation is a sub-problem in a larger calculation. •Top-down dynamic programming involves formulating a complex calculation as a recursive series of simpler calculations.

More on Dynamic Programming

Some programming languages can automatically memorize the result of a function call with a particular set of arguments

Some languages make it possible portably (e.g. Scheme, Common Lisp or Perl), some need special extensions (e.g. C++, see [2]). Some languages have automatic memoization built in. In any case, this is only possible for a referentially transparent function.

18

Backtracking Backtracking is a general algorithm for finding all (or some) solutions to some computational problem, that incrementally builds candidates to the solutions, and abandons each partial candidate c ("backtracks") as soon as it determines that c cannot possibly be completed to a valid solution

•problems which admit the concept of a "partial candidate solution“• and a relatively quick test of whether it can possibly be completed to a valid solution.

19

Branch and BoundDivides a problem to be solved into a number of subproblems, similar to the strategy backtracking. Systematic enumeration of all candidate solutions, where large subsets of fruitless candidates are discarded en masse, by using upper and lower estimated bounds of the quantity being optimized

•Efficiency of the method depends strongly on the node-splitting procedure and on the upper and lower bound estimators. All other things being equal, it is best to choose a splitting method that provides non-overlapping subsets.