Do We Teach the Right Algorithm Design Techniques ? Anany LevitinACM SIGCSE 1999

OutlineIntroductionFour General Design TechniquesA Test of Generality Further RefinementsConclusion

Introduction-1According to many textbooks, a consensus

seems to have evolved as to which approaches qualify as major techniques for designing algorithms.

This is includes: divide-and-conquer, greedy approach, dynamic programming, backtracking, and branch-and-bound.

Introduction-2However, this widely accepted taxonomy has

serious shortcoming. First, it includes techniques of different levels

of generality. For example, it seems obvious that divide-and-conquer is more general than greedy approach and branch-and-bound.

Second, it fails to distinguish divide-and-conquer and decrease-and-conquer.

Third, it fails to include brute force and transform-and-conquer.

Introduction-3Fourth, its linear, as opposed to hierarchical,

structure fails to reflect important special cases of techniques.

Finally, it fails to classify many classical algorithms (e.g., Euclid’s algorithm, heapsort, search trees, hashing, etc.)

This paper seeks to rectify these shortcomings and presents new taxonomy.

Four General Design Techniques-1Brute Force

It usually based on the problem’s statement and definitions of concepts involved

This technique can not be overlooked by the following reasons Applicable to a very wide variety of problems, e.g.,

computing the sum of n numbers, adding two matrices ...

Useful for solving small-size instances of a problem Serving an important theoretical or educational

purpose, e.g., NP-hard problem, as a yardstick for more efficient alternatives for solving a problem

Four General Design Techniques-2Divide-and-conquer

It based on partitioning a problem into a number of smaller subproblems, usually of the same kind and ideally of about the same size

The subproblems are then solved and their solutions combined to get a solution to the original problem. Divide-before-processing: the bulk of the work is done

while combining solutions to smaller subproblems, e.g., mergesort.

Process-before-dividing: processing before a partition into subproblems, e.g., quicksort.

Four General Design Techniques-3Decrease-and-conquer

This technique is solving a problem by reducing its instance to a smaller one, solving the latter, and then extending the obtained solution to get a solution to the original instance decrease by a constant: insertion sort decrease by a constant factor(a.k.a. prune-and-

search): binary search variable size decrease: Euclid’s algorithm

Four General Design Techniques-4Transform-and-conquer

This technique is based on the idea of transformation simplification: solves a problem by first

transforming its instance to another instance of the same problem which makes the problem easier to solve, e.g., Gaussian elimination, heapsort ...

representation change: it is based on a transformation of a problem’s input to a different representation, e.g., hashing, heapsort …

Four General Design Techniques-4 cont. preprossing: The idea is to process a part of the

input or the entire input to get some auxiliary information which speeds up solving the problem, e.g., KMP algorithms.

reduction: An instance of a problem is transformed to an instance of a different problem altogether, e.g, NP-hard problems.

A Test of Generality-1We partition design techniques into two

categories: more general and less general techniques.

How to make such a determinate ? We would like to suggest the following test. In order to qualify for inclusion in the category of most general approaches, a technique must yield reasonable algorithms for the two problems: sorting and searching.

A Test of Generality-2Sorting Searching

Brute force Selection sort Sequential searchDivide-and-conquer Mergesort ApplicableDecrease-and-conquer

Insertion sort Applicable

Transform-and-conquer

Heapsort Search trees, hashing

The others - greedy approach, dynamic programming, backtracking, and branch-and-bound - fail to qualify as the most general design techniques.

Further Refinements-1Local search techniques

Greedy methods: it builds solutions piece by piece … the choice selected is that which produces the largest immediate gain while maintaining feasibility, e.g., Prim’s algorithm.

Iterative methods: it start with any feasible solution and proceed to improve upon the solution by repeated applications of a simple step, e.g., Ford-Fulkerson algorithm.

Further Refinements-2Dynamic programming

Bottom-up: a table of solutions to subproblems is filled starting with the problem’s smallest subproblems. A solution to the original instance of the problem is then obtained from the table constructed.

Top-down: memory function

Further Refinements-3State-space-tree techniques

Backtracking: take coloring problem as an example:

1 2

4 3

Use three colors to color the vertices of this graph. How many different ways ?

Further Refinements-3 cont.S

1 2 3

21 3

21 3

V1

V2

V3

Further Refinements-3 cont.Branch-and-Bound: take TSP problem as an

example. If there are four cities, all feasible solutions are

Further Refinements-3 cont.Consider the traveling costs between any two

cities:

=3+3+5+4+1

Further Refinements-3 cont.Start to branchChoose 12

Otherwise

Further Refinements-3 cont.The current branch:

Further Refinements-3 cont.The difference between them lies in that

backtracking is not limited to optimization problems, while branch-and-bound is not restricted to a specific way of traversing the problem’s space tree.

ConclusionThis paper gives a new taxonomy of algorithm design

techniques. No matter how many general design techniques are

recognized, there will always be algorithms that cannot be naturally interpreted as an application of those techniques.

Some algorithms can be interpreted as an application of different techniques, e.g., selection sort, as a brute-force algorithm and as a decrease-and-conquer method.

Some algorithms may incorporate ideas of several techniques, e.g. Fourier transform takes advantage of both the transform and divide-and-conquer ideas.