Theory of Algorithms:Brute Force

Outline

ExamplesBrute-Force String MatchingClosest-PairConvex-HullExhaustive Search

brute-force strengths and weaknesses

Brute Force

A straightforward approach usually directly based on problem statement and definitionsMotto: Just do it!Crude but often effectiveExamples already encountered:

Computing an (a > 0, n a nonnegative integer) by multiplying a together n timesComputation of n! using recursionMultiplying two n by n matricesSelection sort

Brute Force String Matching

Find a substring in some text Pattern: m characters to search forText: n characters to search in

1. Align pattern at beginning of text2. compare each character 3. while pattern is not found and the text is not exhausted,

move pattern one position to the right and repeat

Brute Force String Matching

Problem: Find a substring in some text that matches a patternPattern: a string of m characters to search forText: a (long) string of n characters to search in

1. Align pattern at beginning of text2. Moving left to right, compare each character of pattern

to the corresponding character in text UNTIL• All characters are found to match (successful

search); or• A mismatch is detected

3. WHILE pattern is not found and the text is not yet exhausted, realign pattern one position to the right and repeat step 2.

String Matching

Example:Pattern: AKA Text: ABRAKADABRATrace: AKA AKA

AKA AKA

Number of Comparisons: In the worst case, m comparisons before shifting, for each of n-m+1 tries

Efficiency: (nm)

Brute Force Closest Pair

Problem: Find the two points that are closest together in a set of n 2-D points P1 = (x1, y1), …, Pn = (xn, yn)Using Cartesian coordinates and Euclidean distance

Algorithm:

Efficiency: (n2)

dmin ∞for i 1 to n-1 do

for j i+1 to n dod sqrt((xi - xj)2 + (yi - yj)2)if d < dmin

dmin d; index1 i; index2 jreturn index1, index2

Brute Force Closest Pairs

Find the two points that are closest together in a set P1 = (x1, y1), …, Pn = (xn, yn)

using Euclidean distance

P1

P2

P3

P4

P5

P6

P7

P8

P9

The Convex Hull Problem

Find the convex hull enclosing n 2-D pointsConvex Set: A set of points in the plane is convex if for any two points P and Q, the line segment joining P and Q belongs to the set Convex Hull: If S is a set of points then the Convex Hull of S is the smallest convex set containing S

ConvexNon-Convex

Brute Force Convex Hull

Problem: given a set of points like those above, find its convex hull (the smallest convex set enclosing them all)

P1

P2

P3

P4

P5

P6

P7

P8

P9

Brute Force Convex Hull

The answer for this example is shown above.

P1

P2

P3

P4

P5

P6

P7

P8

P9

Brute Force Convex Hull

Algorithm: For each pair of points p1 and p2 determine if all other points lie to the same side of the line (drawn across the plane) through p1 and p2

Efficiency: for n(n-1)/2 point pairs, check (n-2) others O(n3)

P1

P2

P3

P4

P5

P6

P7

P8

P9

Pros and Cons of Brute Force

Strengths:Wide applicabilitySimplicityYields reasonable algorithms for some important problems and standard algorithms for simple computational tasksA good yardstick for better algorithmsSometimes doing better is not worth the bother

Weaknesses:Rarely produces efficient algorithmsSome brute force algorithms are infeasibly slowNot as creative as some other design techniques

Exhaustive Search

Definition:A brute force solution to the search for an element with a special propertyUsually among combinatorial objects e.g. permutations or subsets

Method:1. Find a systematic way of listing all potential

solutions • All solutions eventually listed• No solution repeated

2. Evaluate solutions one by one 3. When search ends, announce the winner

Travelling Salesman Problem

Problem:Given n cities with known distances between each pairFind the shortest tour that passes through all the cities exactly once before returning to the starting city

Alternatively: Find shortest Hamiltonian Circuit in a weighted connected graph

Example: a b

c d

8

2

7

53

4

Travelling Salesman Problem Tour Cost . abcda 2+3+7+5 = 17abdca 2+4+7+8 = 21acbda 8+3+4+5 = 20acdba 8+7+4+2 = 21adbca 5+4+3+8 = 20adcba 5+7+3+2 = 17

a b

c d

8

2

7

53

4

Travelling Salesman by Exhaustive Search

Tour Cost . abcda 2+3+7+5 = 17abdca 2+4+7+8 = 21acbda 8+3+4+5 = 20acdba 8+7+4+2 = 21adbca 5+4+3+8 = 20adcba 5+7+3+2 = 17Improvements:

Start and end at one particular cityRemove tours that differ only in direction

Efficiency: (n-1)!/2 = O(n!)

Knapsack Problem

Find the most valuable subset of items that fit in the knapsack, given n items

weights: w1 , w2 … wn

values: v1 , v2 … vn

a knapsack of capacity W

Example (W = 16): Item Weight

Value

1 2kg R200

2 5kg R300

3 10kg R500

4 5kg R100

Knapsack by Exhaustive Search

Efficiency: Ω(2n)

Subset Total Wgt

Total Value

1 2 kg R200

2 5 kg R300

3 10 kg R500

4 5 kg R100

1,2 7 kg R500

1,3 12 kg R700

1,4 7 kg R300

2,3 15 kg R800

Subset Total Wgt

Total Value

2,4 10 kg R400

3,4 15 kg R600

1,2,3

17 kg n/a

1,2,4

12 kg R600

1,3,4

17 kg n/a

2,3,4

20 kg n/a

1,2,3,4

22 kg n/a

Generating Combinatorials: Subsets

Combinatorics uses Decrease (by one) and Conquer AlgorithmsSubsets: generate all 2n subsets of A = a1, …, an

Divide into subsets of a1, …, an-1 that contain an and those that don’tSneaky Solution: establish a correspondence between bit strings and subsets. Bit n denotes presence (1) or absence (0) of element an

Generate numbers from 0 to 2n-1 convert to bit strings interpret as subsetsExamples: 000 = Ø , 010 = a2 , 110 = a1, a2

Generating Combinatorials: Permutations

Permutations: generate all n! reorderings of 1, …, n

Generate all (n-1)! permutations of 1, …, n-1Insert n into each possible position (starting from the right or left, alternately)Implemented by the Johnson-Trotter algorithm

Satisfies Minimal-Change requirementNext permutation obtained by swapping two elements of previousUseful for updating style algorithms

Example:Start: 1Insert 2: 12 21Insert 3: 123 132 312 321 231 213

Final Comments on Exhaustive Search

run in a realistic amount of time only on very small instances Often there are much better alternatives!

Euler circuitsShortest pathsMinimum spanning treeAssignment problem

In some cases exhaustive search is the only known solution