Introduction to ECE 2401 Data Structure

Fall 2005

Supplementary: Introduction to Problem Solving

Chen, Chang-Sheng

2005.09.13

Outline: Problem solving and search

• Introduction to Problem Solving– Example problems– Problem formulation– Problem solving as Search

• Search Space

• Common Problem-Solving Strategies

Introduction to Problem solving

• Introduction– What is a problem ?– What is a solution ?– Problem Solving and Representation

• Alternative View: Search vs. Planning

Example: The River Problem

• A farmer wishes to carry a wolf (i.e., the family pet), a goat and a cabbage across a river, from the south to the north shore. The farmer is the proud owner of a small rowing boat called Bounty which he feels is easily up to the job. Unfortunately the boat is only large enough to carry at most the farmer and one other item. Worse again, if left unattended the wolf will eat the goat and the goat will eat the cabbage. How can the farmer safely transport the wolf, the goat and the cabbage to the opposite shore?

F WC G

Example: The 8-puzzle Problem• This 8-puzzle is the classic `toy' search problem. It is

based on the familiar, tile-sliding puzzle of the same name.• We have a small plastic tray divided into a 3x3 grid. This is

covered with eight tiles numbered 1 to 8. One of the positions on the grid is empty and into this space we can slide a tile from left, right, above or below (depending on where the space is).

• To solve the puzzle we have to slide tiles around so as to get them into numeric order reading left-to-right and top-to-bottom, i.e, we have to produce a tile pattern which looks like this.

1 2 3 4 5 67 8

Example: Planning Problem

• Initial state: (on A Table) (on C A) (on B Table) (clear B) (clear C)

• Goals: (on C Table) (on B C) (on A B) (clear A)

A

C

B C

B

AInitial state Goals

Table

Example: Romania

What is a Problem ?

• A problem is the difference between your current state and your goal state. – A problem can result from new knowledge or

thinking. When you know where you are and where you want to be, you have a problem to solve in getting to your destination.

– When you can identify the difference between what you have and what you want, you have defined your problem and can aim toward your goal.

Problem Solving

A problem is characterized as:

• An initial state

• A set of actions (functions that map states to other states)

• A goal test

• A cost function (optional)

Defining The State Space

• A problem is solved by moving from the initial state to the goal state by applying valid operators (actions) in sequence.

• Thus the state space is the set of states reachable from a particular initial state.

initial

s1 s2

s3

goal

A problem solution is simply the set of operators (actions) needed to reach the goal state from the initial state.

Problem Solving and Representation

• To solve a particular problem, we need to do 3 things:Define the problem precisely. Isolate and represent the task knowledge

that is necessary to solve the problem.Choose and apply the best problem-solving

technique(s) to the particular problem.

Problem Representation (1)• The previous example problems, described in

English, are confusing enough to be difficult to solve. – It emphasizes irrelevant details (the wolf being a

family pet, the name of the boat etc.) and disguises very relevant information (the positions of the objects, etc.).

Problem Representation (2)

• The key to effective problem solving is good representation.

• A good representation scheme has the following properties:– Make important objects and relations explicit.– Suppress unimportant and irrelevant detail.– Expose natural constraints.– Concise – efficiently describe a given scenario.– Complete – everything necessary can be

described.

Problem formulation - States and Operators

• To come up with a much better representation.– One way is to think in terms of states. A state is a

description of a system at some given point in time.

• Many problems can be formally defined by the following components:

– An initial state

– A goal state

– An operator set (or actions) - each operator can transform one state into another.

– A cost function (optional)

Problem solving as Search• Solving a problem involves finding a sequence of steps

which achieves some particular goal state, starting from some initial state. – One way of achieving this is simply to explore the

possible steps that can be taken in the initial situation, and then move on to do the same thing with all the situations that are achieved by one of the possible steps.

– This process is, of course, just a search through the `tree' of possible situations that are achievable starting from the initial situation.

– Thus problem solving can be done by tree-search.

Problem solving as Search (cont.)

• To solve the problem we simply search through this tree, with the aim of finding a sequence of branches (steps) which connects the initial state with the goal state.

• Possible situations are states and the root of the tree is therefore the initial state.– The first branches are the steps that can be taken

from the initial state.– The next branches are the steps that can be taken

from the states reachable from the initial state, and so on.

Search Space: state-space & plan-state search

• State-space search– In the search graph:

• Node: a state of the world• Arc: a single

action

– Example:

State Space for Blocks World

AC

B A B C A CB

CBA

BA

C

BAC

B CA

CAB

ACB

BCA

A BC

AB

C

ABC

Search Space: state-space & plan-state search

• Plan-space search– In the search graph:

• Node: partially-specified plans, a set of actions• Edge: operations such as the addition of an

action to a plan

– Example:

Plan Space for Blocks World

Move A to BMove B to CMove A to B

Move C to TableMove B to CMove A to B

Move A to TableMove A to B

Using Search to solve the river problem

• Consider the river problem once again. In this problem information can be represented in the form of the following 4-tuple.

• Each variable position stores the location of a particular object. – For instance, the initial state where the farmer, the goat,

the wolf and the cabbage are on the south bank is given by (S,S,S,S).

– The goal state is represented by (N,N,N,N).

A? B? C? D?( ),,,

farmer wolf goat cabbage

Example – Defining Operators

• There are 4 transforming operators:– Farmer-Takes-Self

(A?,B?,C?,D?) (Opposite(A?), B?,C?,D?)– Farmer-Takes-Wolf

(A?,B?,C?,D?) (Opposite(A?), Opposite(B?),C?,D?)– Farmer-Takes-Goat

(A?,B?,C?,D?) (Opposite(A?), B?, Opposite(C?),D?)– Farmer-Takes-Cabbage

(A?,B?,C?,D?) (Opposite(A?), B?,C?, Opposite(D?))

• Finally, a state is safe as long as the wolf and the goat or the cabbage are not left unattended – this is the precondition to the above operators.

Example – One Solution…While there are other possibilities here is one 7 step solution to the river

problem.

F W G C

F W G C

F-Takes-G

S1: (S,S,S,S)

F

W

G

C

F-Takes-G

S2: (N,S,N,S)

S8: (N,N,N,N)

F-Takes-SF

W

G

C

S7: (S,N,S,N)

F-Takes-C

F W

G

C

S6: (N,N,S,N)

F

W

G C

F-Takes-G

S5: (S,N,S,S)

F W G

C

F-Takes-W

S4: (N,N,N,S)

F-Takes-S

F W

G

CS3: (S,S,N,S)

Example: The 8-puzzle

• states? locations of tiles • actions? move blank left, right, up, down • goal test? = goal state (given)• path cost? 1 per move

[Note: optimal solution of n-Puzzle family is NP-hard]

••

The 8-puzzle as a search problem

• The 8-puzzle is a type of problem which is naturally solved using search.– The aim of the search is to find a solution path (a

sequence of tile movements) which transforms the initial state into the goal state.

• To solve the 8-puzzle, we need to find a sequence of steps (i.e., tile movements) which achieves a particular goal state (all tiles in numeric order) starting from some given starting state (the initial tile configuration).

Implementing the search

• A particular search strategy works the same way for all problems. – It is just the search nodes which differ between

different problems.

• To implement a new search, we need to find some way of generating and testing relevant nodes. In other words, we have to implement the search tree for the problem.– For example, with the 8-puzzle and other types of

problem-solving search, we have to provide an implementation or representation of the tree.

The successor function• We don't want to explicitly represent all the nodes in

the search space, since this would involve explicitly generating all the nodes. – What we do need is some machinery which will

generate the nodes we need on demand.• The nodes required at any given point are simply the

immediate successors (i.e., children) of the node currently being considered.– Thus, to implement an 8-puzzle search procedure,

we need to produce a function which will generate the successors of any given state in the 8-puzzle. In other words, we need to implement a valid successor function for the 8-puzzle.

Representing states

• So, before we can decide how to implement the successor function we need to decide how we are going to represent 8-puzzle states.

• Intuitively, a successor function for the 8-puzzle must take a representation of one state and give us back representations for all the states which can be achieved by moving one tile. – Using such a function we can generate any part of the

search tree we like. The function therefore provides an implicit or virtual representation of the entire space.

Representing states (cont.)• The 8-puzzle grid is a 2-dimensional structure so it

might make sense to use a 2-dimensional data structure for purposes of representing it. – For example we might use a 3x3 array of integers, using

the convention that the number 0 represents the `space'.

• Alternatively, we might use a 1-dimensional array of integers, using the convention that the positions in the array represent tile-positions working left-to-right and top-to-bottom on the grid.

• The most effective strategy will be one which takes advantage of the operations we have available in the programming language we are using.

Successors in the 1-dimensional representation

• Imagine that we use a 1-dimensional (i.e., array) representation. How do we go about implementing the successor function? Let's say the initial state in the problem is

3 5 6 2 1 4 7 8

• Representing this as a 1-dimensional array of numbers would give us

3 5 6 2 1 0 4 7 8

Successors in the 1-dimensional representation (cont.)

• Given a representation of this state, the successor function must give us back three new states: – the one resulting from moving the 6 downwards:

3 5 0 2 1 6 4 7 8 – The one resulting from moving the 1 right:

3 5 6 2 0 1 4 7 8 – And the one resulting from moving the 8 up:

3 5 6 2 1 8 4 7 0

Pseudocode sketch• One way to implement the function would be to write code

which checked for the presence of a space in position 6, and then switched around the stored integers in the appropriate way. The bit of code handling the upwards move of the 8 might look something like this.if (state[6] == 0) {

newstate = new State(state); // generate a copy of state newstate[6] = state[9]; newstate[9] = 0; }

• We could then add in further conditionals to handle the other three moves that are possible from this state.

• If we make sure to handle all the cases we should have code which will generate the successors of any valid 8-puzzle state.

Common Problem-Solving Strategies -- Schunk(1996)

• Generate-and-Test Strategy – E.g., via search space

• Means-Ends Analysis – E.g., via planning space

• Analogical Reasoning

• Brainstorming

Generate and Test - simplest approach to search

• The simplest form of brute-force search (also called exhaustive search).– It is indeed the manner which people often solve

problems because it assumes no additional knowledge about how to reach the solution (other than how to traverse the search tree).

• Three properties to satisfy– It must be complete.– It must be non-redundant.– It must be well-informed (i.e., it should only propose

suitable solutions matching the search space).

Generate and Test Strategy

• Examples:– Tower of Hanoi – The River Problem

• Problem formulation

– The 8-puzzle (most detailed)• Problem formulation• Implementation and Discussion

Tower of Hanoi

Operators: Moving disks from peg to pegConstraints: Can only move one disk at a time

A larger disk can’t be stacked on a smaller diskA disk can’t be moved if there is another disk

on top of it

Start State Goal State

3

Idea of a Problem Space..

At each state: Number of possible “next moves”Can be thought of as a “space of all possible moves”

4

Problem Space

• Legal moves between two states

• Problem solving = search in problem space

5

Towers of Hanoi• Less than optimal solution: 19 moves

7

Means-Ends Analysis

• Simon (1972)• Solving problems by repeatedly determining

the difference between the current state and the goal or subgoal state, then finding an operator that reduces this difference.

• Used the Tower of Hanoi Problem to study means-end analysis.

Heuristics: Means-Ends Analysis

• Heuristics:– Identify a difference between current

state and goal state – Set a subgoal to reduce the difference.– Apply an operator to reduce the

difference– (If operator can’t be applied, new

subgoal = remove obstacle that prevents applying the operator)

13

Heuristics: Means-Ends AnalysisI want to solve the Tower of Hanoi

S1: Set a subgoal to move the largest disk to the right

S1: I can’t move it because the middle disk is on top

S2: Set a subgoal to move the middle disk

S2: I can;t move it because the small disk is on top

S3: Set a subgoal to move the small disk

S3 Move the small disk

S2: Move the middle disk

S1:Move the large disk.

S1: I can’t move it because there is no free disk

14

Analogical Reasoning• There are four forms of analogical reasoning:

– similarity in terms of category, – similarity in terms of relationship, – similarity of the content and assuming the

premise is true, – and negating the opponent’s  argument

(indirect negation). • The last form is to identify examples similar to

the opponent’s examples but obviously wrong. – These concepts are very similar to Aristotle’s

reasoning involving analogical logic .

類比推理的性質 • 類比推理屬於一種歸納推理，在日常生活

中相當普遍，– 例如當我們遇到新問題無法解決時，總會先找

尋記憶中是否有其他類似的例子可以應用 。– 而這種能力也是促成學習的重要方式，可以用

來獲得新知、形成科學假設、說明科學概念、或解決問題。

類比學習• 類比 (analogy)的定義不一而足而且眾說紛紜，一般傾向於如下定義：– 類比是兩個不同領域間的知識系統，藉由彼此間某種屬性或結構的相似性，由已知的知識系統推導到欲知的知識系統，從而獲得或理解知識的過程。

– 類比推理 (analogy reasoning)常用於新的問題之解決，它是人類智力的重要特質，教師常利用它來幫助學生建構抽象或微觀的概念，經由類比遷移來學習新知識。

類比實驗• 科學知識的學習通常視為比其他學科較為困難，其原因極可能是其具有抽象、微觀的特性，使得學習趨於複雜化，類比學習可幫助學習遷移。

• 有些科學研究，甚至理論的建立也透過類比，– 如英國物理學家 Campbell提出撞球模式來類比氣體動力學，

– 利用保利龍球和牙籤加以組合來類比分子中各原子間的鍵結情形。

Brainstorming

• Four steps of Brainstorming:– Define the problem – Generate as many solutions as possible

without evaluating them – Decide on criteria for judging potential

solutions – Use these criteria to select the best solution