Machine Learning

A Quick look

Sources:• Artificial Intelligence – Russell & Norvig• Artifical Intelligence - Luger

By: Héctor Muñoz-Avila

What Is Machine Learning?“Logic is not the end of wisdom, it is just the beginning” --- Spock

System

Knowledge

Environment

Action1

time

Knowledge

Environment

System

changed

Action2

Learning: The Big Picture

•Two forms of learning:

Supervised: the input and output of the learning component can be perceived (for example: friendly teacher)

Unsupervised: there is no hint about the correct answers of the learning component (for example to find clusters of data)

4

• Online – during gameplay– Adapt to player tactics– Avoid repetition of mistakes– Requirements: computationally cheap,

effective, robust, fast learning (Spronck 2004)

• Offline - before the game is released– Devise new tactics– Discover exploits

Offline vs. Online Learning

Classification(According to the language representation)

• Symbolic Version Space Decision Trees Explanation-Based Learning …

• Sub-symbolic Reinforcement Learning Connectionist Evolutionary

Classification(According to the language representation)

• Symbolic Version Space Decision Trees Explanation-Based Learning …

• Sub-symbolic Reinforcement Learning Connectionist Evolutionary

Version Space

Idea: Learn a concept from a group of instances, some positive and some negative

Example: •target: obj(Size,Color,Shape) Size = {large, small} Color = {red, white, blue} Shape = {ball, brick, cube}

•Instances: +:

obj(large,white,ball)obj(small,blue,ball)

−: obj(small,red,brick)

obj(large,blue,cube)

Two extremes (temptative) solutions:

obj(X,Y,Z)

obj(large,white,ball) obj(small,blue,ball) …

too general

too specific

obj(large,Y,ball) obj(small,Y,ball)

obj(X,Y,ball)

…

con

cep

t sp

ace

How Version Space Works

++

+

+

++

+

+− −

If we consider only positives If we consider positive and negatives

++

+

+

++

+

+− −

What is the role of the negative instances?

to help prevent over-generalizations

Classification(According to the language representation)

• Symbolic Version Space Decision Trees Explanation-Based Learning …

• Sub-symbolic Reinforcement Learning Connectionist Evolutionary

Explanation-Based learning

AC

B A B C A CB

CBA

BA

C

BAC

B CA

CAB

ACB

BCA

A BC

AB

C

ABC

Can we avoid making this error again?

?

?

?

Explanation-Based learning (2)

AC

B A B CCBA

ACB

A BC

?

?

?

Possible rule: If the initial state is this and the final state is this, don’t do that

More sensible rule: don’t stack anything above a block, if the block has to be free in the final state

Classification(According to the language representation)

• Symbolic Version Space Decision Trees Explanation-Based Learning …

• Sub-symbolic Reinforcement Learning Connectionist Evolutionary

Motivation # 1: Analysis Tool

•Suppose that a company have a data base of sales data, lots of sales data

•How can that company’s CEO use this data to figure out an effective sales strategy

•Safeway, Giant, etc cards: what is that for?

Motivation # 1: Analysis Tool (cont’d)

Ex’ple Bar Fri Hun Pat Type Res wait

x1 no no yes some french yes yes

x4 no yes yes full thai no yes

x5 no yes no full french yes no

x6

x7

x8

x9

x10

x11

Sales data

“if buyer is male & and age between 24-35 & married then he buys sport magazines”

induction

Decision Tree

The Knowledge Base in Expert Systems

A knowledge base consists of a collection of IF-THEN rules:

if buyer is male & age between 24-50 & married then he buys sport magazines

if buyer is male & age between 18-30then he buys PC games magazines

Knowledge bases of fielded expert systems contain hundreds and sometimes even thousands such rules. Frequently rules are contradictory and/or overlap

Main Drawback of Expert Systems: The Knowledge Acquisition Bottle-NeckThe main problem of expert systems is acquiring knowledge from human specialist is a difficult, cumbersome and long activity.

Name KB #Rules Const. time

(man/years)

Maint. time

(man/years)

MYCIN KA 500 10 N/A

XCON KA 2500 18 3

KB = Knowledge BaseKA = Knowledge Acquisition

Motivation # 2: Avoid Knowledge Acquisition Bottle-Neck

•GASOIL is an expert system for designing gas/oil separation systems stationed of-shore

•The design depends on multiple factors including: proportions of gas, oil and water, flow rate, pressure, density, viscosity, temperature and others

•To build that system by hand would had taken 10 person years

•It took only 3 person-months by using inductive learning!

•GASOIL saved BP millions of dollars

Motivation # 2 : Avoid Knowledge Acquisition Bottle-Neck

Name KB #Rules Const. time

(man/years)

Maint. time

(man/months)

MYCIN KA 500 10 N/A

XCON KA 2500 18 3

GASOIL IDT 2800 1 0.1

BMT KA (IDT)

30000+ 9 (0.3) 2 (0.1)

KB = Knowledge BaseKA = Knowledge AcquisitionIDT = Induced Decision Trees

Example of a Decision TreePatrons?

no yes

nonesome

waitEstimate?

no yes

0-10>60

Full

Alternate?

Reservation?

Yes

30-60

no

yes

No

no

Bar?

Yes

no

yes

Fri/Sat?

No Yes

yes

no yes

Hungry?

yes

No

10-30

Alternate?

yes

Yes

no

Raining?

no yes

yes

no yes

Definition of A Decision Tree

A decision tree is a tree where:

•The leaves are labeled with classifications (if the classification is “yes” or “no”. The tree is called a boolean tree)

•The non-leaves nodes are labeled with attributes

•The arcs out of a node labeled with an attribute A are labeled with the possible values of the attribute A

Induction

Ex’ple Bar Fri Hun Pat Type Res wait

x1 no no yes some french yes yes

x4 no yes yes full thai no yes

x5 no yes no full french yes no

x6

x7

x8

x9

x10

x11

Data

patternpattern

Databases: what are the data that matches this pattern?

database

Induction: what is the pattern that matches these data?

induction

Induction of Decision Trees

•Objective: find a concise decision tree that agrees with the examples

•The guiding principle we are going to use is the Ockham’s razor principle: the most likely hypothesis is the simplest one that is consistent with the examples

•Problem: finding the smallest decision tree is NP-complete

•However, with simple heuristics we can find a small decision tree (approximations)

Induction of Decision Trees: Algorithm

Algorithm:

1. Initially all examples are in the same group

2. Select the attribute that makes the most difference (i.e., for each of the values of the attribute most of the examples are either positive or negative)

3. Group the examples according to each value for the selected attribute

4. Repeat 1 within each group (recursive call)

Example

Ex’ple Bar Fri Hun Pat Alt Type wait x1 no no yes some yes French yes

x4 no yes yes full yes Thai yes

x5 no yes no full yes French no

x6 yes no yes some no Italian yes

x7 yes no no none no Burger no

x8 no no yes some no Thai yes

x9 yes yes no full no Burger no

x10 yes yes yes full yes Italian no

x11 no No no none no Thai no

IDT: Example

Lets compare two candidate attributes: Patrons and Type. Which is a better attribute?

Patrons?

none

X7(-),x11(-)

some

X1(+),x3(+),x6(+),x8(+)

full

X4(+),x12(+), x2(-),x5(-),x9(-),x10(-)

Type?

french

X1(+),x5(-)

italian

X6(+),x10(-)

burger

X3(+),x12(+), x7(-),x9(-)X4(+),x12(+)

x2(-),x11(-)

thai

Example of a Decision TreePatrons?

no yes

nonesome

waitEstimate?

no yes

0-10>60

Full

Alternate?

Reservation?

Yes

30-60

no

yes

No

no

Bar?

Yes

no

yes

Fri/Sat?

No Yes

yes

no yes

Hungry?

yes

No

10-30

Alternate?

yes

Yes

no

Raining?

no yes

yes

no yes

Decision Trees in Gaming

• Black & White, developed by Lionhead Studios, and released in 2001 used ID3

• Used to predict a player’s reaction to a certain creature’s action

• In this model, a greater feedback value means the creature should attack

Decision Trees in Black & White

Example Attributes     Target

  Allegiance Defense Tribe Feedback

D1 Friendly Weak Celtic -1.0

D2 Enemy Weak Celtic 0.4

D3 Friendly Strong Norse -1.0

D4 Enemy Strong Norse -0.2

D5 Friendly Weak Greek -1.0

D6 Enemy Medium Greek 0.2

D7 Enemy Strong Greek -0.4

D8 Enemy Medium Aztec 0.0

D9 Friendly Weak Aztec -1.0

Decision Trees in Black & White

Allegiance

Defense

Friendly Enemy

0.4 -0.3

-1.0

Weak Strong

0.1

Medium

Note that this decision tree does not even use the tribe attribute

Decision Trees in Black & White

• Now suppose we don’t want the entire decision tree, but we just want the 2 highest feedback values

• We can create a Boolean expressions, such as

((Allegiance = Enemy) ^ (Defense = Weak)) v ((Allegiance = Enemy) ^ (Defense = Medium))

Classification(According to the language representation)

• Symbolic Version Space Decision Trees Explanation-Based Learning …

• Sub-symbolic Reinforcement Learning Connectionist Evolutionary