cs 621 artificial intelligence lecture 29 – 22/10/05 prof. pushpak bhattacharyya

22.10.2005 1Prof. Pushpak Bhattacharyya, IIT Bombay

CS 621 Artificial Intelligence

Lecture 29 – 22/10/05

Prof. Pushpak Bhattacharyya

SOM, Theoretical Aspects Of Machine Learning, Probably

Approximately Correct Learning.


…

… … .

P neurons o/p layer

n neurons

Clusters:

AA :

A

Wp

B :

C :

: :


Wp(n+1) = Wp

(n) + η (I(n) – Wp(n))

Competitive learning: Weights to the winner neuron only are changed.


Generating function application to recurrence relation solving.

Example: Max. No of regions on 2-Dim plane produced by n lines.


L2L1L3

L4

0 x

y R1 = 2

R2 = R1 + 2

= 2+ 2

= 4

R3 = R2 + 3

= 4 + 3 = 7

R4 = 7 + 4 = 11 Rn = Rn-1 + n R1 = 2


Rn = Rn-1 + n

Rn-1 = Rn-2 + (n-1)

Rn-2 = Rn-3 + (n-2)

.

.

.

R2 = R1 + 2


Rn = n + (n-1) + (n-2)+ …..+R1 + 2

= n+(n-1)+(n-2)+…..+2+2

=n(n+1)/2 + 1


Generating Function Method

Key idea: Rns are generated by infinite

series function ,i.e,

F(x) = R1 + R2x + R3x2 + ………………+ Rnxn-1+ ….. ∞

Rn = Coeff of xn-1


Rn = Rn-1 + n

Rn - Rn-1 = n

R2 - R1 = 2

R3 – R2 = 3

.

.

(Continued)


F(x) = R1 + R2x + R3x2 + ………………+ Rnxn-1+

….. ∞ (1)

xF(x) = R1x + R2x2 + R3x3 + ……………+ Rnxn+

….. ∞ (2)

1 - 2 gives

(1-x) F(x) = R1 + (R2 – R1)x + (R3 – R2)x2 +

……………+ (Rn – Rn-1)xn-1+ ….. ∞

(Continued)


(1-x) F(x) = 2 + 2x + 3x2 + ……………+ nxn-1+ ….. ∞

F(x) = (1-x)-1(2 + 2x + 3x2 + ……………+ nxn-1+ ….. ∞)

= ( 1 + x + x2 + x3 +…+ xn-1+…..)

(2 + 2x + 3x2 + 4x3 +…+ nxn-1+…..)

(Continued)


Co eff of Xn-1 from the expression for f(x)

= ( n + n-1 + n-2 + … + 2+ 2 )

n(n+1)/ 2 + 1

(Continued)


Clustering Algos

1. Competitive learning

2. K – means clustering

3. Self organization / Kohonen net

4. Counter Propagation


Two packages

1. SNNS – Stuttgart Neural Net Simulator

2. WEKA Package

3. MATLAB


Data to train

1) IRIS data: 3 - class data

2) X- OR, parity, Majority


IRIS Data

Classification of flowers based on four attributes in 3 classes c1, c2 and C2

1 Petal length A1

2 Petal width A2

3 Sepal length A3

4 Sepal width A4


Google on “Machine learning Data”

IRIS

120 – data points

< A1, A2, A3, A4 >1 --- C1

< A1, A2, A3, A4 >2 --- C3

……….

……….


C1, C2, C3 each has 50 data points in it

80 – 20 rule for ML

Training – 40 and Testing -- 10


What you have to do for IRIS choose 40 points randomly from C1, C2 and C3

1. Run BP on FF n/w

2. Run SOM assuming unlabelled data. See if you can discover 3 cluster


Two part assignment

Supervised

Hidden layer to be found

4 neurons for 4 attributes


BP for IRIS error

Training iterations

EPOCHS

1%

Very good performance on test data should be achieved.


A1 A2 A3

4 I/p neuronsA4

Cluster Discovery By SOM/Kohenen Net


K – means clustering

K o/p neurons are required from the knowledge of k clusters being present.

26 neurons

Full connection

n neurons

……

……


Steps

1. Initialize the weights randomly.

2. Ik is the vector presented at kth iteration.

3. Find W* such that

|w* - Ik| < |wj - Ik| for all j


4. make W*(new) = W* (old) + η(Ik - w* ).

5. k K +1 ; go to 3.

6. Go to 2 until the error is below a threshold.


K means

1. Initialize weights randomly.

2. Ik in the I/p vector presented at kth.

step of one iteration over all the training data (EPOCH).

3. Find the W* such that |w* - Ik| is minimum.

4. Find W* for Ik+1..


5. See the partitions at the end of the epoch.

6. Find centroid of each partition.

7. Make the weight equal to the centroid of the partition.

8. Take next epoch.


9. Keep doing until the error is below a threshold.

Key idea form competitive learning: Weight change incorporate after every epoch.

cs 621 artificial intelligence lecture 29 – 22/10/05 prof. pushpak bhattacharyya

Documents