bab 5 classification: alternative techniques part 4 artificial neural networks based classifer
DESCRIPTION
Bab /21 Artificial Neural Networks (ANN) / 2TRANSCRIPT
![Page 1: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/1.jpg)
Bab 5Bab 5Classification: Classification:
Alternative TechniquesAlternative Techniques
Part 4Part 4Artificial Neural Networks Artificial Neural Networks
Based ClassiferBased Classifer
![Page 2: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/2.jpg)
Bab 5-4 - 2/21
Artificial Neural Networks (ANN) / 1
X1 X2 X3 Y1 0 0 01 0 1 11 1 0 11 1 1 10 0 1 00 1 0 00 1 1 10 0 0 0
X1
X2
X3
Y
Black box
Output
Input
Output Y is 1 if at least two of the three inputs are equal to 1.
![Page 3: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/3.jpg)
Bab 5-4 - 3/21
Artificial Neural Networks (ANN) / 2
X1 X2 X3 Y1 0 0 01 0 1 11 1 0 11 1 1 10 0 1 00 1 0 00 1 1 10 0 0 0
X1
X2
X3
Y
Black box
0.3
0.3
0.3 t=0.4
Outputnode
Inputnodes
otherwise0 trueis if1
)( where
)04.03.03.03.0( 321
zzI
XXXIY
![Page 4: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/4.jpg)
Bab 5-4 - 4/21
Artificial Neural Networks (ANN) / 3
Model is an assembly of inter-connected nodes and weighted links
Output node sums up each of its input value according to the weights of its links
Compare output node against some threshold t
X1
X2
X3
Y
Black box
w1
t
Outputnode
Inputnodes
w2
w3
)( tXwIYi
ii Perceptron Model
)( tXwsignYi
ii
or
![Page 5: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/5.jpg)
Bab 5-4 - 5/21
General Structure of ANN
Activationfunction
g(Si )Si Oi
I1
I2
I3
wi1
wi2
wi3
Oi
Neuron iInput Output
threshold, t
InputLayer
HiddenLayer
OutputLayer
x1 x2 x3 x4 x5
y
Training ANN means learning the weights of the neurons
![Page 6: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/6.jpg)
Bab 5-4 - 6/21
Algorithm for Learning ANN
Initialize the weights (w0, w1, …, wk)
Adjust the weights in such a way that the output of ANN is consistent with class labels of training examples– Objective function:
– Find the weights wi’s that minimize the above objective function e.g., backpropagation algorithm
2),( i
iii XwfYE
![Page 7: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/7.jpg)
Bab 5-4 - 7/21
Artificial Neural Networks (ANN) / 2
![Page 8: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/8.jpg)
Bab 5-4 - 8/21
Perceptron
![Page 9: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/9.jpg)
Bab 5-4 - 9/21
Let D = {(xi, yi) | i= 1,2,…,N} be the set of training examples Initialize the weights Repeat
– For each training example (xi, yi) do Compute f(w, xi) For each weight wj do
Update the weight
Until stopping condition is met
Perceptron Learning Rule / 1
ok
ooo www ....,,, 1
ijik
ikj
kj xxwfyww ,1
![Page 10: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/10.jpg)
Bab 5-4 - 10/21
Weight update formula:
Intuition:– Update weight based on error– If y = f(w,x), e = 0, no update is needed– If y > f(w,x), e = 2, weight must be increased so that
f(w,x) will increase– If y < f(w,x), e = -2, weight must be decreased so that
f(w,x) will decrease
Perceptron Learning Rule / 2
ratelearning
ijik
ikj
kj xxwfyww
;,1
ik
i xwfye ,
![Page 11: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/11.jpg)
Bab 5-4 - 11/21
Terminating condition: Training stops when either1. all wij in the previous epoch (i.e., iteration) were so
small as to be below some specified threshold, or2. the percentage of samples misclassified in the previous
epoch is below some threshold, or3. a pre-specified number of epochs has expired.
In practice, several hundreds of thousands of epochs may be required before the weights will converge
Perceptron Learning Rule / 3
![Page 12: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/12.jpg)
Bab 5-4 - 12/21
Example of Perceptron Learning
d
ii
ki
k
ijik
ikj
kj
xwsignxwf
xxwfyww
0
1
)(,
0.2;,
![Page 13: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/13.jpg)
Bab 5-4 - 13/21
Perceptron Learning
![Page 14: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/14.jpg)
Bab 5-4 - 14/21
Nonlinearly Separable Data
![Page 15: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/15.jpg)
Bab 5-4 - 15/21
Multilayer Neural Network / 1
![Page 16: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/16.jpg)
Bab 5-4 - 16/21
Multilayer Neural Network / 2
![Page 17: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/17.jpg)
Bab 5-4 - 17/21
Learning Multilayer Neural Network
![Page 18: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/18.jpg)
Bab 5-4 - 18/21
Gradient Descent for Multilayer NN / 1
![Page 19: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/19.jpg)
Bab 5-4 - 19/21
Gradient Descent for Multilayer NN / 2
![Page 20: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/20.jpg)
Bab 5-4 - 20/21
Design Issues in ANN
![Page 21: Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer](https://reader035.vdocuments.site/reader035/viewer/2022062909/5a4d1bc87f8b9ab0599d5ac3/html5/thumbnails/21.jpg)
Bab 5-4 - 21/21
Characteristics of ANN