nural network er.abhishek k. upadhyay
TRANSCRIPT
04/15/2023
HAMMING NET AND MAXNET
PRESENTED BY:
ER.Abhishek k. upadhyay
ECE(REGULAR),2015
1
04/15/2023
A neural network is a processing device, whose design was inspired by the design and functioning of human brain and their components.
Different neural network algorithms are used for recognizing the pattern.
Various algorithms differ in their learning mechanism.
All learning methods used for adaptive neural networks can be classified into two major categories:Supervised learning Unsupervised learning
Introduction
2
04/15/2023
Its capability for solving complex pattern recognition problems:-Noise in weightsNoise in inputsLoss of connectionsMissing information and adding information.
Problems
3
04/15/2023
Hamming net and MaxnetThe primary function of which is to retrieve in a
pattern stored in memory, when an incomplete or noisy version of that pattern is presented.
This is a two layer classifier of binary bipolar vectors.
The first layer of hamming network itself is capable of selecting the stored class that is at minimum HD value to the test vector presented at the input.
The second layer MAXNET only suppresses outputs.4
04/15/2023
Contd
5
04/15/2023
Contd The hamming network is of the feed forward
type. The number of output neurons in this part equals the number of classes.
The strongest response of a neuron of this layer indicated the minimum HD value between the input vector and the class this neuron represents.
The second layer is MAXNET, which operates as a recurrent network. It involves both excitatory and inhibitory connections.
6
04/15/2023 7
04/15/2023
The purpose of the layer is to compute, in a feed forward manner, the values of (n-HD).
Where HD is the hamming distance between the search argument and the encoded class prototype vector.
For the Hamming net, we have input vector Xp classes => p neurons for outputoutput vector Y = [y1,……yp]
Contd
8
04/15/2023
for any output neuron ,m, m=1, ……p, we have
Wm = [wm1, wm2,……wmn]t and m=1,2,……p
to be the weights between input X and each output neuron.
Also, assuming that for each class m, one has the prototype vector S(m) as the standard to be matched.
Contd
9
04/15/2023
For classifying p classes, one can say the m’th output is 1 if and only if
output for the classifier areXtS(1), XtS(2),…XtS(m),…XtS(p)
So when X= S(m), the m’th output is n and other outputs are smaller than n.
Contd
X= S(m) W(m)
=S(m) => happens only
10
04/15/2023
Xt S(m) = (n - HD(X , S(m)) ) - HD(X , S(m)) ∴½ XtS(m) = n/2 – HD(X , S(m))
So the weight matrix:
WH=½S
Contd
)()(2
)(1
)2()2(2
)2(1
)1()1(2
)1(1
2
1
pn
pp
n
n
H
SSS
SSS
SSS
W
11
04/15/2023
By giving a fixed bias n/2 to the input then
netm = ½XtS(m) + n/2 for m=1,2,……p
or netm = n - HD(X , S(m))
To scale the input 0~n to 0~1 down, one can apply transfer function as
f(netm) = 1/n(netm) for m=1,2,…..p
Contd
12
04/15/2023
Contd
13
04/15/2023
So for the node with the the highest output means that the node has smallest HD between input and prototype vectors S(1)……S(m) i.e.
f(netm) = 1
for other nodes f(netm) < 1
The purpose of MAXNET is to let max{ y1,……yp } equal to 1 and let others equal to 0.
Contd
14
04/15/2023 15
04/15/2023
So ε is bounded by 0<ε<1/p and
ε: lateral interaction coefficient
Contd
)(1
1
1
1
pp
MW
16
04/15/2023
And
So the transfer function
Contd
0
0 0)(
netnet
netnetf
17
kk
kM
k
netfY
YWnet
1
04/15/2023
Each entry of the updated vector decreases at the k’th recursion step under the MAXNET update algorithm, with the largest entry decreasing slowest.
18
Contd
04/15/2023
Step 1: Consider that patterns to classified are a1, a2 … ap,each pattern is n dimensional. The weights connecting inputs to the neuron of hamming network is given by weight matrix.
Algorithm
pmpp
n
n
H
aaa
aaa
aaa
W
21
22121
11211
2
1
19
04/15/2023
Step2: n-dimensional input vector x is presented to the input.
Step3: Net input of each neuron of hamming network is
netm = ½XtS(m) + n/2 for m=1,2,……p
Where n/2 is fixed bias applied to the input of each neuron of this layer.
Step 4: Out put of each neuron of first layer is, f(netm) =1/n( netm) for
m=1,2,…..p
Contd
20
04/15/2023
Step 5: Output of hamming network is applied as input to MAXNET
y0=f(netm)
Step 6: Weights connecting neurons of hamming network and MAXNET is taken as,
Contd
)(1
1
1
1
pp
MW
21
04/15/2023
Where ε must be bounded 0< ε <1/p. the quantity ε can be called the literal interaction coefficient. Dimension of WM is p×p.
Step 7: The output of MAXNET is calculated as,
k=1, 2, 3…… denotes the no of iterations.
Contd
0
0 0)(
netnet
netnetf
k1k
kM
k
netfY
YWnet
22
04/15/2023
Ex: To have a Hamming Net for classifying C , I , T
thenS(1) = [ 1 1 1 1 -1 -1 1 1 1 ]t
S(2) = [ -1 1 -1 -1 1 -1 -1 1 -1 ]t
S(3) = [ 1 1 1 -1 1 -1 -1 1 -1 ]t
So,
23
Example
111111111
111111111
111111111
HW
04/15/2023 24
04/15/2023
For
And
25
22
1 nXWnet H
Y
netft
9
5
9
3
9
7
t
t
net
X
537
111111111
04/15/2023
Input to MAXNET and select =0.2 < 1/3(=1/p)
So,
And
26
15
1
5
15
11
5
15
1
5
11
MW
kk
kM
k
netfY
YWnet
1
04/15/2023
K=o
27
333.0
067.0
599.0
333.0
067.0
599.0
555.0
333.0
777.0
12.02.0
2.012.0
2.02.01
01
0
netfY
o
net
04/15/2023
K=1
K=2
28
t
t
Y
net
2
0
1
120.00520.0
120.0120.0520.0
t
t
Y
net
3
0
2
096.00480.0
096.014.0480.0
04/15/2023
K=3
The result computed by the network after four recurrences indicates the vector x presented at i/p for mini hamming distance has been at the smallest HD from s1.
So, it represents the distorted character C.
29
t
t
Y
net
4
7
0
3
00461.0
10115.0461.0
04/15/2023
Noise is introduced in the input by adding random numbers.
Hamming Network and MaxNet network recognizes correctly all the stored strings even after introducing noise at the time of testing.
Effect of Noise in Inputs
30
04/15/2023
In the network, neurons are interconnected and every interconnection has some interconnecting coefficient called weight.
If some of these weights are equated to zero then how it is going to effect the classification or recognition.
The number of connections that can be removed such that the network performance is not affected.
Loss of Connection
31
04/15/2023
Missing information means some of the on pixels in pattern grid are made off.
For the algorithm, how many information we can miss so that the strings can be recognized correctly varies from string to string.
The number of pixels that can be switched off for all the stored strings in algorithm.
Missing Information
32
04/15/2023
Adding information means some of the off pixels in the pattern grid are made on.
The number of pixels that can be made on for all the strings that can be stored in networks.
Adding Information
33
04/15/2023
The network architecture is very simple.
This network is a counter part of Hopfield auto associative network.
The advantage of this network is that it involves less number of neurons and less number of connections in comparison to its counter part.
There is no capacity limitation.
Merits
34
04/15/2023
The hamming network retrieves only the closest class index and not the entire prototype vector.
It is not able to restore any of the key patterns. It provides passive classification only.
This network does not have any mechanism for data restoration.
It’s not to restore distorted pattern.
Demerits
35
04/15/2023
Jacek M. Zurada, “Introduction to artificial Neural Systems”, Jaico Publication House. New Delhi, INDIA
Amit Kumar Gupta, Yash Pal Singh, “Analysis of Hamming Network and MAXNET of Neural Network method in the String Recognition”, IEEE ,2011.
C.M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, Oxford, 2003.
References
36
04/15/2023 37
Thanks