mathematical foundation of discrete time hopfield networks

A Seminar Presentation for the degree of

Master of Technology in

Computer Science and Engineering

PRESENTED BY:-AKHIL UPADHYAYM-TECH 3rd SEM CSEROLL NO.- 121140002

SUBMITED TO:-MR. ROHIT MIRIH.O.D. OF COMPUTER SCIENCE DEPARTMENT

Mathematical Foundation of Discrete time Hopfield Networks

INTRODUCTION A Hopfield Networks is a form of recurrent artificial neural Network popularized by John Hopfield in 1982, but described earlier by Little in 1974

Hopfield has developed a number of neural Networks based on fixed weights and adaptive activations

These Networks can serve as associative memory Networks and can be used to solve constraint satisfaction problems such as the "Travelling Salesman Problem (Cont..)

Two types:

1. Discrete Hopfield Network.

2. Continuous Hopfield Network.

Discrete Hopfield Network

Hopfield has proposed two basic models of associative memories (Hopfield 1982, 1984).

(Cont..)

Discrete Hopfield NetworkThe first of these is a ‘DISCRETE MODEL’ while the second is a ‘CONTINUOUS’ version of the same.

The terms ‘DISCRETE’ or ‘CONTINUOUS’ refer to the nature of the state variables and time, in these models.

In the discrete Hopfield network, each neuron has a binary state

{1,-1} The state of the network with N neurons is represented

by the vector

(Cont..)


V={ The network is fully-connected, i.e., each neuron

connected to all others.

The weight from j’th neuron to i’th neuron is given by, and weight matrix is given as

W={} Since the network has loops, computations are dynamic

and the network state evolves through time, which is a discrete variable.

(Cont..)


Hopfield net differ from iterative auto associative net in 2

things.

1. Only one unit updates its activation

at a time (based on the signal it receives from each other

unit)

2. Each unit continues to receive an

external signal in addition to the signal from the other units

in the net.

(Cont..)

Surprise

The asynchronous updating of the units allows a function,

known as an energy function, to be found for the net.

The existence of such a function enables us to prove that the

net will converge to a stable set of activations, rather than

oscillating.

The original formulation of the discrete Hopfield net showed

the usefulness of the net as content-addressable memory.

(Cont..)


(Cont..)


Algorithm There are several versions of the discrete Hopfield net.

Binary Input Vectors

To store a set of binary patterns s ( p ) ,

p = 1 , . . . , P, where

))().....().....(()( 1 pspspsps ni

(Cont..)


The weight matrix W = is given by}{ ijw

]12][12[ )()( pjp

piij sswji for

and

.0iiw

(Cont..)

Discrete Hopfield Network Bipolar Inputs

To store a set of binary patterns s ( p ) ,

p = 1 , . . . , P, where

))().....().....(()( 1 pspspsps ni

The weight matrix W = is given by,}{ ijw

)()( pjp

piij ssw ji for

and0iiw (Cont..)

PROPERTIES OF HOPFIELD NETWORK

A recurrent network with all nodes connected to all other nodes. Nodes have binary outputs (either 0,1 or -1,1). Weights between the nodes are symmetric . No connection from a node to itself is allowed. Nodes are updated asynchronously ( i.e. nodes are selected at random). The network has no hidden nodes or layer.

(Cont..)


Applications:-A binary Hopfield net can be used to determine whether an input

vector is a "known” or an "unknown" vector.

The net recognizes a "known" vector by producing a pattern of

activation on the units of the net that is the same as the vector

stored in the net.

If the input vector is an "unknown" vector, the activation vectors

produced as the net iterates will converge to an activation vector

that is not one of the stored patterns.

Thank You