Artificial Neural NetworksArtificial Neural Networks

An Overview and Analysis

Modeling the Human BrainModeling the Human Brain

Artificial Neural Networks concentrate on imitating humans rather than acting as rational agents.

The goal in ANNs is to imitate the learning

that takes place in the brain.

The motivation behind this was to find a way

for a machine to LEARN.

Structure of an ANNStructure of an ANN The basic function component of an ANN is the

unit.

These units are connected to each other through links.

Each link has a weight associated with it. These weights are the means of long term storage in ANNs. Also, learning is usually accomplished by updating these weights.

Some units are connected to the outside world, and are designated as input or output units.

Units on the same level make up a layer.

ANN LearningANN Learning

An ANN learns when the weights of the links are adjusted.

Learning is complete when desired output is

very close to actual output.

Learning takes place when desired output

and actual output are compared.– The difference between the two is measured

and adjustments are made to the weights

inside the ANN.

Network StructuresNetwork Structures

Feedforward– Perceptrons– Multilayered

Feedback– Recurrent

Feedforward SystemsFeedforward Systems

Links are unidirectional

Acyclic

In a typical, layered Feedforward network,

each unit is linked only to units in the next

layer.

PerceptronsPerceptrons

Single layered networks.

Perceptron learning is very easy.

However, only linearly separable functions

can be represented by perceptrons.

Optimal Linear Associative MemoryOptimal Linear Associative Memory

Architecture: Single layer Feedforward System.

MultilayerMultilayer

Contain one or more layers of “hidden” nodes.

Not limited to Linearly Separable functions.

Can learn any function

There exists a popular method for learning:

back-propagation.

Maxnet-Hamming NetworkMaxnet-Hamming Network

Architecture: Feedforward Multilayer System

Feedback SystemsFeedback Systems

Cyclic

Output can be directed back as inputs to

previous or same level nodes.

Much more complex than a Feedforward

system.

A Recurrent system is simply a Feedback

system with closed loops.

Adaptive Resonance TheoryAdaptive Resonance Theory

Architecture: Bi-directional Feedback System

ApplicationsApplications

Handwriting Character Recognition

English Text Pronunciation

Driving– ALVINN (Automated Land Vehicle In a Neural Network):

requires 5 minutes of watching a human drive, and 10

minutes of back-propagation. Can drive at speeds of up to

70 mph for about 90 minutes.

Classification

Multilayer vs. PerceptronMultilayer vs. Perceptron

Perceptron learns the fastest.

Perceptrons have a limited learning capacity

Perceptron is too simple for most practical

applications.

Multilayer vs. FeedbackMultilayer vs. Feedback

A Feedback System more closely models the brain.

Both systems can learn any formula.

A Multilayer system uses less overhead.

Feedback Systems are a lot more complex.

Most Efficient ANNMost Efficient ANN

Multilayered Feedforward system

– Relative simplicity

– Learning Capacity

– The Back-Propagation Algorithm is very good.

The EndThe End

Any Questions?