september 15, 2010priti srinivas sajja neural network and its applications dr. priti srinivas sajja...

September 15, 2010 Priti Srinivas Sajja

Neural Network and its Applications

Dr. Priti Srinivas SajjaAssociate Professor

G H Patel P G Department of Computer Science and Technology

Sardar Patel University, Vallabh Vidyanagar 388 120, India

1

Priti Srinivas Sajja

Introduction and Contact Information Name: Dr. Priti Srinivas Sajja

Communication: Email : [email protected] Mobile : +91 9824926020 URL :http://pritisajja.info

Academic qualifications : Ph. D in Computer Science

Thesis title: Knowledge-Based Systems for Socio- Economic Development

Subject area of specialization : Artificial Intelligence

Publications : 84 in International/ National Conferences,

Journals & BooksSeptember 15, 2010 2


Lecture PlanIntroduction and Background

Artificial Intelligence and Data Pyramid Connectionist and Symbolic Systems Biological Neuron and Artificial Neuron Characteristics of Artificial Neural Network

Architecture of ANN Hopfield Model and Parallel Relaxation Single Perceptron and Linearly Separable Models Multi Layer Perceptron and Back Propogation Supervised Learning and Training Data

Example ANNsHybrid SystemsConclusion and References

September 15, 2010 3


Artificial Intelligence

“Artificial Intelligence(AI) is the study of how to make

computers do things at which, at the moment, people

are better” Elaine Rich, Artificial Intelligence,

McGraw Hill Publications, 1986 September 15, 2010 4


Knowledge-Based Systems (Symbolic Representation of Knowledge)

KK

Knowledge-Based Systems (KBS) are Productive Artificial Intelligence Tools working in a narrow

domain.September 15, 2010 5

Data

Information

Knowledge

Wisdom

Understanding

Experience

Novelty

Researching Absorbing Doing Interacting Reflecting

Raw Data through fact

finding

Concepts

Rules

Heuristics and

models

Convergence from data to intelligence


Structure of KBS

Knowledge Base

Inference Engine

User Interface

Explanation/ Reasoning

Self Learning

September 15, 20106


Limitations of Symbolic Systems

Nature of knowledge Hard to characterize Voluminous Dynamic

Knowledge acquisition Fact finding methods support only Tacit and higher level knowledge Multiple experts

Knowledge representation Limited knowledge structures support

KBS development models Only SAD/SE guidelines and a few quality metrics

Large size of knowledge base



Biological and Artificial Neurons

8


Artificial Neural Network(towards Connectionist Representation of Knowledge)

Objective: Not to mimic brain functionality but to receive inspiration

from the fact about how brain is working.

Characterized by: A large number of very simple neuron like

processing elements. A large number of weighted connection between

the elements. This weights encode the knowledge of a network.

Highly parallel and distributed control. Emphasis on learning internal representation

automatically.September 15, 2010 9


Architectures of ANN

Hopfield network Perceptron Multi-layer Perceptron Self Organizing Network etc.



In a Hopfield network, all processing units/elements

are in two states either active or inactive. Units are connected to each other with weighted

Connections.

A positively weighted connection indicates that the units tend to active each other.

A negative connection allows an active unit to deactivate a neighbouring unit.

Active

Inactive

-1

-1

-1

+1

+3

+3

-1

-2

+1

+2

+1


A Sim

ple H

opfield

Network

A Sim

ple H

opfield

Network


Paralle

l Rela

xatio

n

Paralle

l Rela

xatio

n

Active

Inactive

-1

-1

-1

+1

+3

+3

-1

-2

+1

+2

+1

• A random unit is chosen.• If any of its neighbours are active, the unit computes the sum of

weights on the connections to those active neighbours.• If the sum is positive, the unit becomes active else new random unit

is chosen.• This process will continue till the network become stable. That is no

unit can change its status. This process is known as parallel relaxation.



Perceptron

X1

X2

∑Wi Xi T

W1

W2

Mom

(0.3)

Dad

(0.5)

∑Wi Xi 0.6

0.6

0.4

Going to

Arm

y:

To B

e or n

ot to

Be? Goin

g to A

rmy:

To B

e or n

ot to

Be? Importance to Mom

Importance to Dad

= 0.3*0.6 + 0.5*0.4 = 0.18 +0.20= 0.38 which is < 0.6



Logical Gate AND and ORX1

X2

∑Wi Xi0.6

0.5

0.5

X1 X2 X1AND X2

0 0 0*0.5 + 0*0.5 = 0 <0.6 0

0 1 0*0.5 + 1*0.5 = 0 <0.6 0

1 0 1*0.5 + 0*0.5 = 0.5 <0.6 0

1 1 1*0.5 + 1*0.5 = 1 >0.6 1

Logica

l AND Tr

uth

Table

Logica

l AND Tr

uth

Table

(1,1) (1,1)



Logical Gate AND and OR

(1,1) (1,1)

(1,1)



X1

X2

1

ƒ

-1.5

1.0

1.0-9.0

X1

X2

1

ƒ

-0.5

1.0

1.0

K wo w1 w2

10 .41 -.17 .14

100 .22 -.14 .11

300 -.1 -.008 .07

635 -.49 -.1 .14

K wo w1 w2

10 .41 -.17 .14

100 .22 -.14 .11

300 -.1 -.008 .07

635 -.49 -.1 .14

A Perception Learning to Solve

a Classification Problem

A Perception Learning to Solve

a Classification Problem



O1

h1 h3h2 hB

1 x1 x2 x3 x4 xA

Oc O2Output Layer

Hidden Layer

Input Layer

W2ij

w1ij

Fully connected, multi layered,

feed-forward network structure

Fully connected, multi layered,

feed-forward network structure

……This network has three layers but there may be many.



Examples of Multilayer Perceptron

X1

X2

H1

H2

Two digite

dOne degited

Three

digited

O2

O1

O3

Training Set Data2, 3, 1, 0, 0

10, 10, 0, 1, 0

90, 90, 0, 0, 1

…… ….. …

X1 and X2 are two one /two digited positive numbers



Example of

Multilayer

PerceptronExample of

Multilayer

Perceptron

X1

X2

H1

H2

Nokia Base

Reliance

Nokia

HigherO2

O1

O3

Sony

Ericss

on

Blackberry

X3

X4

H3

H4

O4

O5

Budget

Color

Camera

Radio

Training Set Data2, 3, 0, 0, 1, 0, 0, 0, 0

6, 6, 7, 5, 0, 0, 1, 0, 0

8, 8, 8, 8, 0, 0, 0, 0, 1

…… ….. …



Example of

Multilayer

PerceptronExample of

Multilayer

Perceptron

X1

X2

H1

H2

Job B

Job A

Job CO2

O1

O3

X3

X4

H3

H4

Salary

Interest

Distance

Future

Design and Train the above structure using your own choices considering the following practical situations:

A. Job At Bengaluru, salary Rs.30, 000 per month of your field

B. Job At USA, salary Rs.80, 000 per month of other field

C. Job At Anand, salary Rs.25, 000 per month of your fieldSeptember 15, 2010 20


ANN to

Determine

Aptitude of U

sers

ANN to

Determine

Aptitude of U

sers



Source: www.wldelft.nl/cons/area/wds/neural/index.html

A method was developed to

provide a generic design tool for

estimating wave overtopping

discharges for a very wide range

of coastal structures.

A method was developed to

provide a generic design tool for

estimating wave overtopping

discharges for a very wide range

of coastal structures.



Sprayer Sensor and Nozzle Element (Zhang, Yang, & El-Faki, 1994)

ANN

Simulator ANN

Simulator

23


Optimization, function approximation, time series prediction, and modeling Classification, pattern matching and recognition (three-dimensional object

recognition), novelty detection, and sequential decision-making Data processing (including filtering, clustering, blind source separation and

compression), data mining, data compression (speech signal, image—for example, faces), and data validations

System identification and control (vehicle control, process control) and signal processing

Game playing and decision making (backgammon, chess, racing) Sequence recognition (gesture, handwritten text recognition) Medical diagnosis (for example, hepatitis or storing medical records based on

case information) Financial applications (automated trading systems, time series analysis, stock

market prediction) and customer research Cognitive science, neurobiology, and the study of models of how the brain works Biological neural networks, which communicate through pulses and use the

timing of the pulses to transmit information and perform computations Integration of fuzzy logic and neural networks for applications in automotive

engineering, screening applicants for jobs, controlling a crane, or monitoring a medical condition like glaucoma

Robotics (navigation and vision recognition) Speech production and recognition Vision (face recognition, edge detection, visual search engines) New topologies and hardware implementations New learning algorithms In hybrid systems and soft computing—for example, rule extraction for fuzzy

systems, self-evolving ANNs, and neuro-fuzzy systems

Applications of

ANN Applications of

ANN

24


Hybrid Systems/Soft Computing

Neural network

Probabilistic reasoning

Genetic algorithms

Fuzzy logic

Evolving neuro-fuzzy systems

Evolving neuro-fuzzy systems

Genetic bayesian network

Genetic bayesian network

Soft computing

Neuro-fuzzy and fuzzy neural

network

Neuro-fuzzy and fuzzy neural

network

Genetic fuzzyGenetic fuzzy

Modular rough networks

Modular rough networks

Application Layer 1

Application Layer 2

Constituents of soft computing



Strength of a Hybrid Soft Computing System

Fuzzy logicFuzzy logic Approximate reasoning, impressionsApproximate reasoning, impressions

Field Strength Offered

Artificial neural networkArtificial neural network Learning and implicit knowledge representation

Learning and implicit knowledge representation

Genetic algorithmGenetic algorithmNatural evolution and optimizationNatural evolution and optimization

Probabilistic reasoningProbabilistic reasoning UncertaintyUncertainty



Hybrid Applications

Data ANNFuzzy Sets

Fuzzy Rules

FIS

Output

Y1

Y2

YN

X1

X2

Xn

Fuzzy

Interface

(a) fuzzy neural model (b) co-operative neuro-fuzzy model

Data ANNFIS

Output

(c) concurrent neuro-fuzzy model (d) hybrid neuro-fuzzy model

Approaches of neuro-fuzzy computing

Neuro-fuzzy Systems

Neuro-fuzzy Systems



Reference

Knowledge-Based SystemsRajendra Akerkar and Priti Srinivas Sajja

Jones & Bartlett PublishersSudbury, MA, USA (2009)



To the CVM and ISTAR Family

Vidyanagar, Gujarat, India


september 15, 2010priti srinivas sajja neural network and its applications dr. priti srinivas sajja...

Documents

intelligence slide

knowledgebased systems

artificial neurons

data pyramid connectionist

models convergence

simple neuron

computer science thesis

contact information