chapter0. overview of the neural network
DESCRIPTION
Chapter0. Overview of the Neural Network. Neural Network = Computational Structure of the Brain = Distributed, Adaptive, Nonlinear Learning Machine built from many Processing Elements - PowerPoint PPT PresentationTRANSCRIPT
1
1. Brain = Neural Network in Topology
Chapter0. Overview of the Neural NetworkNeural Network = Computational Structure of the Brain= Distributed, Adaptive, Nonlinear Learning Machine built from many Processing Elements
Synonyms :
Artificial Neural System, Artificial Neural Network, Neuromorphic System,Parallel Distributed Processing, Adaptive Network, Connnectionism, Neurocomputer.
Cell body
2
● Sensory Perception (Pattern Recognition) and motor control – Vision, audition, olfaction, touch, temperature sensing
● Learning By Examples – Non-algorithmic, Trainability, Self-Organization
● Planning and Reasoning
● Learning and Adaptation by Generalization
● Reflex and Intuition (Similar to Table Lookup)
● Processing Ill-defined (Unstructured, Inconsistent, Probabilistic, Noisy) Information – fault-tolerant, flexible, robust
● As Computer: Massively parallel and distributed : Algorithm + Architecture
● As Computer: Wetware (Netware) vs. Software/Hardware ● As Math: Nonlinear and Adaptive Modeling Scheme
2. Features of the Human Brain (Differences from Digital Computers)
3
At birth, 10 11 neurons Innate, Tends to Decrease with Time. However, their interconnections evolve and new ones are created.
4
● Webster = Ability to learn or understand or to deal with new or trying situations : REASON
● Jang = Humanlike Expertise, adapt and learn to do better in changing env. And explain its decisions/actions
● Constituent Tech. = Synergy of NN, FL, EC EC = Systematic random search = Biological genetics + Natural selection)
● CI = Any methodology involving computing exhibiting ability to learn (do better) with new situations by reasoning (generalization, discovery, association, abstracting); also explain how it reasons.
Intelligence ?
3. Intelligent [Learning] Machines
5
History : Intelligence to Machines Freedom to Mankind !
Industrial Revolution (Machine) Information Technology Revolution Artificial Brain (Intelligence) Revolution : Creative, human-friendly, autonomous (adaptive) Brain = Final Frontier = Neural Nets
● Left Brain (Logic) – Program (Symbolic, Software, Structured) Machine Learning, Expert System : AI
● Right Brain (Intuition, Emotion) – Neural Network (Numeric, Hardware, Unstructured) Cybernetics, Bionics
6
(Ref. Eberhart, Chap. 1 of Computational Intelligence, IEEE Press, 95)
Intelligence
CIBI
AI
SymbolicComp.
Numeric Comp.
Biology
MI = AI or CI
NN
FL
EC
NFE
NE
NF
FE
CI
BI
Intelligence
Bezdek
4. Computational Intelligence and Soft Computing
7
8
9
10
5. Neural Network Architecture
w1
dendrite
cell (soma)
axon
x1
wm
xm
wm+1xm+1
y
ftn activation :
1
1
m
jjj xwy
Activation function
Linear Combination
11
12
13
(1) Feedforward Architecture
Input
Hidden Output
.... ....General
Layered(3-2-2)
Input Hidden Output
14
15
(2) Feedback, Recurrent Architecture
16
17
18
19
6 . Usage of the Neural Network – Function Approximation and Generalization
◆ Training
NN
NN
Teacher
Word
e-
+
Voice NN
Teacher
Object Image
e
Object Name
-
+
Amorphous
◆ Training
20
◆ Apply to Different Tasks After Training
NN
Voice Word
Object Image Object Name
21
Learning Mode (weights change) Performance Mode (weights fixed)
◆ Training – Function Approximation = Create Internal Representations Only through Examples
NeuralNetwork
x)(),( xwxy fF
y
x
y f(x)
f (x): Normally Unknown
7. Generalization By Nonlinear Interpolation
Digital Computer vs. Neural Computer
• Digital – recalculate even for same inputs
• Neural – can memorize and recall results of previous calculation [previous answers].
if w is fixed after training →
NN is a Model-Free EstimatorTraining Data
Discrete Samples
Underlying Function
22
◆ Generalization
xy = F ( x , w * )
NN
Function Approximation
F ( x , w * )
x
F ( x , w )F ( x , w " )
F ( x , w ' )
f ( x ) ② ①
②
① ① Approximation ② Generalization
23
24
25
26
8. Applications
1) Pattern Recognition & Character Recognition, Document Search
Face and Speech Recognition - Biometrics
Computer Vision : Image Understanding, Object Recognition, PCB Inspection
2) Model Building from Experimental Data: Function Mapping,
Regression, System Identification, Data Mining (Knowledge Discovery), Prediction
3) Image / Signal Processing and Communication
4) Optimization
5) Time Series Analysis and Financial Engineering
6) Medicine - Patient Care and Clinical Decision Support,
Biomedical Engineering, Bio-mimetics, Bio-informatics
7) Robotics and Automation
◆ Service Robots, Pet Robots, Surveillance Robots
27
◆ Process Control Kodak - Film Making Amoco – Oil Exploration ◆ Aircraft Control
◆ Automotive Control ◆ Machine Control / Maintenance ◆ Machine Health Monitoring / Diagnosis ◆ Diagnostics and Quality Control ◆ Power System Control ( Canada Vancouver Island Power) ◆ Chemical Product Design ( AIWARE 사의 CAD/Chem ) ◆ Airline Luggage Inspection System
( -20 % cost + 50 % performance ) ◆ Active Vibration Cancellation
8) Music Composition 9) Neural Network Products in Korea:
Green Technology – Counterfeit Recognizer ( W 66B for 5 yrs) Korea Axis – Speech Recognition Toy (2000. 12) Slip Processing Machine for Banks using Handwritten Recognition (2000.12) Speech Recognition Chip Product which is Robust to Noise Applying Human
Auditory Model (2000. 7)
28
Industrial App of CIIndustrial App of CIGE
Imagination at work
Insurance underwriting, proactive maintenance Recom.
Paper Web time-to-break prediction with Fuzzy+nn
Equip Prognosis, anomally dete
CI model = domain knowledge + field data
Ford: CIS system
DOD Air Force Res. Lab.
Image Patterns below Clutter
Tracking & detection below clutter
NN appli – bioinfo, drug design, financial mkr pred, internet search engine, medical app, 30 commercial componemts,
Future Dir.
Drug design (big) nat lang under, search eng, high leve sensor fusion, sensor-web, neural & psycholinguistic study – working of mind, cog and emo, lang & musi
c.
29
9. NN Development Tools
◆ TypesGeneral Purpose Computer
1) S/W Simulation 2) S/W Simulation with H/W Accelerators3) S/W Simulation on a Parallel Computer
Special Purpose H/W1) Neurocomputing Workstations2) Electronic - VLSI 3) Optical - Laser Holography
10. Government Sponsored NN Research Worldwide 1990-2000 Decade of the Brain (US)1990-2090 Century of the Brain (Japan)1998-2007 Braintech 21(Korea) – Brain Research Promotion Act