mehdi ghayoumi msb rm 132 [email protected] ofc hr: thur, 11-12 a machine learning
TRANSCRIPT
Mehdi Ghayoumi
MSB rm 132
Ofc hr: Thur, 11-12 a
Machine Learning
Machine Learning
THE NAÏVE BAYES CLASSIFIER
In the naïve Bayes classification scheme, the required
estimate of the pdf at a point x=[x(1),...,x(l)]T R∈ l is given as
That is, the components of the feature vector x are assumed
to be statistically independent.
Machine Learning
Example .
Generate a set X1 that consists of N1 = 50 5-dimensional data
vectors that stem from two classes, ω1 and ω2. The classes are
modeled by Gaussian distributions with means
m1 = [0,0,0,0,0]T and m2 = [1,1,1,1,1]T and respective covariance matrices
Machine Learning
Step 1. Classify the points of the test set X2 using
the naive Bayes classifier, where for a given x, p(x|
ωi ) is estimated as
Machine Learning
Step 2. Compute the ML estimates of m1, m2, S1,
and S2 using X1. Employ the ML estimates in the
Bayesian classifier
Machine Learning
Step 3. Compare the results obtained in steps 1 and 2.The resulting classification errors—naive_error and
Bayes_ML_error—are 0.1320 and 0.1426,
respectively.
In other words, the naive classification scheme
outperforms the standard ML-based scheme.
Machine Learning
The techniques that are built around the optimal
Bayesian classifier rely on the estimation of the pdf
functions describing the data distribution in each
class.
Machine Learning
The focus is on the direct design of a discriminant
function/decision surface that separates the classes
in some optimal sense according to an adopted
criterion.
Machine Learning
Machine learning involves adaptive mechanisms that enable computers to Machine learning involves adaptive mechanisms that enable computers to
learn from experience, learn by example and learn by analogy. Learning learn from experience, learn by example and learn by analogy. Learning
capabilities can improve the performance of an intelligent system over capabilities can improve the performance of an intelligent system over
time. The most popular approaches to machine learning are time. The most popular approaches to machine learning are artificial artificial
neural networks neural networks and and genetic algorithmsgenetic algorithms. This lecture is dedicated to . This lecture is dedicated to
neural networks.neural networks.
• Networks of processing units (neurons) with connections (synapses)
between them
• Large number of neurons: 1010
• Large connectitivity: 105
• Parallel processing
• Distributed computation/memory
• Robust to noise, failures
Machine Learning
Understanding the Brain• Levels of analysis (Marr, 1982)
1. Computational theory
2. Representation and algorithm
3. Hardware implementation
• Reverse engineering: From hardware to theory
• Parallel processing: SIMD vs MIMD
Machine Learning
Real Neural Learning
• Synapses change size and strength with experience.
• Hebbian learning: When two connected neurons are
firing at the same time, the strength of the synapse
between them increases.
• “Neurons that fire together, wire together.”
Machine Learning
Machine Learning
BiologicalNeuralNetwork Artificial NeuralNetworkSomaDendriteAxonSynapse
NeuronInputOutputWeight
Neural Network Learning
• Learning approach based on modeling adaptation in
biological neural systems.
• Perceptron: Initial algorithm for learning simple neural
networks (single layer) developed in the 1950’s.
• Backpropagation: More complex algorithm for learning
multi-layer neural networks developed in the 1980’s.
Machine Learning
Perceptron Learning Algorithm
• First neural network learning model in the 1960’s
• Simple and limited (single layer models)
• Basic concepts are similar for multi-layer models so this
is a good learning tool
• Still used in many current applications
Machine Learning
Machine Learning
Step function Sign function
+1
-1
0
+1
-1
0X
Y
X
Y
1 1
-1
0 X
Y
Sigmoid function
-1
0 X
Y
Linear function
0if,0
0if,1
X
XYstep
0if,1
0if,1
X
XYsign X
sigmoid
eY
1
1XYlinear
x1
xn
x2
w1
w2
wn
z
• Learn weights such that an objective
function is maximized.
• What objective function should we use?
• What learning algorithm should we use?
Machine Learning