a fast nearest neighbor classifier based on self-organizing incremental neural network (soinn)
DESCRIPTION
A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN). Presenter : Lin, Shu -Han Authors : Shen Furao ,, Osamu Hasegawa. Neuron Networks (NN, 2008). Outline. Introduction Motivation Objective Methodology Experiments Conclusion Comments. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/1.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
A fast nearest neighbor classifier based on
self-organizing incremental neural network(SOINN)
Neuron Networks (NN, 2008)
Presenter : Lin, Shu-HanAuthors : Shen Furao,, Osamu Hasegawa
![Page 2: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/2.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
2
Outline
Introduction Motivation Objective Methodology Experiments Conclusion Comments
![Page 3: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/3.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
3
Introduction - self-organizing incremental neural network (SOINN)
Distance: Too far
Node = prototype
![Page 4: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/4.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
4
Introduction - self-organizing incremental neural network (SOINN)
Link age
![Page 5: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/5.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
5
Introduction - self-organizing incremental neural network (SOINN)
Age: Too old
![Page 6: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/6.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
6
Introduction - self-organizing incremental neural network (SOINN)
Run two times
Insert node if error is large
Cancel Insertion if insert is no use
![Page 7: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/7.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
7
Introduction - self-organizing incremental neural network (SOINN)
Run two times
Delete outlier:Nodes without
neighbor(low-density assumption)
![Page 8: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/8.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Motivation
SOINN classifier (their first research in 2005) Use 6 user determined parameters Do not mentioned about noise Too many prototypes Unsupervised learning
Their second research (in 2007)talk aboutthese weakness
8
![Page 9: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/9.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
9
Objectives
Propose a Improved version of SOINN, ASC (Adjust SOINN Classifier) FASTER: delete/less prototype
Training phase Classification phase
CLASSIFIER: 1-NN (prototype) rule INCREMENTAL LEARNING ONE LAYER: easy to understand the setting,
less parameters~ MORE STABLE: help of k-means
![Page 10: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/10.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Methodology – Adjusted SOINN
10
Distance: Too far
A node is a cluster
![Page 11: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/11.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Methodology – Adjusted SOINN
11
Link age
![Page 12: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/12.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Methodology – Adjusted SOINN
12
Winner
Neighbor
![Page 13: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/13.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Methodology – Adjusted SOINN
13
Age: Too old > ad
![Page 14: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/14.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Methodology – Adjusted SOINN
14
Delete outlier:Nodes without
neighbor(low-density assumption)
![Page 15: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/15.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Methodology – Adjusted SOINN
15
Lambda = iterations
![Page 16: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/16.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Methodology – k-means
16
Help of k-means clustering, k = # of neurons Adjust the result prototypes: assume that each node nearby the centroid
of class
![Page 17: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/17.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Methodology – noise-reduction
17
Help of k-Edit Neighbors Classifier (ENC), k=? Delete the node which label are differs from the majority voting of its
k-neighbors: assume that are generated by noise
![Page 18: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/18.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Methodology – center-cleaning
18
Delete neurons: if it has never been the nearest neuron to other class: assume that are lies in the central part of class
![Page 19: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/19.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Experiments: Artificial dataset
19
dataset Adjusted SOINN
ASC
Error: sameSpeed: faster
![Page 20: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/20.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Experiments: Artificial dataset
20
dataset Adjusted SOINN
ASC
Error: sameSpeed: faster
![Page 21: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/21.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Experiments: Artificial dataset
21
dataset Adjusted SOINN
ASC
Error: betterSpeed: faster
![Page 22: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/22.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Experiments: Artificial dataset
22
dataset Adjusted SOINN
ASC
Error: betterSpeed: faster
![Page 23: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/23.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.Experiments: Real dataset
23
Compression ratio (%)
Speed up ratio (%)
![Page 24: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/24.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
Experiments: Compare with other prototype-based classification method
24
Nearest Subclass Classifier (NSC) k-Means Classifier (KMC) k-NN Classifier (NNC) Learning Vector Quantization (LVQ)
![Page 25: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/25.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
Experiments: Compare with other prototype-based classification method
25
![Page 26: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/26.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
26
Conclusions
ASC Learns the number of nodes needed to determine the decision boundary Incremental neural network Robust to noisy training data Fast classification Fewer parameters: 3 parameters
![Page 27: A fast nearest neighbor classifier based on self-organizing incremental neural network (SOINN)](https://reader036.vdocuments.site/reader036/viewer/2022062400/56815e44550346895dccb742/html5/thumbnails/27.jpg)
Intelligent Database Systems Lab
N.Y.U.S.T.I. M.
27
Comments
Advantage Improve many things A previous paper to demonstrate the thing they want to modify
Drawback NO Suggestion of parameters
Application A work from unsupervised learning to supervised learning