model-based clustering by probabilistic self-organizing maps
DESCRIPTION
Model-Based Clustering by Probabilistic Self-Organizing Maps. Presenter : Chien-Hsing Chen Author: Shih-Sian Cheng Hsin-Chia Fu Hsin -Min Wang. 2009.IEEE TNN.22. Outline. Motivation Objective Method Experiments Conclusion Comment. Motivation. develop a mixture clustering model - PowerPoint PPT PresentationTRANSCRIPT
Intelligent Database Systems Lab
國立雲林科技大學National Yunlin University of Science and Technology
Model-Based Clustering by Probabilistic Self-Organizing Maps
Presenter: Chien-Hsing Chen
Author:
Shih-Sian Cheng
Hsin-Chia Fu
Hsin-Min Wang
1
2009.IEEE TNN.22
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
2
Outline Motivation Objective Method Experiments Conclusion Comment
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
3
develop a mixture clustering model EM, CEM, DAEM are applied to combine with PbSOM
Background knowledge competition, cooperation in SOM EM (E-step, M-step), K-means ? likelihood multivariate Gaussian distribution when K-means = SOM ?
Motivation
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
4
1. introduce three approaches, and a PbSOM2. combine the three approaches with PbSOM
Objective
PbSOM
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
5
EM
1 5 9
26
488
=p(2; θk=1)= the value is large
p(48; θk=1)= the value is small
expect that each xi can be close to a certain k
5
6
assume t=15need update?
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
6
CEM
1 5 9
2 6488
expect that each k has good quality of data projection
5
4
3
assume t=15need update?
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
7
DAEM
1 5 9
26
488
expect that do not believe f(k|xi ; θ) too much, when t=1 believe f(k|xi ; θ) larger, when t=10
EM:
large
5
4
small
initialization bias, thenlocal optimal, <1 gradual increase to 1
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
8
EM
CEM
DAEM
EM based approaches
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
9
Overall
PbSOM
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
10
Principle concept of PbSOM
3
53
2
23
87
53
98 9
8
1
xi xi
xi
When selecting the winning neuron,PbSOM considers the neighborhood information;in contrast, SOM, does not.
|| 3- 1||
|| 3-8 |||| 3-9 ||
|| 8-1 |||| 8-1 ||
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
11
PbSOM (Probabilistic SOM) xi k, if k=arg mink ||xi - nk||
xi
xi
(energy function to be maximized)
5
4
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
12
Multivariate Gaussian distribution
lx1
x7
x5
x8
x1, x5, x7, x8~iid~N(ul, )
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
13
PbSOM (Probabilistic SOM)
5
4
xi
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
14
PbSOM (Probabilistic SOM)
5
4
xi
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
15
Overall
PbSOM
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
16
EM
CEM
DAEM
EM based approaches
PbSOM
h
h
h h
h
h
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
17
CEM
SOCEM (PbSOM+CEM)
h h
1 5 9
2 6488
conversional SOM update:||xi - nl||||nk - nl||
batch update:xi / N||nk - nl||
3
similar to a batch K-means algorithm with considering h
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
18
EM
SOME (PbSOM+EM)
h
h
1 5 9
26
488
similar to a batch K-means algorithm with considering h
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
19
DAEM
SODAEM (PbSOM+DASOM)
h h
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
20
overall
PbSOM
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
21
Experiment- SOCEM
σ= 0.6
σ= 0.45 σ= 0.3 σ= 0.15
σ in hkl gradually recued 0.6 to 0.15
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
22
Experiment- SOEM
compare to previous page, this result is more global
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
23
Experiment- SODAEM
SODAEM is almost equivalent to SOME and SOCEM, respectively. When uses different β.It is not able to obtain ordered map during learning process if the value of σ is to small.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
24
Experiment-SOCEM
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
25
Experiment-SOEM
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
26
Experiment-SODAEM
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
27
Experiment
stability
without PbSOM
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
28
Experiment 1/2 distinguishKohonenSOM
SOCEM
SOCEM
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
29
Experiment
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
30
Experiment
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
31
Experiment
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
32
Experiment
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
33
Experiment
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
34
Conclusion
PbSOM
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
35
Comment Advantage
a mixture approach, sounds solid, is presented
Drawback less novelty Is it better than conversional SOM?
Application SOM