zhonglong zheng, jie yang, yitan zhu engineering applications of artificial intelligence 20 (2007)...
DESCRIPTION
3TRANSCRIPT
![Page 1: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/1.jpg)
Initialization enhancer for non-negative matrix factorization
Zhonglong Zheng, Jie Yang, Yitan Zhu
Engineering Applications of Artificial Intelligence 20 (2007) 101–110
Presenter Chia-Cheng Chen 1
![Page 2: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/2.jpg)
Introduction
Non-negative matrix factorization algorithm
Initializing NMF with different techniques
Experimental results
Conclusion
Outline
2
![Page 3: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/3.jpg)
Background(1/2)
3
![Page 4: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/4.jpg)
Background(2/2)
4
![Page 5: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/5.jpg)
NMF has been applied to many areas such as dimensionality reduction, image classification, image compression.
However, particular emphasis has to be placed on the initialization of NMF because of its local convergence, although it is usually ignored in many documents.
Introduction
5
![Page 6: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/6.jpg)
Non-negative matrix factorization (NMF) algorithm
where
Dimensionality reduction is achieved when r < N
Non-negative matrix factorization algorithm(1/4)
6
![Page 7: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/7.jpg)
Euclidean distance
◦Update rule
Non-negative matrix factorization algorithm(2/4)
7
![Page 8: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/8.jpg)
KL divergence
Update rule
Non-negative matrix factorization algorithm(3/4)
8
![Page 9: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/9.jpg)
SJTU-face-database◦ 400 images ◦ Size: 64x64
Non-negative matrix factorization algorithm(4/4)
9
![Page 10: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/10.jpg)
Three techniques
◦ PCA-based initialization
◦Clustering-based initialization
◦Gabor-based initialization
Initializing NMF with different techniques(1/5)
10
![Page 11: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/11.jpg)
PCA-based initialization
m x N matrix X
Use SVD compute the eigenvectors and eigenvalues
Initializing NMF with different techniques(2/4)
11
![Page 12: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/12.jpg)
PCA-based initialization
Initializing NMF with different techniques(3/5)
12
![Page 13: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/13.jpg)
Clustering-based initialization (Fuzzy c-means) Membership matrix
Objective function
Update rule
Initializing NMF with different techniques(4/5)
13
![Page 14: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/14.jpg)
Gabor-based initialization Gabor kernals
where
Gabor feature
Initializing NMF with different techniques(5/5)
14
![Page 15: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/15.jpg)
Experimental results
15
![Page 16: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/16.jpg)
Experimental results
16
![Page 17: Zhonglong Zheng, Jie Yang, Yitan Zhu Engineering Applications of Artificial Intelligence 20 (2007) 101110 Presenter Chia-Cheng Chen 1](https://reader035.vdocuments.site/reader035/viewer/2022062401/5a4d1b6a7f8b9ab0599b2a9d/html5/thumbnails/17.jpg)
Non-negative matrix factorization is a useful tool in the analysis of a diverse range of data.
Researchers often take random initialization into account when utilizing NMF.
In fact, random initialization may make the experiments unrepeatable because of its local minima property, although neural networks are not.
Conclusion
17