understanding rbm by wangyuantao
DESCRIPTION
TRANSCRIPT
![Page 1: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/1.jpg)
Understanding RBM
Wang Yuantao
08/22/2009
![Page 2: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/2.jpg)
2
Outline
• RBM Model– For Netflix Prize Problem
• RBM Algorithm– Implementation– Technical detail
• Contribution– Model– Result
![Page 3: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/3.jpg)
3
Taxonomy
• By Determinacy – Deterministic: SVD/kNN– Stochastic: RBM
• By Optimisation– Empirical: kNN– Optimal
• Gradient descent: SVD• Maximum likelihood: RBM
![Page 4: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/4.jpg)
4
Model Structure
By Hinton
![Page 5: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/5.jpg)
5
Model Assumption
• 1
• 2
![Page 6: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/6.jpg)
6
Optimal Object
Maximize:
![Page 7: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/7.jpg)
7
Training
Sampling
![Page 8: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/8.jpg)
8
Training Phases
1. Init v0 by real rating
2. Sample h0 by v0
3. Reconstruct v1 by h0
4. Re-sample h1 by v1
5. Update W
6. Compute Eh
![Page 9: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/9.jpg)
9
Prediction
![Page 10: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/10.jpg)
10
Technical detail
• Sparseness• Sample
– Gibbs 1-step sampling
• Update– Batch v.s. online(per-case)– Training bias
• Learning rate– Weight Decay– Momentum– Anneal
![Page 11: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/11.jpg)
11
Temporal RBM
F=16, lrate=0.002
Original RBM 0.9392
0.9385
0.9381
0.9373
![Page 12: Understanding Rbm by WangYuanTao](https://reader035.vdocuments.site/reader035/viewer/2022073116/54b74b654a7959ef448b4610/html5/thumbnails/12.jpg)
12
Contribution
0.8694
0.8688
RBM