deep learning実装の基礎と実践
TRANSCRIPT
- 1. Deep LearningDeep Learning2014/08/26 Preferred Networks
- 2. l (Seiya Tokui)l Preferred Networks l Jubatus Jubatus: NTTPFI http://jubat.us/ l 2
- 3. l SGDl Pylearn2, Torch7, Caffe, Cuda-convnet2l l 3
- 4.
- 5. Deep LearningNeural Network: Convolutional NN: NNDeep Belief Network(),Deep Boltzmann Machine():Deep 35
- 6. Feed-Forward Neural Networkx 1wj1x2wj2x3wj3x4wj4hj = f(wj1x1 + + wj4x4 + bj) = f(wj x + bj)
- 7. Feed-Forward Neural Networkh = f(Wx+ b)3y = f3(W3f2(W2f1(W1x + b1) + b2) + b3)W1 W2 W3x h1 h2 y
- 8. l lossl L(W) =1NXNi=1`(y(xi;W), yi)l `y(x;W)(xi, yi)i=1,...,NWl 8minimize WL(W)`(y, yi) = ky y1k2
- 9. gradient W W rL(W)9L
- 10. Stochastic Gradient Descent (SGD)l l 10B|B|W W |B|Xi2Br`(y(xi;W), yi)
- 11. l SGD l 11h h + 1|B|Xni2Br`(y(xi;W), yi),W W h.W(t)r`W(t