「知識」のdeep learning
TRANSCRIPT
Distance model (Structured Embedding) [Bordes+11]
! e ! Rleft, Rright
!
! f
f(x, r, y) = || Rleft(r) e(x) – Rright(r) e(y) ||1
Neural Tensor Network (NTN) [Socher+13]
f(x, r, y) = ur tanh(e(x)Wre(y) + V1
r e(x) + V2r e(y) + br)
! r *5'3 !
1/4
! [Mikolov+10] T. Mikolov, M. Karafiat, L. Burget, J. H. Cernocky, S. Khudanpur. Recurrent neural network based language model. Interspeech 2010.
! [ 15] . kUCIv7���!2�<t.
2015. ! [ 15] .
NLP Introduction based on Project Next NLP. PyData.Tokyo Meetup #5, 2015.
! [Bordes&Weston14] A. Bordes, J. Weston. Embedding Methods for Natural Language Processing. EMNLP2014 tutorial.
! [ 15] . j`�<oCI�^HG�MW1+3�=e. JSAI 2015 .
2/4
! [Bordes+11] A. Bordes, J. Weston, R. Collobert, Y. Bengio. Learning structured embeddings of knowledge bases. AAAI2011.
! [Bordes+13] A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, O. Yakhnenko. Translating Embeddings for Modeling Multi-relational Data. NIPS 2013.
! [Fan+14] M. Fan, Q. Shou, E. Chang, T. F. Zheng. Transition-based Knowledge Graph Embedding with Relational Mapping Properties. PACLIC 2014.
! [Wang+14] Z. Wang, J. Zhang, J. Feng, Z. Chen. Knowledge Graph Embedding by Translating on Hyperplanes. AAAI 2014.
3/4
! [Socher+13] R. Socher, D. Chen, C. D. Manning, A. Y. Ng. Reasoning With Neural Tensor Networks for Knowledge Base Completion. NIPS 2013.
! [Yang+15] B. Yang, W. Yih, X. He, J. Gao, L. Deng. Embedding Entities and Relations for Learning and Inferenece in Knowledge Bases. ICLR 2015.
! [Nickel+11] M. Nickel, V. Tresp, H. P. Kriegel. A Three-Way Model for Collective Learning on Multi-Relational Data. ICML 2011.
4/4
! [Bordes+14] A. Bordes, J. Weston, N. Usunier. Open Question Answering with Weakly Supervised Embedding Models. ECML/PKDD 2014.
! [Weston+15] J. Weston, S. Chopra, A. Bordes. Memory Networks. ICLR 2015.
! [Peng&Yao15] B. Peng, K. Yao.
Recurrent Neural Networks with External Memory for Language Understanding. arXiv:1506.00195, 2015.