表現学習時代の生成語彙論ことはじめ
DESCRIPTION
生成語彙論の入門と、単語の表現学習、compositionalityの学習に関する入門ですTRANSCRIPT
¿Ó½Ǝ�»: �ÐŇǖĦ!5;&D�
�ĉ�Preferred Infrastructure Ù¦�Ʋǁ (@unnonouno) �
2014/10/16 PFIVZX^�
�ƝLJŘ�
Ù¦�Ʋǁ (@unnonouno) ! �ŪåŇĭ¸GdV\fuSh�XGÁnj½Ǝ ! ĘƬĭ¸E/H05GKA'
! ��ú߸ýŵġ�IBMôLjŎ�PFI
8�
mrep5��c�}P1�KA%-�
! řĹƑƚ8�%3ÖĨåŇ4íƾ4�A' ! pip install mrep 4S�\f�}�
9�
�§:Ú�
! word2vecP�ù8ƄŇ:¿Ó½ƎŹK:;7%P%A%-�
:�
¿Ó½Ǝ:ÚPň�3Ĉ0-!5�
;�
ƣƵ.5ă�JM3�LƄŇ:�+&)P�rWf}Į :�8�ƅ%ñDJML�:4;7���
�ÐŇǖĦ Generative Lexicon (GL)
<�
®�:£Ć�
! �ÐŇǖĦ81�3:ƴƄ7�ʼn
! ¿Ó½Ǝ5Êů:�Ð5:¨Ē�
=�
�18EA%3Œ:Ú�O�J7�,�
The Generative Lexicon [Pustejovsky95] �
! James Pustejovsky�Ādž ! ÄƸA4;ƟDA)Q4%-
! TimeML:Ādž�4EĐL ! �Å;� ¿Ó81�3:ÚP%I��ƙ03-:4<0�K%-�
>�
ÊÌ5¥%��
http://www.cs.brandeis.edu/~jamesp/�
��Ň:�7J�·¦é�:� [·¦05] �
! ·¦ǍƆ��ÐŇǖÊůĦ�
! ®ƟQ4L�Ǖƥ� ; _ ;�
?�
Æ8ħă8%-´ē�
! ·¦ǍƆ��ÐŇǖÊůĦ�[·¦05]:Û1�2ƽ
! James Pustejovsky, Introduction to Generative Lexicon. [Pustejovsky05]
! ě�Ʋë�Generative Lexicon:NjÇ� [ě�06] �
76�
�É��:Êů81�3ă�3BI��
! �č8�É��5å03E�Ĭî5%3:É�5�æŲ5%3:É���L
77�
ƛ�:��8�0-�
��P��8ƍ�-�
¥ƺ4��8�0-�
�gR�:Êů81�3ă�3BI��
! wj5%3:gR5���Ò'L�Ï5%3:gR:Êů��L
! é:�É��:ŗ5�&I�7!5�ĸ!03�7����
78�
�Ps�V4Ǒ0-�
��J�03«#�
! ă�3BL5ƄŇ;��1�:ƌ7LĎćPâ03�L!58Ñ2�
! ċOM�8I03Êů�ģO03�L ! !:Êů:ģµ8;ęª:ĨƏÎ:I�7E:�ĐL
! %�E�ƌ7LåŇ4Eǃ-Ół��L �
79�
�ÐŇǖĦ:ă���
! �1:Ňǖ8�ęª:�ļŋ�Pâ03�L ! đǓ8Ī&3�+:�ļŋ��J�Ʀ§ƌ7LÊů���Ð�#M3�L5ă�L�
7:�
��
Ĭî:É��
æŲ:É��
$�_�
ŅÊ��ŸƌĺŇ5êĺŇ;ĔĖ#ML�
7;�
! êĺŇ;Êů¢8ŒJ�:¨ĒPâ03�LÔ�çÇ
! -A-A�&Ÿ870-Ň;��ŸƌĺŇ5%3ĔĖ#ML�
Ù¦;7*!���ÚÝ8ŰůPâ03�L��
Ī¹��ƄŇGƣ�Ň~r}:�Êů�P�A�Ʊ�7�ËÝ��ŏê�5ă�3�L�
7<�
�%(�¿þ�ņ��6�&Êů�
�%(�¿þ��&.�6ņ�Êů�
Ňǖ:�ĝŃļŋ��WU|Rļŋ�5��E:Pă�L�ŇPń'LƠÎG�ł:ğ� 1. ļÐWU|R�Constitutive�
! ƋēG£Ć76�Œ4�Ò3�L�Pā'
2. řĚWU|R�Formal� ! +M�Œ7:���ì:NjōPā'
3. ²¢WU|R�Telic� ! ÁĊG²¢Pā'
4. ƱWU|R�Agentive� ! +MP�B�'¾ÿGÄƉPā'�
7=�
Œ��ģ+�. �
7>�
ŗ�ò:WU|Rļŋ�
7?�
1. ļÐ�`Sy�ǀ�k�g}�
2. řĚ�ŀKî�Ánj��
3. ²¢�ŀL�Ąĩ'L��
4. Ʊ�æB©3L�ÿL��
@1AKN�`KBC0 L4GM*!J&DIBQS[^T�
Œ4!�%-úß�ĠÇ7:���
! ò;�Ŋ¾'L�5��ƿǏ:Ƥø�!!4;²¢WU|R��ĐL
! +:-D��;«:đ:Êů.5��øDŽ�4�L
86�
ŔŌ:ò;ş��
ŔŌ:ò;\n�g�ş��
Ƈő��ƿǏ:Ƥø�PįŁ'L5Y�f87L��
! Ɯö;ä8�LE:�5��ƿǏ:ƤøPįŁ%3�L ! ƶ8��ä���rbg�8703�-J�ƭǎ:^|o;J�8À�Ū
�1��Ƨ�:åŇĭ¸5��:PGK-�9�
87�
�ƭǎ��ä4Dž3-Q.IƜö4� �lh�t�S����60/�ä4�Ɯö4��
�ƭǎ��.�J�ä:�8�LƜö:�.I ��\^[]U5@CLASSICAOP�
7Q4WU|RļŋPă�L:���
! ê�:ƄŇ:Êů8¨%3�é8�-I�7�ƿǏ:�ø��ūĢ'L
! +:-D��-�Eáơ#M-�:I�7¿Ó4Êů�ª0-K�+0�K7đ�Ė:Êů87L!5�ĐL
! !:��ƿǏ:�ø��J³ï:đ�4:Êů��Ð#ML:4;7�� �
88�
begin8;į×:Êů��Ð#ML�
! +E+Ebegin:�58;�ƒG�ł4;� ! begin�į×:ÊůPâ03�L5Ĉ�IK�đǓ8Ī&3Êů��Ð#ML5ă�L?���Ū�
89�
begin the meeting �
begin a dance �
begin the book �
begin the movie �
ę�Ð�co-composition��
1. begin:�N8;�ł�ÒL;(.��book;�ł4;7�,
2. bookP�ł5%3øDŽ%I��`SqÍÈ��3. WU|R�JúßPÂ03�L
�¶:�Ðq�^\P ��5Ž=
8:�
begin� the book ���
cf. ƿǏ:ŝģŶ�
1. double:�N8;double:Ş�ÒL;(.��10;double4;7�,
2. 10Pdouble:Ş5%3øDŽ%I��3. Vx\f:�¯:úßPÂ03�L
�¶:�Ðq�^\P13�*",5Ž=
8;�
double x = � 10���
ħă�³;�ÐŇǖĦ8;ĴK31:ļŋ��L �
ĝŃļŋ;�ÐŇǖĦ:41ļŋ:�:11
! Ƴļŋ ! �łļŋ
! ĝŃļŋ ! Ňǖżƕļŋ
®�;ėŷ%A'I�
8<�
!!A4:A5D�
! ĥŇǖ8;�+:ċOM�8¨'LúßPâ03�L�WU|Rļŋ�
! ŹŴ:ƄŇ5śćªK:øDŽ�4�7��8�!:WU|R:úßPċ03¥%�ÊůP�Ð'L�ę�Ð��
8=�
�ŪåŇĭ¸:�4�ÐŇǖĦ;6M�J�ċOM3�-��! åŇĭ¸½�:«430÷ ! ACL:«4300÷�
8>�
ÊÌ5Ĥ7���
/H05!M;�ģ �
! 11:Ňǖ4!M�J�:úß�
8?�
����5� �. $�.�/L�HE�,@A M5BAM��+3�
4�5�. $�.�#/R%BFSY]W�2M($�
WU|Rļŋ;�ĨũdV\f�J�¾Ʈõ%-��
! Web:íƾóČ�J���4ÿÐ%-m`��Pܹ%3ǐ�'L�¯ [Cimiano+07]
! ęĸǔ°5ðǒG¾ǒ:ƠÎPĝƯ8%3�{�V�X½ƎPƂ¹'L�¯ [ĻŖ+12] �
96�
5;��ċ�:EƴƄ4;7#+��
Ĥ%Ɩ��J�7�%3BL�
! ƄŇ�6:I�7ƃ:ƄŇ5�Ƽ8ċOML�5��!58�'LüŭP±ųµ%3�L
! 6:I�7ƄŇ5ċOML�8I03�ƄŇ:ÊůEģOL
97�
!:Ú6!�4�
�§: Statistical Semantics:Ú8ǃ3�L�
98�
!!4�°��ÐŇǖĦ�JŮM3� Êů:�Ð:ÚP%A'�
99�
Statistical Semantics:�#J��
! ŹŴƄŇ:�Óǔ°76:IJè¢7ÎŃ�J�+:Ň:ÊůPă�L
! �ƚ�øGhz�{}ibf76:�¯�¹�JML�
9:�
Statistical Semantics is the study of "how the statistical patterns of human word usage can be used to figure out what people mean, at least to a level sufficient for information access”�(ACL wiki��)�
ƄŇ:rWf}¿Ó�
! �§:Ú;��ƄŚ:ƄŇ�:ÊůP¿Ó%-
! ®§;ŹŴƄŇ5:¨Ē4Êů�ģOLÚ7:4�/��(�J7Lo~�]:Êů�6�7L:��
9;�
�ÐŇǖĦ5Statistical Semantics:¨Ē�
! �ÐŇǖĦ ! ĝ¡:đǓ8��LƄŇ:�Ó:ÊůPă�3�L ! 5�+:I�7ģµPĸ!'-D:Ňǖ:ļŋPă�3�L�
! Statistical Semantics ! ŹŴƄŇ:�Ɯ8I03��Ɣ¢7+:ƄŇ:ÊůPă�3�L
! ƀ±¢7ƄŇ:�Ó8�'LÊů8åſ%37��®�J'LI�
9<�
Principle of compositionality (Frege1892] �
… the meaning of a complex expression is determined by the meanings of its constituent expressions and the rules used to combine them (wikipedia� ) �����������������������������������
9=�
·�#Q:\{Sg�A5A03�L�
9>�
https://speakerdeck.com/mamoruk/a-tensor-based-factorization-model-of-semantic-compositionality�
¿ÓrWf}:�Ð;ĵ%ĕIKƻ�ĕ�ķ� [Mitchell+08] �
! ĵ%ĕGƻ�ĕ76�rWf}:�Ð�¯Pılj ! ÇƑ"5:ƻ�ĕ�Multiplicative��¬0/F0-�
9?�
Statistical Semantics5:¨Ē�
! Statistical Semantics��§:£Ć�;6/J�5å�5�#-(:rWf}PƮõ'LŎŠ
! Compositionality:ŎŠ;�+:rWf}:����:ŎŠ
Ğ�P��8ø��ØE�L%�¿Ó½Ǝ:�Ó4ƁK��KP�)3�L�
:6�
Êů:�ÐP�ƚ4¿Ó'L (MV-RNN) [Socher+12] �
! ĥƄŇ8;rWf}5�ƚ��Ī'L ! ŵƹ#ML5��ƪ�:�ƚP¼�:rWf}8ƻ�3�#J8žĶř7ģŶ f PŦż¢8��
! RootA4ƗKľ'5đ:¿ÓrWf}�õJML :7�
�G�Gd�_}. [Cruys+13] �
! ÆŇ�¾ǒ�²¢Ň:31P�d�_}Pċ03æ�)3�L:.5ă�L�
:8�
Dynamic Convolutional Neural Network [Kalchbrenner+14] �
! CNNPċ03ƄŇ:A5AKPæB� 3�� ! ļđÞ4;7���Żǃ:ŜČ�õJML��
:9�
Ƈő�CNN5compositionality�
! Ŀ�~Sy�4;ŧD3ijÏ¢�ƄƷ7úß ! �Ă:~Sy�87L8¶M3+MJ�æB�O#K�IKƣƵ7Êů:�LúßPûŭ%3�L�
::�
Layer1�Layer3�
Layer5�
[Zelier+14]OP�'�
åŇĭ¸;ĸÔ�ņ��
:;�
��� 4��
nW^}� Tb[� ¤�� ǂ�
źř� ŸƑ� ƄŇ� �Ú�
æB� L��ø'L�
¿Ó�� ƄŇ� o~�]� đ�
��M-� ��M-�
!:œŐ8�ÐŇǖĦ:ü�Pď�)7�� �
! �ÐŇǖĦ;CompositionalityP��4űƘ%3±ų2�L-D8Ľ¸#M-¸Ħ5�M7���
! šƕ½Ǝ76:m{aSvZof�ĸ�+�7®!+�ĜŨ:ü�GŹŴ�¦P�ą'>�
:<�
Ãà:Ú�
:=�
ƣƵ.5ă�JM3�LƄŇ:�+&)P�rWf}Į :�8�ƅ%ñDJML�:4;7���
ę�ÐP¿ÓrWf}85K!C [Tsubaki+13] �
! �ƪ���ƪ�8ŕŢPſ@%3¥%�rWf}PÿL ! 4�-rWf}�JÊů:�Ð��OML�
:>�
run companyM��
�Mrun�
companyM �JMrun�
company LOQ���
runM �JMcompany�
�Mcompany�
run LOQ���
run companyM)"�
'48 �
ƄŇ:�Ð8�'L��1�:ŁÔ�
! �ÐŇǖĦ ! åŇÓłPűƘ%3��A�we}µ%I�5'LīB
! Ť�:׸we} ! űƘ#M-åŇe�`P���ń'LI�7we}:ŬƘ
! ¿Ó½Ǝų�šƕ½Ǝų� ! ļÐ5¿Ó:½ƎP��8���5%3�L
!MJ:RSeR;ƞţ8¨Ē%3�L�:?�
A5D�
! �ÐŇǖĦ5;� ! Ňǖ;8�'LÀƊº7üŭ�J�đǓ8Ī&3Êů��Ð#ML5ă�L
! Ň:Êů:�Ð8¨'LŎŠ ! ƄŇ:¿ÓrWf}P�Ð%3�ƣ�¿Ó:ÊůPÿL
! ¿Ó½Ǝp�v8ŀ03�®İ�03�L
åŇ½:ü�5�åŇĭ¸:¥%��¯:ã��E05ÕC� �
;6�
"ƓƢ�K�5�"$�A%-�
;7�
ħăđƫ (1/3) �
! [Pustejovsky95] James Pustejovsky. The Generative Lexicon. MIT Press.
! [·¦05] ·¦ǍƆ. ��(4�+$. �N%��ư
! [Pustejovsky05] James Pustejovsky. Introduction to Generative Lexicon. Foundations of
SemanticsƩĺ´ē. http://www.cs.brandeis.edu/~jamesp/classes/LING130/
! [ě�06] ě�Ʋë.
Generative Lexicon�2�. Ɛ:ğ�NJÍ�:ŏÌƈ
;8�
ħăđƫ�(2/3) �
! [Cimiano+07] Philipp Cimiano, Johanna Wenderoth. Automatic Acquisition of Ranked Qualia Structures from the Web. ACL2007
! [ĻŖ+12] ĻŖ�ƨ, ·�ť, ě�Ʋë. ��(4$���� �(�!'&)������.�����0�. NLP2012.�
! [Mitchell+08] Jeff Mitchell, Mirella Lapata. Vector-based Models of Semantic Composition. ACL2008.
! [Socher+12] Richard Socher, Brody Huval, Christopher D. Manning, Andrew Y. Ng. Semantic Compositionality through Recursive Matrix-Vector Spaces. EMNLP2012.
;9�
ħăđƫ�(3/3) �
! [Cruys+13] Tim Van de Cruys, Thierry Poibeau, Anna Korhonen. A Tensor-based Factorization Model of Semantic Compositionality. NAACL2013.
! [Kalchbrenner+14] Nal Kalchbrenner, Edward Grefenstette, Phil Blunsom. A Convolutional Neural Network for Modelling Sentences. ACL2014.
! [Zelier+14] Matthew D. Zeiler, Rob Fergus. Visualizing and Understanding Convolutional Networks. ECCV2014.
! [Tsubaki+13] Masashi Tsubaki, Kevin Duh, Masashi Shimbo, Yuji Matsumoto. Modeling and Learning Semantic Co-Compositionality through Prototype Projections and Neural Networks. EMNLP2013
;:�