japanese sentiment classiﬁcation using a tree-structured ... · the-art performance for a...

Japanese Sentiment Classificationusing a Tree-Structured Long Short-Term Memory with Attention

Ryosuke Miyazaki∗Graduate School of System Design

Tokyo Metropolitan [email protected]

Mamoru KomachiGraduate School of System Design

Tokyo Metropolitan [email protected]

Abstract

Previous approaches to training syntax-based sentiment classification models requiredphrase-level annotated corpora, which are notreadily available in many languages other thanEnglish. Thus, we propose the use of tree-structured Long Short-Term Memory with anattention mechanism that pays attention toeach subtree of a parse tree. Experimental re-sults indicate that our model achieves state-of-the-art performance for a Japanese sentimentclassification task.

1 Introduction

Traditional approaches for sentiment classificationrely on simple lexical features, such as a bag-of-words, that are ineffective for many sentiment clas-sification tasks (Pang et al., 2002). For example, thesentence “Insecticides kill pests.” contains both killand pests, indicating negative polarity. But, the over-all expression is still deemed positive.

To address this problem of polarity shift, Nak-agawa et al. (2010) presented a dependency-tree-based approach for the sentiment classification of asentence. Their method assigns sentiment polarity toeach subtree as a hidden variable that is not observ-able in the training data. The polarity of the overallsentence is then classified by a tree-conditional ran-dom field (Tree-CRF), marginalizing over the hid-den variables representing the polarities of the re-spective subtrees. In this manner, the model canhandle polarity-shifting operations such as negation.However, this method suffers from feature sparse-ness because almost all features are combinationfeatures.

∗Now at Yahoo Japan Corporation.

To overcome the data sparseness problem, deep-neural-network-based methods have attracted muchattention because of their ability to use dense fea-ture representations (Socher et al., 2011; Socher etal., 2013; Kim, 2014; Kalchbrenner et al., 2014; Taiet al., 2015; Zhang and Komachi, 2015). In particu-lar, tree-structured approaches called recursive neu-ral networks (RvNNs) have been shown to performwell in sentiment classification tasks (Socher et al.,2011; Socher et al., 2013; Kim, 2014; Tai et al.,2015). Whereas Tree-CRF employs sparse and bi-nary feature representations, RvNNs avoid featuresparseness by learning dense and continuous fea-ture representations. However, annotation for eachphrase is crucial for learning RvNN models, butthere is no phrase-level annotated corpus in any lan-guage other than English.

We therefore propose an RvNN model with anattention mechanism and augment the training ex-ample with polar dictionaries to compensate for thelack of phrase-level annotation. Although Kokkinosand Potamianos (2017) also provided an attentionmechanism for phrase-level annotated corpus, ourmodel performs well on a sentence-level annotatedcorpus through the introduction of polar dictionar-ies.

The main contributions of this work are as fol-lows.

• We show that RvNN models can be learnedfrom a sentence-level polarity-tagged Japanesecorpus using an attention mechanism and polardictionaries.

• We achieve the state-of-the-art performance ina Japanese sentiment classification task.

PACLIC 32

466 32nd Pacific Asia Conference on Language, Information and Computation

Hong Kong, 1-3 December 2018 Copyright 2018 by the authors

• We have released our code on GitHub.1

The rest of this paper is organized as follows. Sec-tion 2 introduces related work. Section 3 describesour proposed method using Tree-LSTM with an at-tention mechanism and polar dictionaries. Section 4presents the experimental results from Japanese andEnglish sentiment datasets, and Section 5 discussesthe advantages and disadvantages of the proposedmethod. Finally, Section 6 concludes our work.

2 Related Work

This section describes related work on Japanese sen-timent classification, RvNNs, and attentional mod-els.

2.1 Japanese Sentiment Analysis

Nakagawa et al. (2010) proposed a dependency-based polarity classifier. Their model infers polarityfrom the composed nodes of a dependency tree usingTree-CRF. Each subtree is represented as a hiddenvariable in consideration of interactions between thehidden variables. A polar dictionary is used as theinitial variable, and a polarity-reversing word dic-tionary is used to capture whether the constructedphrase polarity is reversed or not. Our model usesonly a polar dictionary and attempts to learn polar-ity shifting via RvNNs.

Zhang and Komachi (2015) adopted a stacked de-noising auto-encoder. Their model treats an inputsentence as an average vector of their word vec-tors, which is then fed into a stacked denoising auto-encoder. Although this model omits syntactic in-formation, it achieves high performance through itsgeneralization ability. However, it is not straightfor-ward to employ polar dictionaries in their model.

2.2 Recursive Neural Networks

There are various RvNN models for sentiment clas-sification (Socher et al., 2011; Socher et al., 2012;Socher et al., 2013; Qian et al., 2015; Tai et al.,2015; Zhu and Sobhani, 2015). All of these mod-els attempt to capture sentence representation in abottom-up fashion in accordance with a parse tree.

1See https://github.com/tmu-nlp/AttnTreeLSTM4SentimentClassification

In this way, sentence representation can be calcu-lated by learning compositional functions for eachphrase.

Several studies have focused on using a composi-tional function to improve compositionality (Socheret al., 2012; Socher et al., 2013; Qian et al., 2015;Tai et al., 2015). Socher et al. (2012) parameter-ized each word as a matrix–vector combination todenote modification and representation. Socher etal. (2013) used a bilinear function enacted by tensorslicing for composition in place of a large matrix–vector parameterization. Qian et al. (2015) incor-porated a constituent label of each phrase as featureembedding to take account of the different composi-tionality based on parent or children label combina-tions. Tai et al. (2015) proposed a more robust modelthan those discussed above in which long short-termmemory (LSTM) units were applied to a RvNN, asmentioned in Section 3.1. Thanks to the applicationof LSTM, this model can learn increased numbers ofparameters appropriately, unlike the other models.

2.3 Attentional ModelsBased on psychological studies, the human abilityof intuition of attention (Rensink, 2000) has beenintroduced into many computer science fields. Themain function of this ability is deciding which partof an input needs to be focused on.

In natural language processing (NLP), the atten-tion mechanism is utilized for many tasks, includingneural machine translation (Bahdanau et al., 2015;Luong et al., 2015; Eriguchi et al., 2016), neuralsummarization (Hermann et al., 2015; Rush et al.,2015; Vinyals et al., 2015), representation learning(Ling et al., 2015), and image captioning (Xu et al.,2015).

These approaches incorporate consideration as tohow each source word or region contributes to thegeneration of a target word. Unlike most of theabove-mentioned NLP tasks for generating word se-quences, our model derives attention informationfor sentiment classification. Eriguchi et al. (2016)proposed an attentional model focusing on phrasestructure; our model omits the word-level recur-rent LSTM layer from their model. Yang et al.(2016) presented a hierarchical attention networkfor document classification, incorporating sentence-and word-level attention mechanisms. Our task ad-

PACLIC 32



dresses sentence-level sentiment classification, sothat our model employs a phrase-level (constituent)attention network rather than sentence-level infor-mation in a document.

Probably the most related work to our study is(Kokkinos and Potamianos, 2017) and (Zou et al.,2018). Kokkinos and Potamianos (2017) built a sim-ilar model to ours, but they did not use the hiddenstate of a RvNN in their final softmax, and they alsodid not emphasize the use of polar dictionaries. Zouet al. (2018) proposed a lexicon-based supervised at-tention model to take advantage of a sentiment lex-icon. They injected type-level lexical informationinto an additional attention network, whereas we in-jected token-level lexical information into a singleRvNN model as a phrase-level annotation.

3 Attentional Tree-LSTM

3.1 Tree-Structured LSTMVarious RvNN models for handling sentence repre-sentation considering syntactic structure have beenstudied (Socher et al., 2011; Socher et al., 2012;Socher et al., 2013; Qian et al., 2015; Tai et al.,2015; Zhu and Sobhani, 2015). RvNNs constructa sentence representation from their phrase rep-resentations by applying a composition function.Phrase representations can be calculated by recur-sively adopting composition functions. Binariza-tions of parse trees are often used to simplify thecomposition function. In a parse tree, the rootnode, non-terminal node, and terminal node repre-sent sentence, phrase, and word representations, re-spectively.

The ith non-terminal node representation hi iscalculated by using the composition function g as

hi = f(g(hli, hri )), (1)

g(hli, hri ) = W

[hlihri

]+ b, (2)

where the matrix W ∈ Rd×2d and the bias b ∈ Rdare the parameters to be learned, hli, h

ri ∈ Rd are

d-dimensional children vectors of node hi, and theresulting vector hi is another d-dimensional vector.The hyperbolic tangent is usually employed as theactivation function f . These RvNN models are es-sentially identical to recurrent neural models in thatthey are not able to retain a long history.

Tai et al. (2015) addressed this problem by intro-ducing LSTM (Hochreiter and Schmidhuber, 1997)to make RvNN less prone to the exploding/vanishinggradient problem. In this paper, we use the BinaryTree-LSTM proposed by Tai et al. (2015) as an ex-ample of a tree-structured LSTM. The Binary Tree-LSTM composes children vectors using the follow-ing equations:

ij = σ

(U (i)

[hljhrj

]+ b(i)

), (3)

fjl = σ(U (fl)hrj + b(fl)

), (4)

fjr = σ(U (fr)hlj + b(fr)

), (5)

oj = σ

(U (o)

[hljhrj

]+ b(o)

), (6)

uj = tanh

(U (u)

[hljhrj

]+ b(u)

), (7)

cj = ij � uj + fjl � cjl + fjr � cjr, (8)

hj = oj � tanh(cj), (9)

where the matrices U ∈ Rd×2d (except for Ufl andUfr, for which U ∈ Rd×d) and the biases b ∈ Rd

are the parameters to learn. The memory state c iscontrolled by i, f , and o (called the input gate, forgetgate, and output gate, respectively), to hold impor-tant information for the entire network. Each gateselectively activates to play a specific role (i.e., theinput, forget, and output gates control uj , cjl, andcjr, respectively, and tanh(cj) is based on which el-ements should be input to the next state cj , forgottenfrom the previous state cjr and cjl, or output as ahidden representation hj , respectively).

Note that the forget gate fjl for the left child statecjl only takes the right child’s hidden representationhrj and vice versa, as described by Tai et al. (2015).

3.2 Softmax Classifier with Attention

We use a softmax classifier to predict the sentimentlabel yj at any node j for which a label is to be pre-dicted. Given the jth hidden representation hj as aninput, the classifier predicts yj :

yj = arg maxy

pθ(y|hj), (10)

pθ(y|hj) = softmax(W (s)hj + b(s)

), (11)

PACLIC 32



where W (s) ∈ Rdl×d and b(s) ∈ Rn are the param-eter matrix and bias vector for the classifier, respec-tively, and dl is the number of labels. The softmaxyields a label distribution y ∈ Rdl , following whichthe classifier chooses the best label corresponding tothe highest element among the y.

However, owing to the lack of phrase-level an-notation, sentence representation may be inaccuratebecause it may fail to propagate errors from the rootof the tree to the terminals and pre-terminals in along sentence. We propose an attention mechanismto address this problem. This so-called classifierwith attention takes an attention vector representa-tion aj in addition to a hidden representation hj asinputs:

pθ(y|hj) = softmax

(W (s′)

[ajhj

]+ b(a)

), (12)

aj =∑i

aji � hi, (13)

aji =g(hi, hj)∑i′ g(hi′ , hj)

, (14)

g(hi, hj) = exp

(W (a2) tanh

(W (a1)

[hihj

])),

(15)

where W (s′) ∈ Rdl×2d,W (a1) ∈ Rd

a×d, andW (a2) ∈ R1×da are the parameter matrices. InEq. 15, the biases for both W (a1) and W (a2) areomitted for simplicity. The attention vector aj rep-resents how much the classifier pays attention to thechildren nodes of the target node. The scalar valuesaji for each node are used to determine the attentionvector.

Figure 1 represents the softmax classifier with at-tention. Kokkinos and Potamianos (2017) also in-vestigated an attentional model for RvNNs. But,their model only feeds the attention vector into thesoftmax classifier, whereas our method inputs boththe attention vector and RvNN vector, as illustratedin Figure 1.

3.3 Distant Supervision with Polar DictionariesUnlike the Stanford Sentiment Treebank, which isannotated with phrase-level polarity, other multilin-gual datasets contain only sentence-level annotation.As shown in Section 4, sentiment classification with-out a phrase-level annotated corpus will not learn

sentence representations in an appropriate manner.Although a phrase-level-polarity-tagged corpus isdifficult to obtain in many languages, polar dic-tionaries are easy to compile (semi-)automatically.Therefore, we opt for the use of polar dictionaries asan alternative source of sentiment information.

We utilize the same polar dictionaries for shortphrases and words as used in Nakagawa et al.(2010). The phrase in the training sets that matchesan entry in the polar dictionaries is annotated withthe corresponding polarity. The key difference fromNakagawa et al. (2010) is that we use polar dictio-naries as a hard label in a manner similar to distantsupervision (Mintz et al., 2009). In contrast, the pre-vious work used polar dictionaries as a soft labelfor an initial hidden variable in Tree-CRF. Teng etal. (2016) also incorporated sentiment lexicons intoan recurrent neural network model. Their methodpredicts weights for each sentiment score of subjec-tive words to predict a sentence label. Our methoduses polar dictionaries only during the training step,whereas the method by Teng et al. (2016) needs po-lar dictionaries for both training and decoding.

3.4 Learning

The cost function is a cross-entropy error functionbetween the true class label distribution, t (i.e., onehot distribution for the correct label) and the pre-dicted label distribution, y, at each labeled node:

J(θ) = −m∑k=1

tk log yk +λ

2‖θ‖22, (16)

where m is the number of labeled nodes in the train-ing set2, and λ denotes an L2 regularization hyper-parameter.

4 Experiments on Sentiment Classification

We conducted sentiment classification on a Japanesecorpus where phrase-level annotation is unavailable.In addition, we performed sentiment classificationon an English corpus where phrase-level annotationis available, but without using phrase-level annota-tion, to see its effect.

2If the dataset contains only sentence-level annotation, m isequal to the size of the dataset.

PACLIC 32



(a) RvNN (b) RvNN with Attention

Figure 1: Sentiment classification by Tree-LSTM with attention.

4.1 DataWord embeddings. For Japanese experiments,we obtained pre-trained word representations fromword2vec3 using a skip-gram model (Mikolov etal., 2013a; Mikolov et al., 2013b; Mikolov etal., 2013c). We learned word representations onJapanese Wikipedia’s dump data (2014.11) seg-mented by KyTea (version-0.4.7) (Neubig et al.,2011). We used pre-trained GloVe word represen-tations4 for the English experiments. We fine-tunedboth word representations in our experiments.

Parse trees. For Japanese constituency parsing,we used Ckylark (Oda et al., 2015) as of 2016.075

with KyTea for word segmentation. For English, weused the automatic syntactic annotation of the Stan-ford Sentiment Treebank.

Dictionaries. We followed Nakagawa et al. (2010)to create polar dictionaries. We employed aJapanese polar dictionary composed by Kobayashiet al. (2005) and Higashiyama et al. (2008)6 thatcontains 5,447 positive and 8,117 negative expres-sions.7 We created an English polar lexicon from

3See https://code.google.com/archive/p/word2vec/

4See http://nlp.stanford.edu/data/glove.6B.zip

5See https://github.com/odashi/ckylark6See http://www.cl.ecei.tohoku.

ac.jp/index.php?OpenResources/JapaneseSentimentPolarityDictionary

7Note that these figures are slightly different from Naka-gawa et al. (2010). We suspect that the reason why they can

Wilson et al. (2005) in the same way as Nakagawaet al. (2010). The dictionary contains 2,289 positiveand 4,143 negative expressions.

Corpora. We used the NTCIR Japanese opinioncorpus (NTCIR-J), which includes 997 positive and2,400 negative sentences (Seki et al., 2007; Seki etal., 2008). We removed neutral sentences followingprevious studies. The corpus comprised two NT-CIR Japanese opinion corpora, the NTCIR-6 cor-pus and the NTCIR-7 corpus, as in (Nakagawa etal., 2010). We performed 10-fold cross-validationby randomly splitting each corpus into 10 parts (onefor testing, one for development, and the remainingeight for training).8 For the English experiments,we used the Stanford Sentiment Treebank (Socheret al., 2013). It includes 11,855 sentences. We fol-lowed the official training/development/testing split(8,544/1,101/2,210). We used only sentence-levelsentiment for our experiment.

4.2 Methods

In the Japanese experiments, we compared ourmethod with seven baselines. In the English experi-ments, we compared our method with two baselines.All input word vectors, other than those for mostfrequent sentiment (MFS) and Tree-CRF, were pre-

use a larger lexicon is that they used an in-house (not publiclyavailable) version of the lexicon.

8Nakagawa et al. (2010) did not use development data totrain the model, which means our model uses only 86.6% of theinstances to train the model compared with theirs.

PACLIC 32



trained by word2vec in Japanese experiments and byGloVe in English experiments. We implemented ourmethod, LogRes, RvNN, Tree-LSTM, and our reim-plementation of Kokkinos and Potamianos (2017)using Chainer (Tokui et al., 2015).

The following methods were used.

MFS. A naıve baseline, as it always selects themost frequent sentiment (which is negative in thiscase).

LogRes. A linear classifier using logistic regres-sion. The input features are an average of word vec-tors in a sentence.

CNN The CNN-based sentiment classification(Kim, 2014).9

Tree-CRF. A dependency-based tree-structuredCRF (Nakagawa et al., 2010). This is the state-of-the-art method among our experimental datasets.10

RvNN. The simplest RvNN.

Tree-LSTM. The LSTM-based RvNN (Tai et al.,2015).

Kokkinos and Potamianos (2017) Our reimple-mentation of Kokkinos and Potamianos (2017). Weimplemented their TreeGRU model with LSTM in-stead of GRU.

Tree-LSTM w/ attn, dict. Our proposed method,which classifies polarity using attention and/or polardictionaries.

4.3 HyperparametersThe parameters used in both experiments are listedin Table 1. For Japanese experiments, we tuned hy-perparameters on each development set of 10-foldcross-validation. For English experiments, we usedsimilar hyperparameters with slight modifications tothe Japanese experiments.

4.4 ResultsThe Japanese experimental results are listed in Table2. The accuracy of RvNN is much lower than that of

9See https://github.com/yoonkim/CNN_sentence

10Note that we cannot directly compare our result with astacked denoising auto-encoder (Zhang and Komachi, 2015)(which achieved slightly higher accuracy in the NTCIR-6 cor-pus,) because we use a different dataset in our experiment.

Parameter ValueJapanese English

Word vector size 200 300Hidden vector size 200 200Optimizer AdaDelta AdaGrad(Weight decay/learning rate) 0.0001 0.005Gradient clipping 5 5

Table 1: The hyperparameters.

Method Accuracy

MFS 0.704LogRes 0.771CNN (Kim, 2014) 0.803Tree-CRF (Nakagawa et al., 2010) 0.826

RvNN 0.517Reimplementation of Tai et al. (2015) 0.709Reimplementation of K&P (2017) 0.807

Tree-LSTM w/ attn 0.810Tree-LSTM w/ dict 0.829Tree-LSTM w/ attn, dict 0.844

Table 2: Accuracy of each method on the Japanesesentiment classification task.

the MFS baseline. Moreover, Tree-LSTM, which isan improved RvNN, is still lower than simple Lo-gRes, despite Tree-LSTM achieving state-of-the-artperformance on the phrase-annotated Stanford Sen-timent Treebank (Tai et al., 2015). The accuracy ofCNN and Kokkinos and Potamianos (2017) is 0.803and 0.807, respectively, which are slightly lowerthan that of our proposed method without polar dic-tionaries. In contrast, Tree-LSTM with dictionaryachieves comparable results to Tree-CRF. Our Tree-LSTM with attention and polar dictionary obtainedthe highest accuracy.

The results of the English experiments are listedin Table 3. Similarly to Kokkinos and Potamianos(2017), we observe slight improvement when us-ing attention for Tree-LSTM. However, unlike theJapanese experiments, using a dictionary degradessentiment classification accuracy. We discuss this inthe following section.

PACLIC 32



Method Accuracy

(Kim, 2014)— CNN 48.0(Tai et al., 2015)— Tree-LSTM 51.0(Kokkinos and Potamianos, 2017)— TreeGRU w/o attn 50.5— TreeGRU w/ attn 51.0

Tree-LSTM 43.52Tree-LSTM w/ attn 44.97Tree-LSTM w/ dict 43.13Tree-LSTM w/ attn, dict 41.67

Table 3: Accuracy of each method on the Englishsentiment classification task. Note that our modellearns only from sentence-level annotation.

5 Discussion

5.1 Effect of Attention and Dictionary

The results described above indicate that Tree-LSTM models without an attention mechanism failto learn sentence representations if phrase-level an-notation is not available.

However, Tree-LSTM models can learn more ac-curate sentence representations if the models receivephrase-level information such as that provided bypolar dictionaries. For example, in our model, atten-tion information and a polar dictionary are fed intothe Tree-LSTM as phrase-level information. Al-though Tree-LSTM with attention and a polar dic-tionary outperforms Tree-CRF by 1.8 points, the ac-curacy of CNN, Tree-LSTM without a polar dic-tionary, and Kokkinos and Potamianos (2017) arelower than that of Tree-CRF. Tree-LSTM with apolar dictionary performs better than Tree-LSTMwith attention, showing that a supervised label foreach phrase seems to be important in learning Tree-LSTM models.

Table 3 indicates that although an attention mech-anism is effective in both sentence-level and phrase-level annotated corpora, the effect of dictionary in-formation varies across datasets. It is known that theeffect of the English lexicon (Wilson et al., 2005) ismilder in the review dataset than in other domains(Nakagawa et al., 2010), and we suppose that it in-

troduces noise in the Stanford Sentiment Treebank.

5.2 Examples

Figures 2a and 2b show correctly classified exam-ples. Figure 2a shows that the model classifies “一貫性 (consistency)” as positive and pays 1/3 atten-tion to it in the final classification step; however,the model correctly classifies the sentence polarityas negative by considering “見られない (cannot befound)” through most of the attention. In Figure 2b,the model correctly classifies both “友好 (friend-ship)” and “憂慮 (apprehension),” and then classi-fies the sentence polarity by paying great attentionto “憂慮 (apprehension).”

Figures 2c and 2d display an incorrectly classi-fied example. In Figure 2c, the model pays atten-tion to both “対立 (confrontation)” and “緩和 (mit-igated)”; however, it fails to predict the correct po-larity of the sentence. It seems that the higher at-tention weight for “対立 (confrontation)” than for“緩和 (mitigated)” has an influence on the sentenceprediction. In Figure 2d, the model fails to capturenegation as it should pay attention to “回避できた(was able to avoid).” To solve these errors, the com-position function should also incorporate an atten-tion mechanism to handle polarity shifting correctly.

6 Conclusion

We have presented a Tree-LSTM-based RvNN us-ing an attention mechanism and a polar dictionary.In this method, each phrase representation is fed intoa classifier to predict the polarity of a phrase basedon the phrase structures. Lexical items from the po-lar dictionary are used as supervised labels for eachcorresponding phrase or word in the same manner asdistant supervision. Our experimental results havedemonstrated that the proposed method outperformsthe previous methods on a Japanese sentiment clas-sification task.

Currently, we have annotated fine-grained phrase-level sentiment tags to a Japanese review corpus.We plan to analyze the effect of phrase-level anno-tation on Japanese sentiment analysis. In addition,we would like to extend our attentional mechanismto use the Transformer (Vaswani et al., 2017) as pro-posed in Shen et al. (2018).

PACLIC 32



(a) Consistency of policy cannot be found. (b) Apprehension about friendship.

(c) Military confrontation was mitigated. (d) Anyway, it was able to avoid the worst-case scenario.

Figure 2: Examples of our attentional Tree-LSTM sentiment classification on the test set. The red squareindicates a word or phrase to which great attention was paid in the softmax step, and the associated valueindicates the attention weight. Root nodes are indicated by the (left) gold and (right) predicted labels (“N”indicates negative, whereas “P” indicates positive). We also show the labels for nodes that match an entryin the polar dictionary.

References

Dzmitry Bahdanau, KyungHyun Cho, and Yoshua Ben-gio. 2015. Neural machine translation by jointlylearning to align and translate. In Proceedings of the3rd International Conference on Learning Represen-tations.

Akiko Eriguchi, Kazuma Hashimoto, and YoshimasaTsuruoka. 2016. Tree-to-sequence attentional neuralmachine translation. In Proceedings of the 54th An-nual Meeting of the Association for Natural LanguageProcessing, page 823–833.

Karl Moritz Hermann, Tomas Kocisky, Edward Grefen-stette, Lasse Espeholt, Will Kay, Mustafa Suleyman,and Phil Blunsom. 2015. Teaching machines to readand comprehend. In Advances in Neural InformationProcessing Systems 28, pages 1693–1701.

Masahiko Higashiyama, Kentaro Inui, and Yuji Mat-

sumoto. 2008. Learning sentiment of nouns from se-lectional preferences of verbs and adjectives. In Pro-ceedings of the 14th Annual Meeting of the Associationfor Natural Language Processing, pages 584–587.

Sepp Hochreiter and Jurgen Schmidhuber. 1997. Longshort-term memory. Neural Computation, 9(8):1735–1780.

Nal Kalchbrenner, Edward Grefenstette, and Phil Blun-som. 2014. A convolutional neural network for mod-elling sentences. In Proceedings of the 52nd AnnualMeeting of the Association for Natural Language Pro-cessing, pages 655–665.

Yoon Kim. 2014. Convolutional neural networks for sen-tence classification. In Proceedings of the 2014 Con-ference on Empirical Methods in Natural LanguageProcessing, pages 1746–1751.

Nozomi Kobayashi, Kentaro Inui, Yuji Matsumoto, andKenji Tateishi. 2005. Collecting evaluative expres-

PACLIC 32



sions for opinion extraction. Journal of Natural Lan-guage Processing, 12(3):203–222.

Filippos Kokkinos and Alexandros Potamianos. 2017.Structural attention neural networks for improved sen-timent analysis. In Proceedings of the 15th Confer-ence of the European Chapter of the Association forComputational Linguistics, pages 586–591.

Wang Ling, Lin Chu-Cheng, Yulia Tsvetkov, Silvio Amir,Ramon Fernandez Astudillo, Chris Dyer, Alan WBlack, and Isabel Trancoso. 2015. Not all contexts arecreated equal: Better word representations with vari-able attention. In Proceedings of the 2015 Conferenceon Empirical Methods in Natural Language Process-ing, pages 1367–1372.

Minh-Thang Luong, Hieu Pham, and Christopher D.Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings ofthe 2015 Conference on Empirical Methods in NaturalLanguage Processing, pages 1412–1421.

Tomas Mikolov, Kai Chen, Greg Corrado, and JeffreyDean. 2013a. Efficient estimation of word representa-tions in vector space. In Proceedings of the 1st Inter-national Conference on Learning Representations.

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado,and Jeff Dean. 2013b. Distributed representations ofwords and phrases and their compositionality. In Ad-vances in Neural Information Processing Systems 26,pages 3111–3119.

Tomas Mikolov, Wen tau Yih, and Geoffrey Zweig.2013c. Linguistic regularities in continuous spaceword representations. In Proceedings of Human Lan-guage Technologies: The 2013 Annual Conference ofthe North American Chapter of the Association forComputational Linguistics, pages 746–751.

Mike Mintz, Steven Bills, Rion Snow, and Dan jurafsky.2009. Distant supervision for relation extraction with-out labeled data. In Proceedings of the Joint Confer-ence of the 47th Annual Meeting of the ACL and the 4thInternational Joint Conference on Natural LanguageProcessing of the AFNLP, pages 1003–1011.

Tetsuji Nakagawa, Kentaro Inui, and Sadao Kurohashi.2010. Dependency tree-based sentiment classificationusing CRFs with hidden variables. In Proceedingsof Human Language Technologies: The 2010 AnnualConference of the North American Chapter of the As-sociation for Computational Linguistics, pages 786–794.

Graham Neubig, Yosuke Nakata, and Shinsuke Mori.2011. Pointwise prediction for robust, adaptableJapanese morphological analysis. In Proceedings ofThe 49th Annual Meeting of the Association for Com-putational Linguistics: Human Language Technolo-gies, pages 529–533.

Yusuke Oda, Graham Neubig, Sakriani Sakti, TomokiToda, and Satoshi Nakamura. 2015. Ckylark: A morerobust PCFG-LA parser. In Proceedings of the 2015Conference of the North American Chapter of the As-sociation for Computational Linguistics: Demonstra-tions, pages 41–45.

Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan.2002. Thumbs up? sentiment classification using ma-chine learning techniques. In Proceedings of the 2002Conference on Empirical Methods in Natural Lan-guage Processing, pages 79–86.

Qiao Qian, Bo Tian, Minlie Huang, Yang Liu, XuanZhu, and Xiaoyan Zhu. 2015. Learning tag embed-dings and tag-specific composition functions in recur-sive neural network. In Proceedings of the 53rd An-nual Meeting of the Association for ComputationalLinguistics and the 7th International Joint Conferenceon Natural Language Processing, pages 1365–1374.

Ronald A. Rensink. 2000. The dynamic representationof scenes. Visual cognition, 7(1-3):17–42.

Alexander M. Rush, Sumit Chopra, and Jason Weston.2015. A neural attention model for sentence summa-rization. In Proceedings of the 2015 Conference onEmpirical Methods in Natural Language Processing,pages 379–389.

Yohei Seki, David Kirk Evans, and Lun-Wei Ku. 2007.Overview of opinion analysis pilot task at NTCIR-6. In Proceedings of the 6th NTCIR Workshop, pages265–278.

Yohei Seki, David Kirk Evans, and Lun-Wei Ku. 2008.Overview of multilingual opinion analysis task atNTCIR-7. In Proceedings of the 7th NTCIR Work-shop, pages 265–278.

Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang,Shirui Pan, and Chengqi Zhang. 2018. Disan: Di-rectional self-attention network for rnn/cnn-free lan-guage understanding. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence,pages 5446–5455.

Richard Socher, Jeffrey Pennington, Eric H. Huang, An-drew Y. Ng, and Christopher D. Manning. 2011.Semi-supervised recursive autoencoders for predictingsentiment distributions. In Proceedings of the 2011Conference on Empirical Methods in Natural Lan-guage Processing, pages 151–161.

Richard Socher, Brody Huvaland Christopher D. Man-ning, and Andrew Y. Ng. 2012. Semantic com-positionality through recursive matrix-vector spaces.In Proceedings of the 2012 Joint Conference on Em-pirical Methods in Natural Language Processing andComputational Natural Language Learning, pages1201–1211.

Richard Socher, Alex Perelygin, Jean Y. Wu, JasonChuang, Christopher D. Manning, Andrew Y. Ng, and

PACLIC 32



Christopher Potts. 2013. Recursive deep models forsemantic compositionality over a sentiment treebank.In Proceedings of the 2013 Conference on Empiri-cal Methods in Natural Language Processing, pages1631–1642.

Kai Sheng Tai, Richard Socher, and Christopher D. Man-ning. 2015. Improved semantic representations fromtree-structured long short-term memory networks. InProceedings of the 53rd Annual Meeting of the Associ-ation for Computational Linguistics and the 7th Inter-national Joint Conference on Natural Language Pro-cessing, pages 1556–1566.

Zhiyang Teng, Duy-Tin Vo, and Yue Zhang. 2016.Context-sensitive lexicon features for neural sentimentanalysis. In Proceedings of the 2016 Conference onEmpirical Methods in Natural Language Processing,pages 1629–1638.

Seiya Tokui, Kenta Oono, Shohei Hido, and Justin Clay-ton. 2015. Chainer: a next-generation open sourceframework for deep learning. In Proceedings of Work-shop on Machine Learning Systems in The 29th An-nual Conference on Neural Information ProcessingSystems.

Ashish Vaswani, Noam Shazeer, Niki Parmar, JakobUszkoreit, Llion Jones, Aidan N Gomez, Ł ukaszKaiser, and Illia Polosukhin. 2017. Attention is allyou need. In Advances in Neural Information Process-ing Systems 30, pages 5998–6008.

Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly.2015. Pointer networks. In Advances in Neural In-formation Processing Systems 28, pages 2692–2700.

Theresa Wilson, Janyce Wiebe, and Paul Hoffmann.2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the 2005Joint Conference on Human Language Technology andEmpirical Methods in Natural Language Processing,pages 347–354.

Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho,Aaron Courville, Ruslan Salakhudinov, Rich Zemel,and Yoshua Bengio. 2015. Show, attend and tell: Neu-ral image caption generation with visual attention. InProceedings of the 32nd International Conference onMachine Learning, pages 2048–2057.

Zichao Yang, Diyu Yang, Chris Dyer, Xiaodong He,Alex Smola, and Eduard Hovy. 2016. Hierarchi-cal attention networks for document classification. InProceedings of The 15th Annual Conference of theNorth American Chapter of the Association for Com-putational Linguistics: Human Language Technolo-gies (NAACL-HLT 2016), pages 1480–1489.

Peinan Zhang and Mamoru Komachi. 2015. Japanesesentiment classification with stacked denoising auto-encoder using distributed word representation. In Pro-

ceedings of the 29th Pacific Asia Conference on Lan-guage, Information and Computation, pages 150–159.

Xiaodan Zhu and Parinaz Sobhani. 2015. Long short-term memory over recursive structures. In Proceed-ings of the 32nd International Conference on MachineLearning, pages 1604–1612.

Yicheng Zou, Tao Gui, Qi Zhang, and Xuanjing Huang.2018. A lexicon-based supervised attention model forneural sentiment analysis. In Proceedings of the 27thInternational Conference on Computational Linguis-tics (COLING), pages 868–877.

PACLIC 32



japanese sentiment classiﬁcation using a tree-structured ... · the-art performance for a...

Documents