anticipatingcorporatefinancialperformancefromceoletters...

Research ArticleAnticipating Corporate Financial Performance from CEO LettersUtilizing Sentiment Analysis

Siqi Che 1 Wenzhong Zhu 2 and Xuepei Li3

1School of English for International Business Guangdong University of Foreign Studies Guangzhou 510420 China2School of Business Guangdong University of Foreign Studies Guangzhou 510420 China3School of Economics and Trade Guangdong University of Foreign Studies Guangzhou 510420 China

Correspondence should be addressed to Wenzhong Zhu 743106832qqcom

Received 15 January 2020 Accepted 19 February 2020 Published 13 May 2020

Guest Editor Weilin Xiao

Copyright copy 2020 Siqi Che et al 0is is an open access article distributed under the Creative Commons Attribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

With the emergence and tremendous growth of text mining a computer-assisted approach for capturing sentiment viewpointsfrom textual data is gradually becoming a promising field particularly when researchers are increasingly facing the problem offiltering bunches of useless information without capturing the essence in the big data era 0is study aims at observing andclassifying the sentiment orientation in CEO letters digging the main corporate social responsibility (CSR) themes and ex-amining the effectiveness of CEO lettersrsquo sentiment on forecasting financial performance A specific sentiment dictionary has beenproposed to identify and classify the sentiment orientation in CEO letters by utilizing the appraisal theory Additionally thequalitative data analysis software NVivo is applied to explore the CSR topics Furthermore a modified Altmanrsquos Z-score modeland machine-learning approach are employed to predict financial performance 0e results of preliminary evaluations validatethat approximately 6214 of the texts represent positive polarity even when companies are not in a promising economicsituation 0e CSR themes mainly focus on business ethical responsibility particularly ethical activities Among various machine-learning approaches the logistic regression approach is appropriate for predicting financial performance with the state-of-the-artaccuracy of 7046 0e encouraging results indicate that the sentiment information inCEO letters is a vital factor for anticipatingfinancial performance0is work not only offers a new analytic framework for associating linguistic theory with computer scienceand economic models but will also improve stakeholdersrsquo decision-making

1 Introduction

Due to researchersrsquo unrivalled and explosive expansion indata mining big data and artificial intelligence naturallanguage processing (NLP) in handling bunches of textualdata becomes an explosive and prevalent field with greatfuture prospects 0e high-tech novel technique of senti-ment analysis offers a more efficient and accurate way fortext processing and its amazing pace of innovation lowcosts and scalability make it a highly attractive and alter-native approach

Sentiment analysis also known as ldquoopinionsentimentminingrdquo or ldquosubjectivity analysisrdquo uncovers a prominentinterdisciplinary field of mixing computational linguisticsand computer science which attempts to extract subjectiveopinions feelings and attitudes contained in the text and

analyzes how to use language to deliver subjectivity andviewpoints on a particular topic [1]

Correspondingly corporate social responsibility report(CSRR) containing abundant sentiments is crucial forreflecting companiesrsquo sustainable standpoints on its oper-ating ideas strategies and methods For this reason CSRRcan be a valuable source for investorsrsquo decision-making Forinstance CEO letter contains the ideas that corporationscredibly desire to convey some information about them-selves to their potential stakeholders and these ideas may beimportant in determining investment and lending decisionsbecause they cover valuable background information aboutinterpreting and explaining financial performance whichmay not be covered in companiesrsquo financial statements [2]0is obviously boosts the requirements of sentiment analysisand turns into extremely precious resources for

HindawiMathematical Problems in EngineeringVolume 2020 Article ID 5609272 17 pageshttpsdoiorg10115520205609272

stakeholdersrsquo decision-making [3] China as one of thebiggest developing countries in the world has developedCSR rapidly especially the Chinese government has paidmuch more attention to sustainable development Howeverfewer researchers have exerted empirical evidence on thesentiment analysis of CSR in China particularly in the caseof information asymmetry

To our knowledge prior studies have proposed manifoldsentiment analysis techniques towards extracting usefulcontent from massive social media [4 5] some scholars havecategorized sentences into positivenegative [4] while otherresearchers have divided sentences into criticalemotional-based on researchersrsquo intuition 0e taxonomy resultshowever are inconsistent and incomparable as various self-designed frameworks have been applied in diverse studiesAlthough sentiment analysis of CSR or nonfinancial infor-mation appears to be a possible trend for predicting financialperformance and assisting investors to make future invest-ment decisions little research has been done in this field

As an attempt to make up the deficiency in the aboveresearch field this paper is divided into three dimensionsfirstly in the microlevel analysis it determines the sentimentpolarity and sentiment attributes in letters to shareholdersby utilizing appraisal theory secondly in the mesolevelanalysis it identifies the prominent CSR themes byemploying NVivo 12 plus software thirdly in themacrolevelanalysis it explores the sentiment elements so as to antic-ipate financial performance by using the expression ofZ-score

0e following parts of the paper are organized as followsin the part of literature review the related CSR CEO lettersprimary sentiment analysis approaches appraisal theoryand forecasting financial performance through textual in-formation will be reviewed the part of research method-ology will introduce the annotation study and theexperimental procedures in the part of experimental resultsand analysis the experimental results will be stated andanalyzed in the conclusion part the research results con-tribution limitations and suggestions will be provided

2 Literature Review

21 CEO Letters in CSR Report 0e CEO letter (hereaftershareholdersrsquo letter or letter to shareholders) is the mostwidely read part of the CSRR and a valuable communicationsource for stakeholdersrsquo decision-making Generally letterto shareholders is widely regarded as a promotional genrewhich tries to provide companiesrsquo subjective sentimentstandpoints and aims to portray a positive image [6] such aswhat stakeholders desired to know about identifying the lastyearrsquos performance tracking important events displayingcorporate social responsibility and predicting the futurevision of the companies 0us the letter stands in a vitalimportant position to deliver companyrsquos competitive ad-vantage [7] Although a number of scholars recognized thesignificance of CEO letters for instance Kohut and Segars[7] characterize the effectiveness of letter to shareholders andhow this information can benefit the company Patelli andPedrini [8] indicate that even under the tough economic

conditions companies still sincerely disclose nonfinancialinformation with stakeholders Surprisingly little researchhas focused on how companies try to construct their imagesand showed to readers through sentiment analysis

22 Sentiment Analysis In artificial intelligence (AI) textmining is an effective and efficient way to process a largenumber of textual contents through extracting the sentimentpolarity based on natural language processing In particularsentiment analysis has gradually become a popular tech-nique A considerable amount of literature has concentratedon analyzing documentsrsquo sentiment polarity [9] Automaticsentiment classification has been extensively applied to re-views of products [10] movies [11] books [12] shopping[13] social networks [14] and studentsrsquo course evaluationcomments [15 16] which is a common application ofclassifying positive or negative reviews Most recent workhas involved in extracting the textual information in thefinancial reports as the text may contain more informationthan the numerical part in an annual report [17 18] 0efinal results indicate that sentiment analysis is an importanttechnique for forecasting financial performance and thuscan be used to support the decision-making process ofpotential stakeholders [19 20] Researchers therefore haverealized the potential value of the textual informationanalysis of financial reports for predicting financial per-formance and managing risks

In fact the challenge of detecting sentiment from texthas been tackled from various perspectives Nonethelessprevious approaches to spot affect have been categorizedinto two main approaches lexicon-based approach andmachine-learning approach [21] which are explicitlydepicted in Figure 1

221 Lexicon-Based Approach Lexicon-based methods arerule-based requiring a predefined word list and polarity toidentify viewpoints and sentiments [4 22] containingcomputing orientation for a text from the semantic orien-tation of words or phrases [23] In other words when a newtext has been selected the words inside the text have tomatch with the words in the sentiment dictionary and thenvarious algorithms have been employed to aggregate values0e total aggregation of positive and negative values of thewords assembles the sentiment orientation of the whole textAn idealized operating mechanism is specified in Figure 2

Table 1 enumerates some related work in the lexicon-based approach for sentiment analysis and illustrates thevarious types of objectives along with the associated modelsused and the experimental results produced We employedprecision recall and F-measure as evaluation metrics theyare common in information retrieval and document clas-sification research Precision measures the number of cor-rectly classified items out of the total classified by aclassification technique and recall measures the amount ofcorrectly classified items of those manually classified as thegold standard 0e F-measure is the harmonic mean ofprecision and recall which offers a better measure than thearithmetic mean of precision and recall

2 Mathematical Problems in Engineering

0e merits of employing lexicon-based approach arethat across diverse domains lexicon-based methods do notrequire to alter dictionaries [4] Similarly Brooke Tofiloskiand Taboada [29] claimed that compared with machine-learning rule-based approach is not a cumbersome taskbecause scholars do not need to take a large amount of timeand effort to annotate the training data in a specific domain

0e lexicon-based approach however has its own limi-tations Sometimes the individual word extraction may missimportant meanings in the text Since the existing sentimentdictionary cannot meet the specific context of letter toshareholders for example in the dictionary words like lowerand decrease simply do not have a negative meaning in theCSR Hence in order to alleviate the dispute sentimentidentification requires a more comprehensive and proprietarysentiment dictionary for the specific CSR domain

222 Machine-Learning Approach 0e machine-learningapproaches are presented with training classifiers such assupport vector machine (SVM) Naive-Bayes (NB) logistic

regression (LR) and maximum entropy (ME) to assess thecontentsrsquo positive or negative orientations based on an initialcoding ten test upon the dataset to see if the sentiment isindeed captured [30] To put it another way human an-notators code a small sample of dataset and then the ma-chine takes over once it ldquolearnsrdquo what kinds of wordsresemble these positive and negative sentiments [31] Adetailed operating mechanism is specified in Figure 2

Generally speaking the machine-learning approach canbe further divided into supervised unsupervised andsemisupervised [32] Normally the training objective is to beable to classify or distinguish instances 0e main differenceis that the supervised machine learning-based approachselects labeled instances to construct the model 0e un-supervised approach is used for data mining to spot what isinside the unlabeled data Semisupervised technique ishalfway between supervised and unsupervised learning 0istechnique is about to use unlabeled data to learn a functionimproving the classification performance

Table 2 summarizes a great diversity of relevant literatureon machine-learning approach for sentiment identificationillustrating that most existing studies analyze text by creatinga large dataset to measure subjectivity

0e advantages of the machine-learning approach arethat once the trained dataset is available the classifier canrapidly determine the textrsquos polarity For instance Yang et al[35] applied a naive Bayes classifier and class associationrules to identify the sentiment of consumer reviews andproved to achieve a satisfactory result AdditionallyTroussas et al [36] classified the peoplersquos feelings and at-titudes about certain topics and stated how sentimentanalysis using naive Bayes can efficiently assist languagelearning Specifically Yuan et al [39] manifested that theclassification accuracy has been improved extremely basedon the sentiment analysis of CSR in financial reports throughthe SVM approach 0is provides the inspiration and basisfor the further research of this study

While the machine-learning approach has long been usedin topical text classification with good results because of theefficiency and accuracy it sometimes encounters withshortcomings as a result of high-level human manual inter-vention0at is to say the classifiers necessitate sizable humanannotated datasets for training and testing especially amid thebig data era which is extremely costly and time-consuming

Sentiment analysisapproaches

Machine-learningapproach

Lexicon-basedapproach

Corpus-basedapproach

Dictionary-basedapproach

Unsupervised learning-based approach

Supervised learning-based approach

Semisupervised learning-based approach

Naive Bayes Support vectormachine

Maximumentropy

Figure 1 Sentiment classification methods

Trainingdataset

(+)

Trainingdataset

(ndash)

Characteristics

Variousalgorithms

Classifier

Newtext

Semanticorientation

Wordlist Rules

New text

Extractwords and apply

rules

Semanticorientation

Machine learningLexicon-based

Figure 2 Machine-learning and lexicon-based approaches ofsentiment analysis

Mathematical Problems in Engineering 3

[31] Moreover sometimes the training datasets are un-available Previous attempts at categorizing movie reviews andbook reviews used the machine-learning approach withlimited success Because the technique may not suitable forvarious texts facing the obstacle of domain specificity theaccuracy of analysis has reduced greatly because the thoroughcustom-made training dataset may not generalize well to textsfrom other domains [40 41] Consequently in this study theparticular characteristics should be captured to perfectly satisfythe specific domain of letters to shareholders

23 Forecasting Financial Performance by Using TextInformation Recently sentiment analysis has been widelyexplored in understanding the relationship between text

information and corporate financial performance 0roughevaluating authorsrsquo opinions attitudes and sentiment po-larity the future financial performance could be predicted Agreat number of scholars have recently engaged in analyzingtextual information in annual reports newspapers andother documents to forecast stock return or financial per-formance Table 3 lists some literature to indicate that thesentiment of documents can significantly correlate with fi-nancial performance

From the above literature our study will try to use lettersto shareholders in CSRR to test whether the sentiment in theletter can predict the financial performance or not byemploying various kinds of machine-learning methodsFurthermore in order to have a better comparison of thesentiment attribute categorization and construct a suitable

Table 1 Related work of lexicon-based approach in sentiment analysis

Authors Objectives Models Data source Evaluationmethod Data set Accuracy

(percent)Precision(percent) Recall F1

Hatzivassiloglouand Mckeown[24]

Assignadjective plusmn

Nonhierarchicalclustering WSJ corpus NA 657 adj (+) 679

adj (minus) 781ndash924 NA NA NA

Turney [23]Assigndocs

sentimentsPMI-IR

Automobilebank movie

travelreviews

NA 240 (+) 170 (minus) 658ndash84 NA NA NA

Turney andLittman [25]

Assigndocs

sentimentsPMI LSA

AV-ENGcorpus AV-CA corpusTASAcorpus

NA 1614 (+) 1982(minus) 828ndash95 NA NA NA

Taboada et al[26]

Assignadjectivesplusmn

SO-PMI Reviews NA 521 adj 495ndash5675 NA NA NA

Ding et al [27]

Assignadjectivesadverbsverbs andnouns plusmn

OpinionObserver

445 customerreviews ofproductsfrom

amazoncom

Macroaveraging(macroaveragingmeans given a setof confusiontables a set ofvalues are

generated andeach value

represents theprecision or recallof an automaticclassifier for each

category)

NA

NA 91 90 90

No contextdependency NA 92 83 87

Without usingequation NA 90 85 87

FBS NA 92 74 82

Taboada et al [4]

Assignadjectivesverbsadverbsnouns plusmn

SO-CAL Reviews Across variousdomains NA 80 NA NA NA

Dey et al [28]Assigndocs

sentiments

SO-CAL Senti-N-Gram

Moviebooks carscookwarephoneshotels

music andcomputersreviews

3-fold crossvalidation

Books 68 66 76 71Cars 84 76 100 86

Computers 80 73 96 83Cookware 74 68 92 78Hotels 74 67 96 79Movies 66 64 72 68Music 64 62 72 67Phones 86 85 88 86


Table 2 Related work of machine-learning approach in sentiment analysis

Authors Objectives Models Data source Evaluation method Data set Accuracy(percent)

Precision(percent) Recall F1

Bruce andWiebe[33]

Assignsentencessubjectivityusing 1 to 4

scale

LCA

14 articles inWall StreetJournalTreebankCorpus

10-fold cross validation

486subjective

515objective

7217 NA NA NA

Pang et al[30]

Assign docssentiments SVM NB ME Movie

reviews 3-fold cross validation 700 (+) 77ndash829 NA NA NA700 (minus)Bo andLee [1]

Assign docssentiments SVM NB Movie

reviews 10-fold cross validation 1000 (+) 864ndash872 NA NA NA1000 (minus)GamonandMichael[34]

Assign docssentiments

using 4-pointscale

SVM Customerfeedback

10-fold cross validation NA 775 NA NA NA


Yang et al[35]

Assign topicsentiments

Classassociation

rules Consumerreviews

Macroaveraged

NA

NA 6764 775 7226Microaveraged

(microaveraging meansgiven a set of confusiontables a new two-by-two contingency table isgenerated each cell inthe new table representsthe sum of the numberof documents from

within the set of tables)

NA 6747 744 7226

NB Macroaveraged NA 7396 6679 7019Microaveraged NA 683 6611 6719

Troussaset al [36]

Assign docssentiments NB

Facebook7000 statusupdates from

90 users


1131 (+)

NA 77 68 721131 (minus)

Singhet al [37]


NB Productreview

movie review4-fold cross validation NA

456 831 NA 812J-48 967 877 NA 917

BFTree 892 883 NA 721OneR 90 913 NA 97

Rathi [38] Assignadjectives plusmn

SVM+KNNhybrid Tweet NA

267 (+)7617 NA NA NA264 (minus)

168 (0)

Table 3 Related work of forecasting financial performance using textual information

Authors Material Period Methods Key findings

Loughran andMcdonald[42]

Annual report 1994ndash2008 Sentimentdictionary

Textual analysis can contribute to the ability tounderstand the impact of information on stock returnsMost importantly researchers should be cautious whenrelying on word classification scheme derived outside

the domain of business usage

Schumaker et al[43] Financial news articles 2005 Machine learning

0e financial news article prediction system can predictprice increase or decrease in sentiment analysis with the

accuracy of approximately 50Hajek and Olej[44]

520 US companiesrsquoannual report 2010 Machine learning Sentiment information in annual reports can

significantly improve the accuracy of the used classifiers

Xin et al[45]

US manufacturingindustry annual report 1997ndash2003 Machine learning

Using machine-learning approach employed with theresults of text analysis can predict next yearrsquos earnings

per shareHajek et al[46]

448 US companiesrsquoannual report 2008 Machine learning

neural networksSentiment information is an important determinant for

forecasting financial performanceTong et al[47] Microblogs in China 20162ndash20169 Machine learning 0ere is a strong correlation between chat room

postsentiment and stock price movement


sentiment classification dictionary for the specific contextappraisal theory will be applied and the seed dictionary willbe manually annotated for the domain of CEO letters

24 Appraisal0eory Although machine-learning methodsshow a better performance the results of each experimentare not comparable because of various kinds of categori-zations A detailed theoretical framework is highly requiredto handle this difficulty Researchers are asked to addressmore challenging tasks and attempt to perform more so-phisticated sentiment analyses based on systematic and well-grounded sentiment theories and frameworks 0eseframeworks will become common theoretical platforms forcomparing results across different studies

To date the existing sentiment techniques may not besufficient for sentiment classification Linguistic frameworktherefore has been employed as a theoretical platform forsentiment analysis By far computational linguists haveprojected several sentiment frameworks Wiebe et al [48]disclosed a sentiment framework based on private stateswith three types of expressions that is explicit mentions ofprivate states speech events expressing private states andexpressive subjective elements0e framework is designed toexpand attitude types to include intention warning un-certainty and evaluation Asher Benamara and Mathieu[49] further denoted an annotation framework involvingfour categories that is reporting expressions judgementexpressions advice expressions and sentiment expressionsto express various kinds of feelings and opinions 0e mostcomprehensive linguistic theory of sentiment however isthe appraisal framework [50] also known as interpersonalsemantics developed within the tradition of SystemicFunctional Linguistics [51] Appraisal theory is a frameworkemployed in conveying the language of evaluation in text[52] and also an approach to linguistics that focuses on thesemantics of text rather than its grammar [53]

0e framework portrayed a taxonomy of the language toconvey attitude engagement and graduation with respect tothe evaluations of other people 0e detailed taxonomy isdepicted in Figure 3 Attitude considers how people conductinterpersonal interaction to express his or her opinions andemotions engagement assesses the evaluation with respectto othersrsquo opinions and graduation denotes how tostrengthen or weaken the attitude through languagefunctions

Attitude engagement and graduation can be furtherdivided into several subtypes which are explained more in-depth in the following paragraphs

Attitude can be further divided into three distinctsubsystems affect appreciation and judgement Affectidentifies the feelings and emotions of the author (happyand sad) which can be a behavioral process or an internalmental state Appreciation represents the reaction that aperson talks about (beautiful and ugly) such as impactquality composition complexity or valuation Judge-ment considers the authorrsquos attitude towards the behaviorof somebody (heroic and idiotic) and the evaluation mayconcern social sanctions or social esteem social sanctions

may involve veracity or propriety and social esteem mayinvolve the assessment of how normally someone be-haves how capable the person is or how tenacious theperson is

Engagement contains two subcategories monoglossicand heteroglossic Monoglossic has no recognition of dia-logistic alternatives that is the writerspeaker directlyexpressed the appraisal Heteroglossic has the recognition ofdialogistic alternative positions and voices which meansthat the writerspeaker has either attributed to othermethods or sources to make it credible

Graduation also include two subclasses force and focusForce assesses the degree of intensity focus covers tosharpen or soften the specification

0e main advantage of utilizing appraisal theory insentiment identification is that it enables us to take a lookdeeper inside the thoughts of authors or publishers re-vealing their real sensations by employing linguistic andpsychological analysis of their texts In our study we want toextract the sentiment inside in the shareholdersrsquo letters andmay possibly draw some statistical conclusions about thetypes of sentiment that companies of shareholdersrsquo lettersexpress via the texts they write At last the analysis can helppotential investors to make better decision-makings

So far scholars have made a great number of usefulattempts at the level of sentiment analysis Table 4 sums upthe relevant sentiment analysis literature employing theappraisal framework From the table the research inte-grating the appraisal theory into the analysis of sentimentorientation is relatively scare Korenek and Simko [60]established the method of manually creating an emotionaldictionary utilizing the appraisal theory to evaluate emo-tional posts on Weibo posts Taboada and Grieve [54]further made some improvements on how to aggregate thevalue of adjectives based on appraisal theory In additionKhoo et al [50] extended the appraisal framework for an-alyzing long news reports compared with microposts and

Appraisal

Engagement

Attitude

Graduation

Monogloss

Heterogloss

Affect

Judgement

Appreciation

Force

Focus

Figure 3 Outline of Martin and Whitersquos [52] appraisal framework(first two levels)


assessed the frameworkrsquos utility and possible problemswhich is definitely a good attempt for our current study

As indicated in Table 4 the aforementioned work hasimproved that appraisal framework is a highly useful tool foranalyzing sentiment classification [50 60] 0is paper thusintends to augment the sentiment classification of presidentrsquosletter by employing the appraisal theory 0e appraisal theoryproves to be useful in uncovering various aspects of sentimentthat should be valuable to researchers and assists researchersto understand the feelings of shareholdersrsquo letters better Byfollowing the doctor dissertation of Bloom [59] who laid thebasis for sentiment classification by utilizing appraisal theorya more systematic and well-grounded approach of sentimentanalysis utilizing Martin and Whitersquos appraisal theory hasbeen heuristically employed which can successfully assistpotential investors to understand corporationsrsquo attitudesaccurately and make investment decisions more efficiently

3 Research Methodology

As mentioned earlier in order to achieve our goals in thisstudy a three-dimensional research framework has been

established to enable us to systematically test differentperspectives of text mining strategies with both quantitativeand qualitative methods (see Figure 4)

0e specific research questions that we address are thefollowing

RQ1 what are the prominent sentiment attributes inCEOrsquos letters that can actively promote companiesrsquoimages and avoid negative viewsRQ2 what are the main sentimental themes in CEOrsquosletters And which one can best encourage othercompaniesrsquo investmentRQ3 how can the sentiment attributes of CEOrsquos lettersanticipate corporate financial performance that wouldbe useful for stakeholdersrsquo decision-making

0e above questions in three dimensions evaluate CEOletters In the microlevel of identifying sentiment attributesappraisal theory and lexicon-based sentiment method havebeen applied to classify various sentiment attributes pre-cisely In the mesolevel analysis we utilize NVivo qualitativedata analysis software to do the thematic analysis of CEOletters In the macrolevel machine-learning approach was

Table 4 Related work of appraisal theory in sentiment analysis

Authors Models Data source Method Attributes type Accuracy(percent)

Taboada andGrieve [54] Lexicon-based Reviews

Taking into account textstructure and adjectives

frequencyAttitude NA

Whitelawet al [55] Lexicon-based Movie reviews Analyzing appraisal adjectives

and modifiersAttitude orientationgraduation polarity 902

Read andCarroll [56] Lexicon-based Book reviews Measuring interannotator

agreement

Attitudeengagement

graduation polarityNA

Argamonet al [57] NB SVM Documents

Automatic determiningcomplex sentiment-related

attributes

Attitude orientationforce NA

Balahur et al[58]

Robert Plutchikrsquoswheel of emotion

Parrotrsquos tree-structured list of

emotions

ISEAR Corpus

EmotiNet (EmotiNet defines tostore action chains and their

corresponding emotional labelsfrom several situations in such away authors could be able toextract general patterns of

appraisal)

Actor action object

NAEmotion

Bloom [59] Lexicon-based

MPQA 20 Corpus UICReview Corpus DarmstadtService Review CorpusJDPA Sentiment CorpusIIT Sentiment Corpus

Functional local appraisalgrammar extractor Attitude 446

Khoo et al[50] Lexicon-based Political news article Analyzing appraisal groups Actor attitude

engagement polarity NA

Korenek andSimko [60] SVM Microblog posts Structuring sentiment analysis

based on appraisal theory

Attitude graduationpolarity orientation

engagement8757

Cui andShibamoto-Smith [61]

Lexicon-basedSelf-constructed corpus ofChinese newspapers and

websites

Discovering the lexicalsyntactic and semantic featuresof different types of sentiment

parameters

Attitude

NAPeripheral sentimentparameters (topic

source field processdegree)


selected to assess the power of CEO letters in predictingcorporate financial performance in terms of the Z-scoremodel

31 Data Collection and Description To date variousmethods have been developed and introduced to measuresentiment A suitable method to adopt for this study is topropose a specific sentiment dictionary based on the prin-ciples of appraisal theory 0e advantage of utilizing theappraisal theory is that it allows researchers to categorize andcompare sentiment attributes based on a systematic lin-guistic theory and also enables them to explore authorsrsquothoughts more thoroughly 0e major difference from theprevious works which use the appraisal theory is that thewhole basic appraisal tree has been selected and share-holdersrsquo letter specifics have been assessed

In terms of the variety of genres such as political newsmovie reviews and product reviews the analysis results willsignificantly differ depending on distinct syntactic structureand lexical choices [56] 0erefore in order to maintainconsistency the same genre has to been examined based onthe appraisal framework In this study shareholdersrsquo lettersare good candidates for this study because they are likely tocontain similar languages in the same genre of writing alsoexamples of appraisalrsquos attributes can be easily detectedMost importantly identifying valuable information fromshareholdersrsquo letters is a sufficient way to help companies toimprove the quality of released information and facilitatestakeholdersrsquo to make investment decisions

All CSR reports with English version were downloadedfrom GRIrsquos Sustainability Disclosure Database (httpsdatabaseglobalreportingorg) 0is database can access toall types of sustainability reports from various industriesrelating to the reporting organizations For the sake ofpreventing problems with both industry-specific attributesand different financial performance evaluation the financialindustries were excluded Furthermore in order to ensureaccurate comparable information all selected reports must

have official English version and should be listed in the HongKong publicly regulated markets

Finally 41 Chinese companiesrsquo English version CSRreports from year 2016 had been collected 0en 41shareholdersrsquo letters were retrieved from the CSR reports inChina in 2016 0e corpus contains 35670 tokens in totalnamed SLC In addition all the statistical data for calculatingZ-score are gathered from the Wind financial database

0e year 2016 was selected because the United NationsGeneral Assembly unanimously adopted the Resolution 701 Transforming ourWorld the 2030 Agenda for SustainableDevelopment0is historic document lays out 17 sustainabledevelopment goals which aim to integrate and balance thethree dimensions of sustainable development economicsocial and environmental 0e new goals and targets willcome into effect on 1 January 2016 and will guide for the nextfifteen years

32 Data Preprocessing Most CEO letters were released inPDF format Firstly letters were manually trans-formedpdf documents to the essentialtxt groups andtexts were sorted to have only one sentence in each line0en the text documents were linguistically preprocessedusing tokenization part of speech tagging andlemmatization

Tokenisationmdashsplitting texts into sentences and wordsPart of speech taggingmdashadding a POS tag to each wordin a sentenceLemmatisationmdashconverting a word into its basiclemma formDeletion of stop words and company name

To control for bias in the analysis in particularshareholdersrsquo letters have a specific content that differs fromother texts which has to be coped with For instance theabbreviation of companyrsquos name is Best Buy including thepositive word of ldquoBestrdquo which may significantly convert the

Textual information fromCEO letters in CSRR

Linguistic preprocessing

Specific sentimentdictionary construction and

classification

Analyzing sentimentattributes

Using NVivo to extractdistinctive themes

Financial indicators fromwind database

Altmanrsquos Z-score modelon data period t

Evaluating predictedresults

Mesolevel analysis Macrolevel analysis

Applying Z-score modelto data period t + 1

Microlevel analysis

Figure 4 An overview of the workflow of the research framework


polarity of the text As a result the pretreatments were acombination of tokenization lemmatization part-of-speech-tagging and deletion of stop words and namedentities which gave the best result

33 Specific Dictionary Construction and Classification0e main issue with the sentiment analysis of textual doc-uments is the right choice of positive terms Obviously thecategorization of words is not always unambiguous andrequires context knowledge 0is is due to the variousmeanings of words and domain specific tone of the wordsrespectively 0erefore the appraisal theory has beenemployed in this study

To evaluate the appraisal theory a dictionary taggedwith attributes from Martin and Whitersquos categorization(2005) has to be constructed In terms of the work ofKorenek and Simko [60] they built a dictionary based onappraisal theory specializing in microblogs By followingtheir work we accumulated approximately five hundredand fifty words from Martin and Whitersquos classification Inorder to broaden the dictionary WordNet database Col-lins0esaurus andMerriam-Webster0esaurus have beenutilized to find synonyms to words identified in the pre-vious step and this formed the basic sentiment lexicons0e next step is to be aimed at extracting candidate sen-timent word lists from the self-constructed SLC corpusemploying the statistical approach based on the amount ofpointwise mutual information (PMI) ratio which can beused to compare candidate words with the existing basicsentiment lexicons PMI calculation has been defined asfollows

PMI word1word2( 1113857 logp word1word2( 1113857

p word1( 1113857p word2( 1113857 (1)

In light of the PMI ratio whether the word should beidentified as a target or not must be decided On completionof selecting targets the target candidate words merged withthe basic sentiment lexicons to construct a new dictionaryspecializing in shareholdersrsquo letter content with approxi-mately six hundred words in total

Due to the specifics of shareholdersrsquo letters the tradi-tional lexicon-based approach cannot identify the complexcontext So the statistical method of PMI ratio may not beaccurate all the time Manual screening is highly needed inthis step

To further categorize the new dictionary each word ismanually classified according to the attributes based onappraisal theory and in line with the research of [60] whoassigned a 10-point Likert scale from minus5 to +5 0e samescaling scope has been considered in this research For thepurpose of manual coding with participantsrsquo subjectivejudgement eight human annotators a to h were invitedto assign polarity independently 0e annotators were allwell-versed in the appraisal framework 0ey were askedto specify the type of attitude engagement or graduationpresent and assign a scaling and polarity to candidate

words In order to address the concern of inconsistentunderstanding regarding some ambiguous words theannotation was executed over two rounds punctuated byan intermediary analysis of agreement and disagreementamong all annotators until a consensus has beenachieved

A partial example of entries is presented in Table 5 Eachentry contains a word a part-of-speech tag a category andsubcategories according to the appraisal theory and ap-praisal value

Based on the newly established sentiment dictionary fora specific corpus we can further precisely analyze thesentiment characteristics of CEO letters frommicro- meso-and macrolevel analysis

34 Data Analysis

341 Microlevel Analysis Regarding RQ1 we categorizedCEO letters based on the specific established sentimentclassification dictionary considering the following elevencategories of terms

A Positive attitude affect (eg happy convinced andsatisfied)B Negative attitude affect (eg sorry and sad)C Positive attitude judgement (eg lucky fortunateand famous)D Negative attitude judgement (eg imperfect un-known and severe)E Positive attitude appreciation (eg exciting dra-matic and excellent)F Negative attitude appreciation (eg imbalanceddisharmonious and conflicting)G Positive engagement (eg certainly obviouslynaturally and evidently)H Negative engagement (eg falsely and compellingly)I Positive graduation force (eg greatly slightly andsomewhat)J Negative graduation force (eg small and remote)K Positive graduation focus (eg true and genuinely)

We assumed that well-performing corporations arebeing more positive optimistic tones Conversely we ex-pected a more active language in the case of poorly per-forming companies that need to take positive actions toimprove their image and attract more investors

342 Mesolevel Analysis With the popularity of computertechnology a range of software packages emerged to assistwith the analysis of qualitative data It is generally acceptedthat computer-assisted qualitative data analysis software(CAQDAS) can enhance the data handlinganalysis processif used appropriately and resolve analysts from complicateddata analysis


0e software package NVivo (now update to version 12)is one of the most distinguished CAQDAS which can helpthe analysis to work more efficiently and rigorously back upfindings with grounded data [62]

In this study NVivo has been prompted to do a thematicanalysis for identifying the broad CSR topics existing in theshareholdersrsquo letters 0ematic analysis is a way of catego-rizing data from qualitative research through analyzingsimilar themes and interpreting the research findings [63]In this study thematic analysis can be employed to detect themain ideas existing in the CEO letters and through NVivosoftware to label or code different types of CSR

NVivo allows nodes to have more than one dimension(tree branch) 0erefore we were able to identify whereconcepts may have more than one dimension or group themwithin a more general concept0is is a revolution in findingconnections because it prompts the analyst to think abouttheir concepts in more detail facilitating conceptual clarityand early discourse analysis [64] Figure 5 reveals a sample ofthe tree node structure of CSR

Coding stripes function is a useful function for re-searchers to annotate various segments in the whole doc-uments which may facilitate the comparison of categoriesFigure 6 depicts an example of data coded at the ethicalresponsibility node

Generally speaking NVivo offers us a valuable tool toexplore the complexities of potential relationships withoutforcing the data to fit specific categories In this way whenwe identified a possible relationship with CSR we definedthis in NVivo using the free code to represent it first and latercreated tree node to identify the internal relationship

343 Macrolevel Analysis 0e Altmanrsquos model of financialhealth (Altmanrsquos Z-score) was selected for the quantitativeevaluation of the assessed companies 0e reason of theselection was that this model was created for assessingcompany financial health in industrial branch with sharestradable on the Hong Kong publicly regulated markets andexactly such companies were selected for the study

0e scope of this so-called bankruptcy model is topredict the probability of survival or bankruptcy of theanalyzed company0e nearer to the bankruptcy a companyis the better Altmanrsquos index works as a predictor of financialhealth It predicts bankruptcy reliably about one to two yearsin advance Altmanrsquos Z-score model uses the followingrelation to define the value of a company in industrial branchwith shares publicly tradable on the stock market

Zi 12X1i + 14X2i + 33X3i + 06X4i + 10X5i (2)

where i denotes the i-th company X1 is the working capitaltotal assets X2 is the retained earningstotal assets X3 is theearnings before interest and taxtotal assets X4 is the marketvalue of equitytotal liabilities and X5 is the salestotal assetsDetailed variable definition can be found in Table 6

0e Zi value is in range minus4 to +8 0e higher the valuethe higher the financial health of a company It holds truethat if (1) Zigt 299 the company is in the ldquosafe zonerdquo (acompany with high probability to survivemdashfinanciallystrong company) (2) 180leZile 299 ldquogrey zonerdquo (the futureof the company cannot be determined clearlymdasha companywith certain financial difficulties) and (3) Zilt 180 ldquodistresszonerdquo (the company has serious financial problemsmdashthecompany is endangered by bankruptcy)

Due to the special historical conditions of Chinarsquos stockmarket formation the types of stocks formed in China aredifferent from those in foreign countries In the stocks oflisted companies they can generally be divided into twocategories tradable shares and nontradable shares In viewof the fact that there is no market price for nontradableshares in the Hong Kong stockmarket the model has made aslight change X4 (share pricelowast tradable shares + net assetvalue per sharelowastnontradable shares)total liabilities and X5is the prime operating revenuetotal assets

0e outputs of the forecasting models were representedby the classes of financial performance obtained using theZ-score bankruptcymodel namely classes ldquosafe zonerdquo ldquogreyzonerdquo and ldquodistress zonerdquo In addition we obtained addi-tional output classes (increase no change and decrease)Given the fact that the classes were imbalanced in thedataset we use the Synthetic Minority OversamplingTechnique (SMOTE) algorithm [65] to modify the trainingdataset 0e algorithm oversamples the minority classes sothat all classes are presented equally in the training dataset

0e set of eleven sentiment attributes presented in theprevious section was drawn from shareholdersrsquo letter Fol-lowing previous studies [46 66] the input attributes werecollected for Chinese companies in the year 2016 while theoutput financial performance (Z-score) was evaluated for theyear 2017 and the change in the financial performance wasmeasured as Z-score in 2017 related to its value in 2016 Forthe sake of preventing problems with both industry-specificattributes and different financial performance evaluationthe financial industry was excluded As a result among 41Chinese companies 9 companies were classified into ldquogreyzonerdquo and 32 companies were classified into ldquodistress zonerdquo

Table 5 A partial example of appraisal dictionary entries

Word Part of speech Main category Second level category Other level categories Appraisal valueGood Adj Attitude Judgement Praise propriety 3Harmonious Adj Attitude Appreciation Composition balance 2Advanced Adj Attitude Appreciation Valuation 5Clear Adj Attitude Appreciation Composition complexity 1Never Adv Engagement NA NA minus5Incredible Adj Attitude Judgement Admiration normality 3Largest Adj Graduation Force Maximization 1


after making a comparison of financial situation between2016 and 2017 only 2 companies were improved and theremaining 39 companies were unchanged

In sum the Z-score of Chinese companies in 2016 and2017 could be spotted in Table 7

Next a various number of forecasting machine-learningmethods have been explored ie logistic regression supportvector machine and naıve Bayes Apart from logistic re-gression the other two methods can process nonlinear data

0e logistic regression (LR) model has been used with aridge estimator defined by Cessie and Houwelingen [67] 0eclassification performance of the logistic regression dependson the number of iterations and ridge factor respectively

Support vector machines (SVMs) are a set of relatedsupervised learning methods which are popular for per-forming classification and regression analysis using dataanalysis and pattern recognition

Naıve Bayes (NB) is a simple multiclass classificationalgorithm with the assumption of independence betweenevery pair of features Naive Bayes can be trained very ef-ficiently Within a single pass on the training data itcomputes the conditional probability distribution of eachfeature given label and then it applies Bayesrsquo theorem tocompute the conditional probability distribution of the labelgiven observation and use it for prediction

In sum logistic regression support vector machinenaıve Bayes are three methods designed to forecast theaccuracy of financial performance

4 Experimental Results and Discussion

In the data filtering we select 41 CEO letters in Chinesecompanies CSR report and try to do in-depth analysis atmicrolevel mesolevel and macrolevel respectively

41 Sentiment Attributes Regarding the eleven sentimentattributes the detailed statistical data can be found in Ta-ble 8 Among all the categories positive attitude judgementappreciation and positive graduation force are the top threemost frequent sentiment attributes

From the previous data collection part we know thatamong 41 Chinese companies 9 companies were classifiedinto ldquogrey zonerdquo and 32 companies were classified intoldquodistress zonerdquo None of the companies were classified intothe safe zone Combing the eleven categorizations with our2016 financial performance interestingly we found thatpoorly performing companies are expected to use a moreactive language to describe and evaluate their CSR 0isphenomenon can be explained further by the impressionmanagement effect Impression management refers to theprocess by which people try to manage and control theimpression others make about themselves [68] For cor-porations the correct impression management can helpcompanies to communicate with stakeholders smoothlyCompanies try to use impression management to activelypromote their images and avoid negative views 0is isprobably the reason why companies may encounter with agloomy economic situation but still concentrate on shapingpositive and optimistic images

42 Sentimental 0emes 0e next segment is about toidentify the sentimental themes in CEO letters throughNVivo software NVivorsquos coding stripes functions enable usto examine all the relevant text and identify the sentenceswhich contributed to that relationship and also to find outwhich node the sentences belong to After coding all therelevant text we gathered comprehensive sentimentalthemes existing in CEO letters (see Table 9)

Figure 5 Tree node structure of CSR in China


According to Table 9 the most popular sentimentaltheme in CEO letters is the ethical responsibility Businessethical responsibility for companies means a system of moraland ethical beliefs that guide companiesrsquo behaviors valuesand decisions and minimizing unjustified harm sufferingwaste or destruction to people and the environment [69]

0is result reveals that a large amount of companies (32among 41 companies) have put great efforts into businessethics 0e concept of business ethics began in the 1960s ascorporations became more aware of a rising consumer-based society that showed concerns regarding the envi-ronment social causes and corporate responsibility In fact

Figure 6 Coding stripes on the ldquoethical responsibilitiesrdquo node

Table 6 Financial variable definition

Variable name DefinitionWorking capital Current assets minus current liabilities

Total assets 0e final amount of all gross investments cash and equivalents receivables and other assets as they arepresented on the balance sheet

Retained earnings 0e amount of net income left over for the business after it has paid out dividends to its shareholdersEarnings before interest and tax(EBIT) Revenue minus expenses excluding tax and interest

Market value of equity 0e total dollar value of a companyrsquos equityTotal liabilities 0e combined debts and obligations that an individual or company owes to outside partiesSales Total dollar amount collected for goods and services provided


the importance of business ethics reaches far beyond thestrength of a management tea bond or employee loyaltyAs with all business initiatives the ethical operation of acompany is directly related to companiesrsquo short-term or

long-term profitability 0e reputation of a business in thesurrounding community other businesses and individualinvestors is paramount in determining whether a com-pany is a worthwhile investment If a company is per-ceived to be unethical investors are less inclined tosupport its operation

In addition in this study we have divided the ethicalresponsibility into three parts ethical accomplishmentethical activity and ethical ability Ethical accomplishmentmeans some awards that have been achieved by companiesEthical ability expresses companiesrsquo core competence andorganizational identity Ethical activity makes investors toknow about the activities and functions taking place in thecompany Detailed samples are scheduled below

(a) We implemented new energy efficiency projectsworldwide including the implementation of the ISO

Table 7 Z-score of Chinese companies in 2016 and 2017

Stock code Country 2016 Z-score 2016 2017 Z-score 20171958hk CN 1260515439 Distress zone 1482271118 Distress zone0606hk CN 1838183342 Grey zone 2216679937 Grey zone1800hk CN 0867078825 Distress zone 0864201511 Distress zone1829hk CN 1357941592 Distress zone 1427298576 Distress zone0941hk CN 2936618457 Grey zone 2789570355 Grey zone3323hk CN 0337950565 Distress zone 0497012524 Distress zone0390hk CN 1192247784 Distress zone 115640714 Distress zone3320hk CN 2201481901 Grey zone 2007730795 Grey zone1055hk CN 0551047016 Distress zone 0659758361 Distress zone0728hk CN 1062436514 Distress zone 1190768229 Distress zone0688hk CN 1961825278 Grey zone 1972625832 Grey zone2196hk CN 1215563686 Distress zone 1074862508 Distress zone1618hk CN 089432166 Distress zone 0879602037 Distress zone0857hk CN 1181842765 Distress zone 1329173357 Distress zone3377hk CN 1083233034 Distress zone 1142850174 Distress zone0347hk CN 071319999 Distress zone 1340165363 Distress zone0992hk CN 1775439241 Distress zone 1686162129 Distress zone0753hk CN 0740145558 Distress zone 0812868908 Distress zone0883hk CN 1927734935 Grey zone 2336603263 Grey zone0232hk CN 0975494042 Distress zone 1838159662 Grey zone1186hk CN 1251163858 Distress zone 1239974657 Distress zone2607hk CN 2345340814 Grey zone 2234627565 Grey zone0763hk CN 1059868156 Distress zone 1363358965 Distress zone0670hk CN 0482355283 Distress zone 0455072698 Distress zone2202hk CN 0807062775 Distress zone 0687163013 Distress zone0489hk CN 2294304149 Grey zone 2115933926 Grey zone0836hk CN 099429478 Distress zone 0843126034 Distress zone0392hk CN 1105575201 Distress zone 111398028 Distress zone0604hk CN 1226738962 Distress zone 0889164039 Distress zone2380hk CN 053081341 Distress zone 0310240584 Distress zone0257hk CN 1719616264 Distress zone 1540477349 Distress zone0103hk CN 0669137242 Distress zone 0689465751 Distress zone1211hk CN 1261617869 Distress zone 1124410212 Distress zone0916hk CN 0406631641 Distress zone 0557081816 Distress zone0175hk CN 2405647254 Grey zone 448691365 Safe zone1088hk CN 1243041938 Distress zone 1543226046 Distress zone0123hk CN 0919420683 Distress zone 1015226322 Distress zone2128hk CN 2456387186 Grey zone 2308100735 Grey zone2866hk CN 0125023125 Distress zone 0230025232 Distress zone3360hk CN 0346809818 Distress zone 0355677289 Distress zone6166hk CN 1678887209 Distress zone 173679922 Distress zone

Table 8 0e frequency of eleven sentiment attributes

Positive graduation focus 15Negative graduation force 15Positive graduation force 274Negative engagement 7Positive engagement 223Negative attitude appreciation 6Positive attitude appreciation 345Negative attitude judgement 11Positive attitude judgement 378Negative attitude affect 2Positive attitude affect 87


50001 Energy Management System in all our EuropeanUnion locations (Ethical activity Lenovo 2016)

(b) In 2016 we have achieved safe flight hours of 238million transported 115 million passengers elimi-nated incidents by human errors continued tomaintain the best safety record in Chinarsquos civilaviation (Ethical accomplishments China SouthernAirlines 2016)

(c) Consequently China Telecom is among the firstbatch of the national demonstration bases for en-trepreneurship and innovation (Ethical abilityChina Telecom 2016)

From the coding results ethical activity occupies thelargest proportion which means that firms would prefer todemonstrate their actions and practices to the societyCorporations have more incentives to be ethical as the areaof socially responsible and ethical investing keeps growingAn increasing number of investors are seeking ethicallyoperating companies to invest which drives more firms totake this issue seriously With the consistent ethical be-havior an increasingly positive public image can be estab-lished and to retain a positive image companies must becommitted to operating on an ethical foundation as it relatesto the treatment of employees respecting the surroundingenvironment and fair market practices in terms of price andconsumer treatment

43 Machine-Learning Approach Forecasting FinancialPerformance We tested many machine-learning ap-proaches and selected only the best outcomes 0e measuresof classification performance were in addition to Accrepresented by the averages of standard statistics applied inclassification tasks [70] true-positive rate (TP) false-posi-tive rate (FP) precision (Pre) recall (Re) F-measure (F-m)

and ROC curve F-m is the weighted harmonic mean ofprecision and recall ROC is a plot of the true-positive rateagainst the false-positive rate for the different cutpoints of adiagnostic test In order to validate the accuracy the ex-periments were realized using 10-fold cross validation 0ebest results in terms of the accuracy of correctly classifiedinstances Acc () are presented in Table 10 (for the financialperformance classes)

0e results obtained by modeling point out that thelogistic regression is more suitable for forecasting financialperformance reaching the highest accuracy of 7046 0isevidence suggests that there exists a linear relationshipbetween the sentiment and financial performance

5 Conclusions

CEO letter contains information about corporate socialresponsibility performance in CSRR which is designated forstakeholders to make their investment decisions In thisstudy sentiment analysis has been applied to the evaluationof CEO letters from three perspectives sentiment dictionary(microlevel) sentimental themes (mesolevel) and machinelearning (macrolevel) In the microlevel analysis a desig-nated sentiment dictionary has been constructed for clas-sifying sentiment attributes 0e results denote that no

Table 9 Main sentimental themes in CEO letters

Responsibility type Detailed categories References Sources

Ethical responsibilityEthical ability 34 23Ethical activity 113 32

Ethical accomplishments 37 20

Technical responsibilityTechnical ability 8 6Technical activity 33 13

Technical accomplishments 18 13

Economic responsibilityEconomic ability 5 5Economic activity 12 9

Economic accomplishments 39 23

Philanthropic responsibilityPhilanthropic ability 2 2Philanthropic activity 47 26

Philanthropic accomplishments 4 3

Administrative responsibilityAdministrative ability 3 3Administrative activity 20 15

Administrative accomplishments 2 2

Political responsibilityPolitical ability 12 10Political activity 11 6

Political accomplishments 0 0

Legal responsibilityLegal ability 1 1Legal activity 2 1

Legal accomplishments 0 0

Table 10 Average accuracy of the analyzed methods and weightedaverage TP FP Pre Re F-m and ROC for classification of financialperformance (safe zone grey zone and distress zone)

Model Acc() TP FP Precision Recall F-m ROC

NB 06925 01187 03333 03097 01187 00451 03358SVM 07022 01183 03333 03130 01183 00442 03298LR 07046 01183 03333 03138 01183 00442 03324


matter the companies are in an active or passive economicsituation they are focusing on using a great proportion ofpositive words to establish a company image and attractinvestors In the mesolevel analysis a comprehensive treenode structure was identified to discover the sentimentaltopics relevant to CSR In terms of outcome the CEO letterscontain a large quantity of ethical-related information es-pecially concentrating on the ethical activities that firmshave organized In the macrolevel the logistic regressionapproach achieves the best result in forecasting future fi-nancial performance which proves that there is a linearrelationship between the sentiment and economic perfor-mance In other words sentiment information in CEOletters can be regarded as a vital determinant for forecastingfinancial performance

0e distinct contribution of this paper is threefoldFirstly an advanced technique of sentiment analysis utilizingappraisal theory has been conducted which is potentiallyuseful for detecting the ldquoconcealedrdquo information in lettersSecondly a sentiment dictionary has been constructedsuccessfully and specifically for shareholdersrsquo letters whichcan significantly upgrade the accuracy of sentiment classi-fication Lastly this research can guide companies to furtherenhance the technique of releasing nonfinancial informationand display fundamental causes for diverse texts

0e current research has its own limitations UtilizingZ-score the quantitative assessment to define companiesrsquoeconomic situation may not be adequate In the Z-scoremodel the stock market value is only a static value at somepoint in time which cannot reflect a dynamic fluctuation Infact the need to interpret the firmsrsquo released information isto predict its future performance it is insignificant whicheconomic model was selected and the critical point is theinfluence of sentiment on the perception of the company byits stakeholders

In the future research it is possible to apply othereconomic models for predicting financial performanceEspecially with the popularity and high development ofsentiment analysis a great number of new approaches wouldbe probed by text mining to detect more concealed infor-mation for investorsrsquo decision-making and to be applied toother languages as well

Data Availability

All data models and code generated or used during thestudy appear in the submitted article

Conflicts of Interest

0e authors declare that they have no conflicts of interest

Acknowledgments

0is study was supported by the 2019 Project of the NationalSocial Science Foundation of China On the Overseas CSRDriving Forces and Influencing Mechanism for ChineseEnterprises (Grant no 19BGL116)

References

[1] P Bo and L Lee ldquoA sentimental education sentiment analysisusing subjectivity summarization based onminimum cutsrdquo inProceedings of the Meeting on Association for ComputationalLinguistics Philadelphia PA USA June 2004

[2] E Abrahamson and E Amir ldquo0e information content of thepresidentrsquos letter to shareholdersrdquo Journal of Business Financeamp Accounting vol 23 no 8 pp 1157ndash1182 1996

[3] P Hajek V Olej and R Myskova ldquoForecasting stock pricesusing sentiment information in annual reports - a neuralnetwork and support vector regression approachrdquo WseasTransactions on Business amp Economics vol 10 no 4pp 293ndash305 2013

[4] M Taboada J Brooke M Tofiloski K Voll and M StedeldquoLexicon-based methods for sentiment analysisrdquo Computa-tional Linguistics vol 37 no 2 pp 267ndash307 2011

[5] T Wilson J Wiebe and P Hoffmann ldquoRecognizing con-textual polarity an exploration of features for phrase-levelsentiment analysisrdquo Computational Linguistics vol 35 no 3pp 399ndash433 2009

[6] C J Anderson and G Imperia ldquo0e corporate annual reporta photo analysis of male and female portrayalsrdquo Journal ofBusiness Communication vol 29 no 2 pp 113ndash128 1992

[7] G F Kohut and A H Segars ldquo0e presidentrsquos letter tostockholders an examination of corporate communicationstrategyrdquo Journal of Business Communication vol 29 no 1pp 7ndash21 1992

[8] L Patelli and M Pedrini ldquoIs the optimism in CEOrsquos letters toshareholders sincere Impression management versus com-municative action during the economic crisisrdquo Journal ofBusiness Ethics vol 124 no 1 pp 19ndash34 2014

[9] P Bo and L Lee Opinion Mining and Sentiment AnalysisNow Publishers Inc Boston MA USA 2008

[10] K Dave S Lawrence and DM Pennock ldquoMining the peanutgallery opinion extraction and semantic classification ofproduct reviewsrdquo in Proceedings of the International Con-ference on World Wide Web Banff Canada May 2003

[11] A Kennedy and D Inkpen ldquoSentiment classification of moviereviews using contextual valence shiftersrdquo ComputationalIntelligence vol 22 no 2 pp 110ndash125 2006

[12] K S Srujan S S Nikhil H R Rao K Karthik B S Harishand H M K Kumar Classification of Amazon Book ReviewsBased on Sentiment Analysis Springer Singapore 2018

[13] J Feng C Gong X Li and R Y K Lau ldquoAutomatic approachof sentiment lexicon generation for mobile shopping reviewsrdquoWireless Communications and Mobile Computing vol 2018Article ID 9839432 13 pages 2018

[14] B Huang G Yu and H R Karimi ldquo0e finding and dynamicdetection of opinion leaders in social networkrdquoMathematicalProblems in Engineering vol 2014 no 1 7 pages 2014

[15] R Quratulain G Sayeed Sajjad and Haider ldquoLexicon-basedsentiment analysis of teachersrsquo evaluationrdquo Applied Compu-tational Intelligence and Soft Computing vol 2016 Article ID2385429 12 pages 2016

[16] M Stewart ldquo0e language of praise and criticism in a studentevaluation surveyrdquo Studies in Educational Evaluation vol 45pp 1ndash9 2015

[17] C L Chen C L Liu Y C Chang and H P Tsai ldquoOpinionmining for relating subjective expressions and annual earn-ings in US financial statementsrdquo Journal of InformationScience amp Engineering vol 29 no 4 pp 743ndash764 2012

[18] J L Hobson W J Mayew and M Venkatachalam ldquoDis-cussion of analyzing speech to detect financial misreportingrdquo


Journal of Accounting Research vol 50 no 2 pp 393ndash4002012

[19] C Huang X Yang X Yang and H Sheng ldquoAn empiricalstudy of the effect of investor sentiment on returns of differentindustriesrdquo Mathematical Problems in Engineering vol 2014Article ID 545723 11 pages 2014

[20] C Xie and Y Wang ldquoDoes online investor sentiment affectthe asset price movement Evidence from the Chinese stockmarketrdquo Mathematical Problems in Engineering vol 2017Article ID 2407086 11 pages 2017

[21] M Taboada ldquoSentiment analysis an overview from linguis-ticsrdquo Linguistics vol 2 no 1 2016

[22] C J Hutto and E Gilbert VADER A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text inProceedings of the Eighth International AAAI Conference onWeblogs and Social Media Ann Arbor MI USA 2014

[23] P D Turney ldquo0umbs up or thumbs down semantic ori-entation applied to unsupervised classification of reviewsrdquo inProceedings of Annual Meeting of the Association for Com-putational Linguistics pp 417ndash424 Philadelphia PA USAJuly 2002

[24] V Hatzivassiloglou and K R Mckeown ldquoPredicting the se-mantic orientation of adjectivesrdquo in Proceedings of the ACLpp 174ndash181 Autrans France 1997

[25] P D Turney and M L Littman ldquoMeasuring praise andcriticismrdquo Acm Transactions on Information Systems vol 21no 4 pp 315ndash346 2003

[26] M Taboada C Anthony and K Voll ldquoMethods for creatingsemantic orientation dictionariesrdquo in Proceedings of theLanguage Resources amp Evaluation Genoa Italy May 2006

[27] X Ding L Bing and P S Yu ldquoA holistic lexicon-basedapproach to opinion miningrdquo in Proceedings of the Inter-national Conference on Web Search amp Data Mining Cam-bridge UK 2008

[28] A Dey M Jenamani and J J 0akkar ldquoSenti-N-Gram ann-gram lexicon for sentiment analysisrdquo Expert Systems withApplications vol 103 Article ID S095741741830143X 2018

[29] J Brooke M Tofiloski and M Taboada ldquoCross-linguisticsentiment analysis from English to Spanishrdquo in Proceedings ofthe International Conference on Recent Advances in NaturalLanguage Processing Borovets Bulgaria September 2009

[30] B Pang L Lee and S Vaithyanathan ldquo0umbs up senti-ment classification using machine learning techniquesrdquo inProceedings of the Proceedings of the ACL-02 conference onEmpirical methods in natural language processing vol 10Philadelphia PA USA July 2002

[31] E Boiy and M-F Moens ldquoA machine learning approach tosentiment analysis in multilingual Web textsrdquo InformationRetrieval vol 12 no 5 pp 526ndash558 2009

[32] L I Jun and M Sun ldquoExperimental study on sentimentclassification of Chinese review using machine learningtechniquesrdquo in Proceedings of the IEEE International Con-ference on Natural Language Processing amp KnowledgeEngineering Beijing China 2007

[33] R F Bruce and J M Wiebe Recognizing Subjectivity A CaseStudy in Manual Tagging Cambridge University PressCambridge UK 1999

[34] Gamon and Michael ldquoSentiment classification on customerfeedback data noisy data large feature vectors and the role oflinguistic analysisrdquo in Proceedings of the International Con-ference on Computational Linguistics 2004

[35] C Yang X Tang Y C Wong and C P Wei ldquoUnderstandingonline consumer review opinion with sentiment analysis

using machine learningrdquo Pacific Asia Journal of the Associ-ation for Information Systems vol 2 no 3 2010

[36] C Troussas M Virvou K J Espinosa K Llaguno andJ Caro ldquoSentiment analysis of Facebook statuses using NaiveBayes classifier for language learningrdquo in Proceedings of theFourth International Conference on Information Mikroli-mano Greece 2013

[37] J Singh G Singh and R Singh ldquoOptimization of sentimentanalysis using machine learning classifiersrdquo Human-centricComputing and Information Sciences vol 7 no 1 p 32 2017

[38] M Rathi A Malik D Varshney R Sharma andS Mendiratta ldquoSentiment analysis of tweets using machinelearning approachrdquo in Proceedings of the 2018 Eleventh In-ternational Conference on Contemporary Computing (IC3)Noida India August 2018

[39] S Yuan H Wang and M Zhu ldquoSustainable strategy forcorporate governance based on the sentiment analysis of fi-nancial reports with CSRrdquo Financial Innovation vol 4 no 1p 2 2018

[40] A Aue and M Gamon ldquoCustomizing sentiment classifiers tonew domains a case studyrdquo in Proceedings of the InternationalConference on Recent Advances in Natural LanguageProcessing Varna Bulgaria September 2005

[41] S Gonzalez-Bailon and G Paltoglou ldquoSignals of publicopinion in online communicationrdquo 0e ANNALS of theAmerican Academy of Political and Social Science vol 659no 1 pp 95ndash107 2015

[42] T Loughran and B Mcdonald ldquoWhen is a liability not aliability Textual analysis dictionaries and 10-ksrdquo0e Journalof Finance vol 66 no 1 pp 35ndash65 2011

[43] R P Schumaker Y Zhang C-N Huang and H ChenldquoEvaluating sentiment in financial news articlesrdquo DecisionSupport Systems vol 53 no 3 2012

[44] P Hajek and V Olej ldquoEvaluating sentiment in annual reportsfor financial distress prediction using neural networks andsupport vector machinesrdquo Engineering Applications of NeuralNetworks Springer Berlin Germany 2013

[45] Y Q Xin P Srinivasan and H Yong ldquoSupervised learningmodels to predict firm performance with annual reports anempirical studyrdquo Journal of the American Society for Infor-mation Science amp Technology vol 65 no 2 pp 400ndash413 2014

[46] P Hajek V Olej and R Myskova ldquoForecasting corporatefinancial performance using sentiment in annual reports forstakeholdersrsquo decision-makingrdquo Technological and EconomicDevelopment of Economy vol 20 no 4 pp 721ndash738 2014

[47] S Tong W Jia P Zhang C Yu and D Wang ldquoPredictingstock price returns using microblog sentiment for Chinesestock marketrdquo in Proceedings of the 2017 3rd InternationalConference on Big Data Computing and Communications(BIGCOM) Chengdu China August 2017

[48] J Wiebe T Wilson and C Cardie ldquoAnnotating expressionsof opinions and emotions in languagerdquo Language Resourcesand Evaluation vol 39 no 2-3 pp 165ndash210 2005

[49] N Asher F Benamara and Y Y Mathieu ldquoAppraisal ofopinion expressions in discourserdquo Linguisticae InvestigationesRevue Internationale De Linguistique Franccedilaise Et De Lin-guistique Generale vol 32 no 2 pp 279ndash292 2009

[50] C S G Khoo A Nourbakhsh and J C Na ldquoSentimentanalysis of online news text a case study of appraisal theoryrdquoOnline Information Review vol 36 no 6 pp 680ndash686 2012

[51] M A K Halliday An Introduction to Functional GrammarOxford University Press Inc London UK 1994

[52] J R Martin and P R R White Language of EvaluationAppraisal in English Palgrave Macmillan London UK 2005


[53] S Eggins An Introduction to Systemic Functional LinguisticsPinter London UK 1994

[54] M Taboada and J Grieve ldquoAnalyzing appraisal automati-callyrdquo in Proceedings of the Aaai Spring Symposium on Ex-ploring Attitude amp Affect in Text 0eories amp Applicationspp 158ndash161 Stanford CA USA January 2004

[55] C Whitelaw N Garg and S Argamon ldquoUsing appraisalgroups for sentiment analysisrdquo in Proceedings of the ACMInternational Conference on Information amp KnowledgeManagement Bremen Germany October 2005

[56] J Read and J Carroll ldquoAnnotating expressions of appraisal inenglishrdquo in Proceedings of the Linguistic AnnotationWorkshop Prague Czech Republic June 2007

[57] S Argamon K Bloom A Esuli and F Sebastiani ldquoAuto-matically determining attitude type and force for sentimentanalysisrdquo in Proceedings of the Human Language TechnologyChallenges of the Information Society Poznan Poland Oc-tober 2009

[58] A Balahur J M Hermida A Montoyo and R MuntildeozEmotiNet A Knowledge Base for Emotion Detection in TextBuilt on the Appraisal 0eories Springer Berlin Heidelberg2011

[59] K Bloom ldquoSentiment analysis based on appraisal theory andfunctional local grammarsrdquo Dissertation thesis Illinois In-stitute of Technology Chicago IL USA 2011

[60] P Korenek and M Simko ldquoSentiment analysis on microblogutilizing appraisal theoryrdquo World Wide Web vol 17 no 4pp 847ndash867 2014

[61] X-L Cui and J S Shibamoto-Smith ldquoA corpus-based studyon Chinese sentiment parameters of Chinese sentimentdiscourserdquo Chinese Language and Discourse vol 5 no 2pp 185ndash210 2014

[62] A Edwardsjones Qualitative Data Analysis with NVIVOSage Publications vol 15 p 868 0ousand Oaks CA USA2010

[63] R Boyatzis Transforming Qualitative Information 0ematicAnalysis and Code Development Sage Publications 0ousandOaks CA USA 1998

[64] P Bazeley Qualitative Data Analysis with NVivo SageLondon UK 2007

[65] N V Chawla K W Bowyer L O Hall andW P Kegelmeyer ldquoSMOTE synthetic minority over-sam-pling techniquerdquo Journal of Artificial Intelligence Researchvol 16 no 1 pp 321ndash357 2002

[66] P Hajek ldquoCredit rating analysis using adaptive fuzzy rule-based systems an industry-specific approachrdquo Central Eu-ropean Journal of Operations Research vol 20 no 3pp 421ndash434 2012

[67] S L Cessie and J C V Houwelingen ldquoRidge estimators inlogistic regressionrdquo Applied Statistics vol 41 no 1pp 191ndash201 1992

[68] E Goffman Presentation of Self in Every Day Life vol 21pp 14-15 Doubleday New York NY USA 1959

[69] A B Carroll ldquo0e pyramid of corporate social responsibilitytoward the moral management of organizational stake-holdersrdquo Business Horizons vol 34 no 4 pp 39ndash48 1991

[70] D M W Powers ldquoEvaluation from precision recall andF-measure to ROC informedness markedness and correla-tionrdquo Journal of Machine Learning Technologies vol 1 no 2pp 37ndash63 2011


stakeholdersrsquo decision-making [3] China as one of thebiggest developing countries in the world has developedCSR rapidly especially the Chinese government has paidmuch more attention to sustainable development Howeverfewer researchers have exerted empirical evidence on thesentiment analysis of CSR in China particularly in the caseof information asymmetry

To our knowledge prior studies have proposed manifoldsentiment analysis techniques towards extracting usefulcontent from massive social media [4 5] some scholars havecategorized sentences into positivenegative [4] while otherresearchers have divided sentences into criticalemotional-based on researchersrsquo intuition 0e taxonomy resultshowever are inconsistent and incomparable as various self-designed frameworks have been applied in diverse studiesAlthough sentiment analysis of CSR or nonfinancial infor-mation appears to be a possible trend for predicting financialperformance and assisting investors to make future invest-ment decisions little research has been done in this field

As an attempt to make up the deficiency in the aboveresearch field this paper is divided into three dimensionsfirstly in the microlevel analysis it determines the sentimentpolarity and sentiment attributes in letters to shareholdersby utilizing appraisal theory secondly in the mesolevelanalysis it identifies the prominent CSR themes byemploying NVivo 12 plus software thirdly in themacrolevelanalysis it explores the sentiment elements so as to antic-ipate financial performance by using the expression ofZ-score

0e following parts of the paper are organized as followsin the part of literature review the related CSR CEO lettersprimary sentiment analysis approaches appraisal theoryand forecasting financial performance through textual in-formation will be reviewed the part of research method-ology will introduce the annotation study and theexperimental procedures in the part of experimental resultsand analysis the experimental results will be stated andanalyzed in the conclusion part the research results con-tribution limitations and suggestions will be provided

2 Literature Review

21 CEO Letters in CSR Report 0e CEO letter (hereaftershareholdersrsquo letter or letter to shareholders) is the mostwidely read part of the CSRR and a valuable communicationsource for stakeholdersrsquo decision-making Generally letterto shareholders is widely regarded as a promotional genrewhich tries to provide companiesrsquo subjective sentimentstandpoints and aims to portray a positive image [6] such aswhat stakeholders desired to know about identifying the lastyearrsquos performance tracking important events displayingcorporate social responsibility and predicting the futurevision of the companies 0us the letter stands in a vitalimportant position to deliver companyrsquos competitive ad-vantage [7] Although a number of scholars recognized thesignificance of CEO letters for instance Kohut and Segars[7] characterize the effectiveness of letter to shareholders andhow this information can benefit the company Patelli andPedrini [8] indicate that even under the tough economic

conditions companies still sincerely disclose nonfinancialinformation with stakeholders Surprisingly little researchhas focused on how companies try to construct their imagesand showed to readers through sentiment analysis

22 Sentiment Analysis In artificial intelligence (AI) textmining is an effective and efficient way to process a largenumber of textual contents through extracting the sentimentpolarity based on natural language processing In particularsentiment analysis has gradually become a popular tech-nique A considerable amount of literature has concentratedon analyzing documentsrsquo sentiment polarity [9] Automaticsentiment classification has been extensively applied to re-views of products [10] movies [11] books [12] shopping[13] social networks [14] and studentsrsquo course evaluationcomments [15 16] which is a common application ofclassifying positive or negative reviews Most recent workhas involved in extracting the textual information in thefinancial reports as the text may contain more informationthan the numerical part in an annual report [17 18] 0efinal results indicate that sentiment analysis is an importanttechnique for forecasting financial performance and thuscan be used to support the decision-making process ofpotential stakeholders [19 20] Researchers therefore haverealized the potential value of the textual informationanalysis of financial reports for predicting financial per-formance and managing risks

In fact the challenge of detecting sentiment from texthas been tackled from various perspectives Nonethelessprevious approaches to spot affect have been categorizedinto two main approaches lexicon-based approach andmachine-learning approach [21] which are explicitlydepicted in Figure 1

221 Lexicon-Based Approach Lexicon-based methods arerule-based requiring a predefined word list and polarity toidentify viewpoints and sentiments [4 22] containingcomputing orientation for a text from the semantic orien-tation of words or phrases [23] In other words when a newtext has been selected the words inside the text have tomatch with the words in the sentiment dictionary and thenvarious algorithms have been employed to aggregate values0e total aggregation of positive and negative values of thewords assembles the sentiment orientation of the whole textAn idealized operating mechanism is specified in Figure 2

Table 1 enumerates some related work in the lexicon-based approach for sentiment analysis and illustrates thevarious types of objectives along with the associated modelsused and the experimental results produced We employedprecision recall and F-measure as evaluation metrics theyare common in information retrieval and document clas-sification research Precision measures the number of cor-rectly classified items out of the total classified by aclassification technique and recall measures the amount ofcorrectly classified items of those manually classified as thegold standard 0e F-measure is the harmonic mean ofprecision and recall which offers a better measure than thearithmetic mean of precision and recall



















Maximumentropy


Trainingdataset

(+)

Trainingdataset

(ndash)

Characteristics

Variousalgorithms

Classifier

Newtext

Semanticorientation

Wordlist Rules

New text


rules

Semanticorientation
















sentimentsPMI-IR


travelreviews



Assigndocs

sentimentsPMI LSA



Taboada et al[26]



Ding et al [27]


OpinionObserver


amazoncom




category)

NA

NA 91 90 90



FBS NA 92 74 82

Taboada et al [4]




sentiments

SO-CAL Senti-N-Gram




Books 68 66 76 71Cars 84 76 100 86






Bruce andWiebe[33]


scale

LCA



486subjective

515objective

7217 NA NA NA

Pang et al[30]






using 4-pointscale




Yang et al[35]


Classassociation


Macroaveraged

NA




NA 6747 744 7226


Troussaset al [36]



90 users


1131 (+)

NA 77 68 721131 (minus)

Singhet al [37]


NB Productreview


456 831 NA 812J-48 967 877 NA 917

BFTree 892 883 NA 721OneR 90 913 NA 97



267 (+)7617 NA NA NA264 (minus)

168 (0)












Xin et al[45]




















Appraisal

Engagement

Attitude

Graduation

Monogloss

Heterogloss

Affect

Judgement

Appreciation

Force

Focus



















agreement

Attitudeengagement




attributes


Balahur et al[58]



emotions

ISEAR Corpus



appraisal)

Actor action object

NAEmotion









engagement8757



websites


parameters

Attitude

















classification








Microlevel analysis







p word1( 1113857p word2( 1113857 (1)







34 Data Analysis













Zi 12X1i + 14X2i + 33X3i + 06X4i + 10X5i (2)
















































5 Conclusions




















NB 06925 01187 03333 03097 01187 00451 03358SVM 07022 01183 03333 03130 01183 00442 03298LR 07046 01183 03333 03138 01183 00442 03324






Data Availability




Acknowledgments


References





























































































Maximumentropy


Trainingdataset

(+)

Trainingdataset

(ndash)

Characteristics

Variousalgorithms

Classifier

Newtext

Semanticorientation

Wordlist Rules

New text


rules

Semanticorientation
















sentimentsPMI-IR


travelreviews



Assigndocs

sentimentsPMI LSA



Taboada et al[26]



Ding et al [27]


OpinionObserver


amazoncom




category)

NA

NA 91 90 90



FBS NA 92 74 82

Taboada et al [4]




sentiments

SO-CAL Senti-N-Gram




Books 68 66 76 71Cars 84 76 100 86






Bruce andWiebe[33]


scale

LCA



486subjective

515objective

7217 NA NA NA

Pang et al[30]






using 4-pointscale




Yang et al[35]


Classassociation


Macroaveraged

NA




NA 6747 744 7226


Troussaset al [36]



90 users


1131 (+)

NA 77 68 721131 (minus)

Singhet al [37]


NB Productreview


456 831 NA 812J-48 967 877 NA 917

BFTree 892 883 NA 721OneR 90 913 NA 97



267 (+)7617 NA NA NA264 (minus)

168 (0)












Xin et al[45]




















Appraisal

Engagement

Attitude

Graduation

Monogloss

Heterogloss

Affect

Judgement

Appreciation

Force

Focus



















agreement

Attitudeengagement




attributes


Balahur et al[58]



emotions

ISEAR Corpus



appraisal)

Actor action object

NAEmotion









engagement8757



websites


parameters

Attitude

















classification








Microlevel analysis







p word1( 1113857p word2( 1113857 (1)







34 Data Analysis













Zi 12X1i + 14X2i + 33X3i + 06X4i + 10X5i (2)
















































5 Conclusions




















NB 06925 01187 03333 03097 01187 00451 03358SVM 07022 01183 03333 03130 01183 00442 03298LR 07046 01183 03333 03138 01183 00442 03324






Data Availability




Acknowledgments


References
























































































sentimentsPMI-IR


travelreviews



Assigndocs

sentimentsPMI LSA



Taboada et al[26]



Ding et al [27]


OpinionObserver


amazoncom




category)

NA

NA 91 90 90



FBS NA 92 74 82

Taboada et al [4]




sentiments

SO-CAL Senti-N-Gram




Books 68 66 76 71Cars 84 76 100 86






Bruce andWiebe[33]


scale

LCA



486subjective

515objective

7217 NA NA NA

Pang et al[30]






using 4-pointscale




Yang et al[35]


Classassociation


Macroaveraged

NA




NA 6747 744 7226


Troussaset al [36]



90 users


1131 (+)

NA 77 68 721131 (minus)

Singhet al [37]


NB Productreview


456 831 NA 812J-48 967 877 NA 917

BFTree 892 883 NA 721OneR 90 913 NA 97



267 (+)7617 NA NA NA264 (minus)

168 (0)












Xin et al[45]




















Appraisal

Engagement

Attitude

Graduation

Monogloss

Heterogloss

Affect

Judgement

Appreciation

Force

Focus



















agreement

Attitudeengagement




attributes


Balahur et al[58]



emotions

ISEAR Corpus



appraisal)

Actor action object

NAEmotion









engagement8757



websites


parameters

Attitude

















classification








Microlevel analysis







p word1( 1113857p word2( 1113857 (1)







34 Data Analysis













Zi 12X1i + 14X2i + 33X3i + 06X4i + 10X5i (2)
















































5 Conclusions




















NB 06925 01187 03333 03097 01187 00451 03358SVM 07022 01183 03333 03130 01183 00442 03298LR 07046 01183 03333 03138 01183 00442 03324






Data Availability




Acknowledgments


References















































































Bruce andWiebe[33]


scale

LCA



486subjective

515objective

7217 NA NA NA

Pang et al[30]






using 4-pointscale




Yang et al[35]


Classassociation


Macroaveraged

NA




NA 6747 744 7226


Troussaset al [36]



90 users


1131 (+)

NA 77 68 721131 (minus)

Singhet al [37]


NB Productreview


456 831 NA 812J-48 967 877 NA 917

BFTree 892 883 NA 721OneR 90 913 NA 97



267 (+)7617 NA NA NA264 (minus)

168 (0)












Xin et al[45]




















Appraisal

Engagement

Attitude

Graduation

Monogloss

Heterogloss

Affect

Judgement

Appreciation

Force

Focus



















agreement

Attitudeengagement




attributes


Balahur et al[58]



emotions

ISEAR Corpus



appraisal)

Actor action object

NAEmotion









engagement8757



websites


parameters

Attitude

















classification








Microlevel analysis







p word1( 1113857p word2( 1113857 (1)







34 Data Analysis













Zi 12X1i + 14X2i + 33X3i + 06X4i + 10X5i (2)
















































5 Conclusions




















NB 06925 01187 03333 03097 01187 00451 03358SVM 07022 01183 03333 03130 01183 00442 03298LR 07046 01183 03333 03138 01183 00442 03324






Data Availability




Acknowledgments


References























































































Appraisal

Engagement

Attitude

Graduation

Monogloss

Heterogloss

Affect

Judgement

Appreciation

Force

Focus



















agreement

Attitudeengagement




attributes


Balahur et al[58]



emotions

ISEAR Corpus



appraisal)

Actor action object

NAEmotion









engagement8757



websites


parameters

Attitude

















classification








Microlevel analysis







p word1( 1113857p word2( 1113857 (1)







34 Data Analysis













Zi 12X1i + 14X2i + 33X3i + 06X4i + 10X5i (2)
















































5 Conclusions




















NB 06925 01187 03333 03097 01187 00451 03358SVM 07022 01183 03333 03130 01183 00442 03298LR 07046 01183 03333 03138 01183 00442 03324






Data Availability




Acknowledgments


References




























































































agreement

Attitudeengagement




attributes


Balahur et al[58]



emotions

ISEAR Corpus



appraisal)

Actor action object

NAEmotion









engagement8757



websites


parameters

Attitude

















classification








Microlevel analysis







p word1( 1113857p word2( 1113857 (1)







34 Data Analysis













Zi 12X1i + 14X2i + 33X3i + 06X4i + 10X5i (2)
















































5 Conclusions




















NB 06925 01187 03333 03097 01187 00451 03358SVM 07022 01183 03333 03130 01183 00442 03298LR 07046 01183 03333 03138 01183 00442 03324






Data Availability




Acknowledgments


References

























































































classification








Microlevel analysis







p word1( 1113857p word2( 1113857 (1)







34 Data Analysis













Zi 12X1i + 14X2i + 33X3i + 06X4i + 10X5i (2)
















































5 Conclusions




















NB 06925 01187 03333 03097 01187 00451 03358SVM 07022 01183 03333 03130 01183 00442 03298LR 07046 01183 03333 03138 01183 00442 03324






Data Availability




Acknowledgments


References
















































































p word1( 1113857p word2( 1113857 (1)







34 Data Analysis













Zi 12X1i + 14X2i + 33X3i + 06X4i + 10X5i (2)
















































5 Conclusions




















NB 06925 01187 03333 03097 01187 00451 03358SVM 07022 01183 03333 03130 01183 00442 03298LR 07046 01183 03333 03138 01183 00442 03324






Data Availability




Acknowledgments


References



















































































Zi 12X1i + 14X2i + 33X3i + 06X4i + 10X5i (2)
















































5 Conclusions




















NB 06925 01187 03333 03097 01187 00451 03358SVM 07022 01183 03333 03130 01183 00442 03298LR 07046 01183 03333 03138 01183 00442 03324






Data Availability




Acknowledgments


References



















































































































5 Conclusions




















NB 06925 01187 03333 03097 01187 00451 03358SVM 07022 01183 03333 03130 01183 00442 03298LR 07046 01183 03333 03138 01183 00442 03324






Data Availability




Acknowledgments


References
















































































Data Availability




Acknowledgments


References










































































