pickon: context-based emoticon recommendation service on...
TRANSCRIPT
PICKON: Context-Based Emoticon Recommendation Service on Mobile Instant messaging
Yujung Yun
Interaction Science
Sungkyunkwan University, Korea
Thesis Supervisor:
Jun-Dong Cho
Committee:
Dong-hee Shin, Sangwon-Lee
1
Contents
1. Introduction
1. Emoticon in CMC
2. Related studies
3. Research Question
2. Literature Review
1. Emoticon Classification
2. Technology Acceptance Model(TAM)
3. Methodology
1. Participants
2. Questionnaire design
3. Procedure
4. Results
1. Reliability & Validity
2. Result1, 2
5. Conclusion
1. Discussion
2. Limitations of the Research
3. Future Research
2
Introduction_
Emoticon in Computer-mediated Communication (CMC)
• Reduced nonverbal cues
• Impersonal
• Anonymity
3
Introduction_
Emoticon usage and sales statistics
Transfering emoticons through Line are
18 billions in one day,
10-15% of all messages in Kakaotalk are
Emoticon exchange (about 5 billions)
4
Attributes ChatInstant
Messaging(IM)Mobile Instant
Messaging(MIM)
Interactive mode
Nearly synchronous
Nearly synchronous Synchronous
Message type Medium Shorter Shorter
Communication frequency
VariedFrequently within a
time periodFrequent
Medium Desktop, laptop Desktop, laptop Smartphone.
Typical software example
sayclub MSN, Skype, Nateon Kakaotalk, Line
5
Related Studies
Study Tested medium
Research summary Dependent variable
Ptaszynski, M., et al (2010).
Web-basedprogram
Presented CAO, a system for affect analysis of emoticons in japaneseonline communication. (based on database of text emoticon)
Capability to sufficiently detect and extract any emoticon.
Reddy, V. R. (2004)
Web-basedprogram
Developed 3D character conversational agent that allow natural language conversation
Preference, intention to use, trust, presence
Nevlarouskaya, A. et al(2010)
Second life(3d virtual world)
Recognition of affective content conveyed through text, and automatic visualization of emotional expression of avatars
Usefulness, helpful inconveying mood/emotions,ease of use, improvecommunication asked in interview
Sánchez,J.A.,etal (2006)
IM Developed instant messageing and smile emoticons based on Russell’s circumplex model of affect.
Appropriateness ofconveyed emotion, qualitative result from Interview
6
How do you select emoticon?
7
purpose of study
Emoticon selection
Context-based emoticon recommendation system
Emotional Status
Examine effect of context-based emoticon recommendation system in mobile instant messaging (MIM). Specially, on the relationships between emoticon selection and usability factors.
8
Research Question
For mobile instant messaging user what is the relationship between
types of emoticon selection and perceived usefulness, perceived ease
of use, attitude, appropriateness, enjoyment, and intention to use.
• Independent variable:
• Dependent variable:
usefulness, ease of use, attitude, appropriateness, enjoyment, intention to use
Type of emoticon selection :
- Flicking UI, Presenting emoticons irregularly.- Touch & Drag to scrolling handle, Presenting
emoticons based on context(emotional state, text)
9
Literature Review_
Emoticon classification in selecting emoticon
• James Russell’s circumplex model
10
Theoretical framework 2
• Technology Acceptance Model(TAM)
[10] Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 319-340.[11] Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: a comparison of two theoretical models. Management science, 35(8), 982-1003.
11
Research model
Perceived Ease of use
Perceived Usefulness
Appropriateness
Intention to UseAttitude
Perceived Enjoyment
Emoticon selection
[28][49]
[28],[24]
[49], [51]
[12],[45]
[29][4] [2]
[18][10]
12
[12] Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1992). Extrinsic and intrinsic motivation to use computers in the workplace1. Journal of applied social psychology, 22(14), 1111-1132.[50] Yousafzai, S. Y., Foxall, G. R., & Pallister, J. G. (2007). Technology acceptance: a meta-analysis of the TAM: Part 1. Journal of Modelling in Management, 2(3), 251-280.
Hypothesis
H1a. Emoticon selection in prototype1 has a positive effect on perceived ease of use(ES>PEOU)
H1b. Emoticon selection in prototype2 has a positive effect on perceived ease of use(ES>PEOU)
H2a. Emoticon selection in prototype1 has a positive effect on perceived appropriateness(ES>APP)
H2b. Emoticon selection in prototype2 has a positive effect on perceived appropriateness(ES>APP)
H3a. Emoticon selection in prototype1 has a positive effect on perceived enjoyment. (PI>PE)
H3b. Emoticon selection in prototype2 has a positive effect on perceived enjoyment. (PI>PE)
H4a. Perceived ease of use while using prototype1 has a positive effect on perceived usefulness(PEOU>PU)
H4b. Perceived ease of use while using prototype2 has a positive effect on perceived usefulness(PEOU>PU)
H5a. Perceived Enjoyment while using prototype1 has a positive effect on Intention to use. (ES>IU)
H5b. Perceived Enjoyment while using prototype2 has a positive effect on Intention to use. (ES>IU)
H6a. Appropriateness while using prototype1 has a positive effect on attitude. (APP>ATT)
H6b. Appropriateness while using prototype2 has a positive effect on attitude. (APP>ATT)
H7a. Attitude while using prototype1 has a positive effect on Intention to use. (ATT>IU)
H7b. Attitude while using prototype2 has a positive effect on Intention to use. (ATT>IU)
13
Methodology
Online Survey (for 2days)
Measurement (Questionnaire)design
Stimulus material(Prototype application) design
14
Experiment Design & execution
What is PICKON?_App design process.
15
Reliability & Validity 1_Reliability
Construct Measured items No. of items Cronbach's α
Perceived Ease of use PEOU1, PEOU2, PEOU3, PEOU4 4 .904
Perceived Usefulness PU1, PU2, PU3, PU4 4 .885
Appropriateness APP1, APP2, APP3, APP4 4 .884
Attitude ATT2, ATT3 2 .846
Perceived Enjoyment PE1, PE2, PE3 3 .909
Intention to Use IU1, IU2 2 .929
ATT1, ATT4, IU were eliminated during reliability check
All the estimated Cronbach's α value exceeded the cut-off value (0.70)
.60 considered acceptable for exploratory purposes,
.70 considered adequate for confirmatory purposes,
.80 considered good for confirmatory purposes
16
Experiment Design & execution
11
A Indicator is divided into 3 part(negative, neutral, positive)
Color association on the side of slides(red, grey, and blue)
Sign for each emotion
Emoticons are displayed based on emotional state by adjusting handle
17
Which one is better for user?
Prototype1 Prototype218
Summary of difference in prototypes
Prototype1 Prototype2(PICKON)
Way to
Manipulate
Flicking (place a finger on the
screen and quickly swipe the
panel)
Touch & Drag to scrolling handle
among three point(negative,
neutral, positive)
Emoticon
Arrangement
Order
Random, partially arranged
according to character
Context-based(affective status
selected by user), arranged by
emoticon type(smile, emoticon
w/ face, sticker)
Way to Access Touch smile tap icon on
prototype1’s bottom
Touch a PICKON tap on
prototype1’s bottom
19
Participants
Measure items frequency percent
GenderFemale 100 50%
Male 100 50%
Age
19-22 58 29%
23-26 92 46%
>27 50 25%
19-22
23-26
>27
male
female
N= 200
20
Measurement
Criteria Questionnaire items
Perceived
usefulness
Q1. Using prototype1/ prototype2 (PICKON) would enable me to select emoticon more quickly.
Q2. Using prototype1/ prototype2 (PICKON) would improve selecting emoticon.
Q3. Using prototype1/ prototype2 (PICKON) would enhance effectiveness on selecting emoticon.
Q4. Using prototype1/ prototype2 (PICKON) would make it easier to select emoticon.
Perceived
Ease of Use
Q5. Learning to operate prototype1/ prototype2 (PICKON) would be easy for me
Q6. My interaction with prototype1/ prototype2 (PICKON) would be clear and understandable
Q7. It would be easy for me to become skillful at using prototype1/ prototype2 (PICKON)
Q8. It would find prototype1/ prototype2(PICKON) easy to use.
Perceived
Appropriateness
Q9. (interface)The interface is appropriate to find certain emoticon right now.
Q10. (emotion) The emoticons are appropriate for emotion I intended right now.
Q11. (system) The way of displaying recommend emoticons list is appropriate(make sense).
Q12.(emoticon) The recommended emoticons are appropriate to me.
Perceived
Attitude
Q13. Using prototype1/prototype2 (PICKON) wise idea.
Q14. I like the idea of using the prototype1/ prototype2 (PICKON).
Intention to UseQ15. Assuming I had access to the prototype1/ prototype2 (PICKON), I intend to use it.
Q16. Given that I had access to the system, I predict that I would use it.
Perceived
Enjoyment
Q17. Using prototype1/ prototype2 (PICKON) is fun
Q18. prototype1/ prototype2 (PICKON) is pleasant
Q19. I find prototype1/ prototype2 (PICKON) to be enjoyable21
Reliability & Validity_results of Confirmatory Factor Analysis(CFA)
Fit index Recommended Value Structural Model
𝑥2/df <3.00 2.384
Tucker-lewis index (TLI) >.90 .929
Comparative fit index (CFI) >.90 .934
RMSEA <.08 .083
The structural model had acceptable fit indices.
The Root Mean Square Error of Approximation (RMSEA):When numerical value of RMSEA is between 0.05-0.08; better modelWhen numerical value of RMSEA is between 0.08-0.10; acceptable modelIf greater than 0.10; it indicates that the model don’t fit well
22
Doll, W. J., Hendrickson, A., & Deng, X. (1998). Using Davis's Perceived Usefulness and Ease‐of‐use Instruments for Decision Making: A Confirmatory and Multigroup Invariance Analysis. Decision Sciences, 29(4), 839-869.
Reliability & Validity_Descriptive analysis and Discriminant validity of the measurement
PEOU PU APP ATT PE IU AVEComposite
reliability
PEOU 1 0.587 0.850
PU .60 1 0.520 0.810
APP .75 .84 1 0.531 0.819
ATT .58 .81 .83 1 0.649 0.786
PE .60 .75 .72 .74 1 0.655 0.851
IU .60 .81 .73 .82 .77 1 0.774 0.873
Average Shared squared variance(AVE): >.50Composite reliability: >.70
- 타당도가 있다없다가 아니라 적절하다 높다 낮다로 표현-이 검사는 무엇을 측정하는데 타당하다고 표현.
AVE values are greater than .50 and less than composite reliability: adequate level of convergent validity
23
Result 1
• NS : non-significant• 0.01< p < 0.05: code as one*• 0.001< p <0.01: code as two*(**); usual cutoff p<0.05• P <0.001, code as *** 24
Result 1_Summary of hypothesis tests
Hypotheses 𝛽 SE t p-value Supported
H1 Emoticon selection ---> PEOU -0.083 0.123 -1.36 0.174 No
H2 Emoticon selection ---> APP -0.052 0.143 -0.699 0.485 No
H3 Emoticon selection ---> PE -0.06 0.181 -0.809 0.419 No
H4 APP ---> ATT 0.828 0.079 12.305 *** Yes
H5 PEOU ---> PU 1.112 0.125 7.52 *** Yes
H6 ATT ---> IU 0.452 0.112 4.079 *** Yes
H7 PU ---> IU 0.331 0.146 3.033 0.002** Yes
H8 PE ---> IU 0.318 0.047 6.319 *** Yes
P-value:p-values quantify the probability of observing Concomitant events under the null hypothesis.
𝛽; Beta(standardized regression coefficient): The greater value affect more to the dependent variable
25
Result 2_Summary of hypothesis tests
Hypotheses 𝛽 SE t p-value Supported
Prototype1
H4_a APP → ATT 0.646 0.101 6.745 *** Yes
H5_a PEOU → PU 0.797 0.079 9.084 *** Yes
H6_a ATT → IU 0.415 0.103 0.103 *** Yes
H7_a PU → IU 0.555 0.094 0.094 *** Yes
H8_a PE → IU -0.005 0.062 0.062 0.943 No
Prototype2
H4_b APP → ATT 0.874 0.104 0.104 *** Yes
H5_b PEOU → PU 0.684 0.13 0.13 *** Yes
H6_b ATT → IU 0.334 0.111 0.111 *** Yes
H7_b PU → IU 0.256 0.157 0.157 *** Yes
H8_b PE → IU 0.689 0.062 0.062 *** Yes
Prototype1, H8_a(PE-IU) was not supported among hypotheses. However, H4_a, H5_a, H6, and H7_a, were supported.
Prototype2, All the hypotheses were supported. 26
Result 2_Critical ratios for differences between parameters
Prototype1
APP → ATT ATT → IU PU → IU PE → IU PEOU → PU
Prototype2
APP → ATT 2.064 3.996 3.639 8.153 1.971
ATT → IU -2.273 -0.369 -0.886 2.718 -2.799
PU → IU -1.6 -0.072 -0.474 2.294 -1.931
PE → IU -0.594 1.783 1.247 7.008 -1.105
PEOU → PU -0.367 1.352 0.939 4.334 -0.665
• c.r(Cirtical Ratio) of ATT-->IU, PU-->IU, PEOU-->PU , had no statistically significant differences between parameters in path coefficient(smaller than t(±1.96))
• c.r of APP-->ATT, PE-->IU were considered that there were statistically significant difference in path coefficient in parameters (bigger than t(±1.96))
• Prototype2 affected more than Prototype1 in H4 (APP-->ATT) and H8 (PE-->IU)1127
• Confirmed TAM path correlation toward emoticon selection
• Explored determinants of intention to use,
- PEOU> PU, APP>ATT,PE
• PU was affected most toward IU in Prototype1, PE was the most important determinant of IU in Prototype2.
• Contributed to mobile instant messaging design by providing an understanding of which emoticon selection style may be good in what aspects.
Conclusion_Discussion
28
• Conclusion drawn are based on specific emoticon selection technology.(hard to generalize)
• Lack of existing studies on the use of emoticon in ‘mobile instant messaging’(difficult to interpret results)
• Participants who well known using emoticon
(need controlling ages)
Conclusion_ Limitation of Research
29
Conclusion_ Future Research
Emoticon selection
Context-based emoticon recommendation system
Text recognition
Spatio-temporal property
Emotional Status
30
Conclusion_ Future Research
• Quantitative study + Qualitative data
(To obtain detailed opinion from the users)
• Design new set of emoticons for emotion based emoticon recommendation
(referred to Ekman’s study)
• Improve prototype fuction (automation of inputting context)
31
References• [1] Adams, D. A., Nelson, R. R., & Todd, P. A. (1992). Perceived usefulness, ease of use, and usage of information
technology: a replication. MIS quarterly, 227-247.
• [2] Ajzen, I. (2001). Nature and operation of attitudes. Annual review of psychology, 52(1), 27-58.
• [3] Azuma, J., & Ebner, M. (2008, June). A Stylistic Analysis of Graphic Emoticons: Can they be Candidates for a Universal Visual Language of the Future?. In World Conference on Educational Multimedia, Hypermedia and Telecommunications (Vol. 2008, No. 1, pp. 972-979).
• [4] Babin, B. J., Chebat, J. C., & Michon, R. (2004). Perceived appropriateness and its effect on quality, affect and behavior. Journal of Retailing and Consumer Services, 11(5), 287-298.
• [5] Baron, N. S. (2004). See you online. Gender issues in college student use of Instant Messaging. Journal of Language and Social Psychology, 23(4), 397-423.
• [6] Bhattacherjee, A. (2001). Understanding information systems continuance: an expectation-confirmation model. MIS quarterly, 351-370.
• [7] Cha, Y. J. (2007). Identifying cross-cultural differences of emoticons in computer-mediated communication: A comparison of North American (US) and South Korean emotional responses to emoticons (Doctoral dissertation, faculty of the School of Informatics in partial fulfillment of the requirements for the degree Master of Science in Human-Computer Interaction, Indiana University).
• [8] Cyr, D., Head, M., & Ivanov, A. (2006). Design aesthetics leading to m-loyalty in mobile commerce. Information & Management, 43(8), 950-963.
• [9] Danesi, M. (2009). Dictionary of media and communications. New York & London: M. E. Sharpe, Inc
• [10] Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 319-340.
• [11] Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: a comparison of two theoretical models. Management science, 35(8), 982-1003.
• [12] Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1992). Extrinsic and intrinsic motivation to use computers in the workplace1. Journal of applied social psychology, 22(14), 1111-1132.
• [13] Derks, D., Bos, A. E., & Grumbkow, J. V. (2007). Emoticons and social interaction on the Internet: the importance of social context. Computers in Human Behavior, 23(1), 842-849.
32
[14] Dey, A. K. (2001). Understanding and using context. Personal and ubiquitous computing, 5(1), 4-7.[15] Dresner, E., & Herring, S. C. (2010). Functions of the nonverbal in CMC: Emoticons and illocutionary force. Communication Theory, 20(3), 249-268.[16] Double, K. (2007). Computer-mediated communication and Internet language: An exploration of emoticons as medium-specific elements of a developing Internet language. New Media, 2, 17-21.[17] Ekman, P., & Friesen, W. V. (2003). Unmasking the face: A guide to recognizing emotions from facial clues. Ishk.[18] Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention and behavior: An introduction to theory and research.[19] Gefen, D., & Straub, D. W. (2000). The relative importance of perceived ease of use in IS adoption: a study of e-commerce adoption. Journal of the Association for Information Systems, 1(1), 8.[20] Holbrook, M. B., Chestnut, R. W., Oliva, T. A., & Greenleaf, E. A. (1984). Play as a consumption experience: The roles of emotions, performance, and personality in the enjoyment of games. Journal of Consumer Research, 728-739.[21] Huang, A. H., Yen, D. C., & Zhang, X. (2008). Exploring the potential effects of emoticons. Information & Management, 45(7), 466-473.[22] Huffaker, D., & Calvert, S. (2005). Gender, identity, and language use in teenage blogs.Journal of Computer-Mediated Communication, 10, article 1. Retrieved May 5, 2008,from http://jcmc.indiana.edu/vol10/issue2/huffaker.html.[23] Kiesler, S., Siegel, J., & McGuire, T. W. (1984). Social psychological aspects of computer-mediated communication. American psychologist, 39(10), 1123.[24] Kim, K. J., & Sundar, S. S. (2014). Does Screen Size Matter for Smartphones? Utilitarian and Hedonic Effects of Screen Size on Smartphone Adoption. Cyberpsychology, Behavior, and Social Networking.[25] Lancaster, S., Yen, D. C., Huang, A. H., & Hung, S. Y. (2007). The selection of instant messaging or e-mail: College students' perspective for computer communication. Information management & computer security, 15(1), 5-22.[26] Lee, M., & Cho, J. D. (2014, September). Logmusic: context-based social music recommendation service on mobile device. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (pp. 95-98). ACM.[27] Lim, S. H., Kim, D., & Watts, S. (2011). An Empirical Study of the Usage Performance of Mobile Emoticons. Journal of Information Technology Applications & Management, 18(4), 21-40.[28] Lu, J., Liu, C., Yu, C. S., & Wang, K. (2008). Determinants of accepting wireless mobile data services in China. Information & Management, 45(1), 52-64.[29] MacInnis, D. J., & Park, C. W. (1991). The differential role of characteristics of music on high-and low-involvement consumers' processing of ads. Journal of consumer Research, 161-173.[30] Malone, T. (1982). What makes computer games fun? (Vol. 13, No. 2-3, p. 143). ACM.[31] Moon, J. W., & Kim, Y. G. (2001). Extending the TAM for a World-Wide-Web context. Information & Management, 38(4), 217-230.[33] Neviarouskaya, A., Prendinger, H., & Ishizuka, M. (2007, January). An expressive avatar for instant messaging endowed with emotional intelligence. In Intelligent Virtual Agents (pp. 395-396). Springer Berlin Heidelberg.[34] Neviarouskaya, A., Prendinger, H., & Ishizuka, M. (2010). User study on AffectIM, an avatar-based Instant Messaging system employing rule-based affect sensing from text. International journal of human-computer studies, 68(7), 432-450.
33
[35] Petrantonakis, P. C., & Hadjileontiadis, L. J. (2010). Emotion recognition from EEG using higher order crossings. Information Technology in Biomedicine, IEEE Transactions on, 14(2), 186-197.[36] Phillips, L. W., & Sternthal, B. (1977). Age differences in information processing: a perspective on the aged consumer. Journal of Marketing Research, 444-457.[37] Plutchik, R. (2001). The nature of emotions. American Scientist, 89(4), 344-350.[39] Reddy, V. R. (2004). Communicating emotions to virtual agents: An emoticon-based approach (Doctoral dissertation, Master’s thesis, International School of New Media, Universitat zu Lubeck, Master of Science in Digital Media (June 2004)).[40] Riva, G. (2002). The sociocognitive psychology of computer-mediated communication: The present and future of technology-based interactions. Cyberpsychology & behavior, 5(6), 581-598.[41] Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological review, 110(1), 145.[42] Sánchez, J. A., Kirschning, I., Palacio, J. C., & Ostróvskaya, Y. (2005, October). Towards mood-oriented interfaces for synchronous interaction. In Proceedings of the 2005 Latin American conference on Human-computer interaction (pp. 1-7). ACM.[43] Sánchez, J. A., Hernández, N. P., Penagos, J. C., & Ostróvskaya, Y. (2006, November). Conveying mood and emotion in instant messaging by using a two-dimensional model for affective states. In Proceedings of VII Brazilian symposium on Human factors in computing systems (pp. 66-72). ACM.[44] Sundar, S. S. (2008). The MAIN model: A heuristic approach to understanding technology effects on credibility. Digital media, youth, and credibility, 73-100.[45] Thelwall, M., Wilkinson, D., & Uppal, S. (2010). Data mining emotion in social network communication: Gender differences in MySpace. Journal of the American Society for Information Science and Technology, 61(1), 190-199.[46] Utz, S. (2000). Social information processing in MUDs: The development of friendships in virtual worlds. Journal of Online Behavior, 1 (1). Retrieved September 15, 2009, from http://www.behavior.net/JOB/v1n1/utz.html[48] Walther, J. B. (1996). Computer-mediated communication impersonal, interpersonal, and hyperpersonal interaction. Communication research, 23(1), 3-43.[49_1]Venkatesh, V. (2000). Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model. Information systems research, 11(4), 342-365.[48] Walther, J. B., & D’Addario, K. P. (2001). The impacts of emoticons on message interpretation in computer-mediated communication. Social science computer review, 19(3), 324-347.[49] Wu, L. L., Lu, H. P., & Tao, Y. H. (2010). The effect of emoticons in simplex and complex task-oriented communication: An empirical study of instant messaging. Computers in Human Behavior, 26(5), 889-895.[50] Yigit, O. T. (2005). Emoticon usage in task-oriented and socio-emotional contexts in online discussion boards.[51] Yousafzai, S. Y., Foxall, G. R., & Pallister, J. G. (2007). Technology acceptance: a meta-analysis of the TAM: Part 1. Journal of Modelling in Management, 2(3), 251-280. [52] Zarmpou, T., Saprikis, V., Markos, A., & Vlachopoulou, M. (2012). Modeling users’ acceptance of mobile services. Electronic Commerce Research, 12(2), 225-248.[53] Zimmermann, A., Lorenz, A., & Oppermann, R. (2007). An operational definition of context. In Modeling and using context (pp. 558-571). Springer Berlin Heidelberg.[54] Zimmerman, D. P. (1987). Effects of computer conferencing on the language use of emotionally disturbed adolescents. Behavior Research Methods, Instruments, & Computers, 19(2), 224-230.
34
Thank you!
35
Q&A
36