affective ux: challenges in ux involving affective computing
Post on 15-Feb-2017
690 Views
Preview:
TRANSCRIPT
Design experience first; design software afterwards:
Affective UXChallenges in UX design involving affective computing
Alfredo SnchezUniversidad de las Amricas Puebla
01WORLD IA DAY 2016PRESENTATION TITLE HEREHEADER OPTIONSub head or short descriptionSome kind of explanatory text, reference or footnote can go here and wrap to two lines, if needed.Some kind of illustration or image?
2006-2014, Alfredo Snchez
http://www.udlap.mx
Architecturing happinessHow do we know users are happy?
Can interactive systems detect happiness (or the lack thereof)?
Can interactive systems intentionally produce happinesss?
Can technology at least not get in the way? 2006-2016, Alfredo Snchez
AgendaTerms of affectionAffective computingDetectionSynthesisMediationAffect in UXRecent developmentOpen issues
Terms of affection
2006-2014, Alfredo Snchez
EmotionA reaction experienced usually toward a specific object and typically accompanied by physiological and behavioral changes in the body
2006-2014, Alfredo Snchez
MoodA conscious state of mind (typically lasting longer than an emotion)
joydepression 2006-2014, Alfredo Snchez
SentimentAn attitude or opinion a state that results from attributes assigned to objects, which are grouped into categories
2006-2014, Alfredo Snchez
The affect spectrumemotionmoodsentiment
may triggermay prompt
Duration( - ) ( + ) 2006-2014, Alfredo Snchez
9
Affective computing
2006-2014, Alfredo Snchez(Computer sytems) related to, arising from, or deliberately influencing affective phenomena
MIT Affective Computing Group
Affective ComputingRosalind Picard1995
Affective computing (for engineers) The study and development of systems and devices that can recognize, interpret, process, simulate and mediate human affects.
2006-2016, Alfredo Snchez
Why affective computing?
2006-2014, Alfredo Snchez
Why affective computing?
2006-2014, Alfredo SnchezI choose not to feelhttp://animala.mx/noticias/2015/02/27/adios-mr-spock/
Why Affective ComputingReason is, and ought to be, only the slave of the passionsDavid Hume, 1711-1776
Without passions we would lack all motivation, all impulse or drive to act, or even to reason 2006-2014, Alfredo Snchez
Why affective computing?A love/hate relationship with the computer and ITs in general 2006-2014, Alfredo Snchez
https://www.youtube.com/watch?v=ZcZBTB-dkqQ
Why affective computing? (for engineers) Studies by many researchers show a vital role of emotion in many processes:
PerceptionDecision makingCreativityComprehensionMemorySocial interaction 2006-2014, Alfredo Snchez
http://www.bintegra.com/wp-content/uploads/2012/05/Fotolia_socila_users_S.jpg
Some application areasEducation intelligent tutoringHealth monitoring patientsSafety monitoring drivers / pilotsSecurity detecting liesE-commerce attitudes towards products or servicesVideo games producing engaging environments, enriching UXUser studies incorporating emotion into evaluation
2006-2014, Alfredo Snchez
Why affective computing (for our community)Emotion is the way we communicate our experience 2006-2014, Alfredo Snchez
Main aspects of affective computing
perception
synthesismediation 2006-2014, Alfredo Snchez
(of emotions, moods and sentiments)
19
Affect detection
2006-2014, Alfredo Snchez
Emotion detection: From facial cues 2006-2014, Alfredo Snchez
http://npx.autismspot.com/facial_expression.jpg
Emotion detection: From postural cues 2006-2014, Alfredo Snchez
http://instridebodywork.com/wp-content/uploads/2013/05/postures-of-emotion1.jpg
Emotion detection: From voice cues 2006-2014, Alfredo Snchez
http://pandodaily.files.wordpress.com/2013/07/whisper-in-ear.jpg?w=584&h=329http://gigaom2.files.wordpress.com/2013/07/emotion-speech.jpg?w=300&h=200&crop=1
Emotion detection: From usage patterns 2006-2014, Alfredo Snchez
http://web.deepnetsecurity.com/products2/typesense.asp
Keystroke dynamicsNavigation patternsEye tracking
Emotion detection: From brain activity 2006-2014, Alfredo Snchez
Emotiv EPOC sensorHonda BCI system
Emotion detection: From sensors of autonomic activitySkin conductance increases even with a little surprise or anxiety
Blood volume/pressure increases with anger or stress
Respiratory rhythm varies with fear, sadness, happiness, anxiety, fear
Muscle tension increases with surprise and anxiety
2006-2014, Alfredo Snchez
Affectiva sensor
NeXus sensor
Emotion detection: From sensors of autonomic activity 2006-2014, Alfredo Snchez
Know your inner selfW/Me
Hack happinessFeel (by Sentio)
Some general techniques to support affective computingFor detection, synthesis and mediation 2006-2014, Alfredo Snchez
Facial Action Coding System (FACS) 2006-2016, Alfredo Snchez
http://www.theroadtosiliconvalley.com/tag/tim-roth/
FACSFacial Action Coding System (FACS), very popularDefines Action Units (AUs), contraction or relaxation of one or more face muscles
2006-2014, Alfredo Snchez
FACS (for universal emotions)
http://www.paulekman.com/facs/ 2006-2014, Alfredo Snchez
Posture, motion and gesture analysis 2006-2014, Alfredo Snchez
Posture analysis
http://people.ict.usc.edu/~gratch/CSCI534/ACII-Handbook-GestureSyn.pdf 2006-2014, Alfredo Snchez
Posture and motion analysisE.g. arm movements and emotion: 2006-2014, Alfredo Snchez
http://people.ict.usc.edu/~gratch/CSCI534/ACII-Handbook-GestureSyn.pdf
Gesture analysis 2006-2014, Alfredo Snchez
http://people.ict.usc.edu/~gratch/CSCI534/ACII-Handbook-GestureSyn.pdf
MPEG-4MPEG-4 (ISO144996) defines 86 Face Animation Parameters (FAPs)196 Body Animation Parameters (BAPs) 2006-2014, Alfredo Snchez
Some existing toolsFaceReader, SHORE, eMotion, RealEyes 2006-2014, Alfredo Snchez
Some challengesPrototypical vs. non-prototypical emotions Pure vs. combined emotionsMultiple subjectsPartly covered faces / bodiesTracking face or body movementLighting conditions
Integrating techniquesSupporting application development 2006-2014, Alfredo Snchez
Emotion in UX evaluation
2006-2014, Alfredo Snchez
Evaluating user experience with novel technologies impact of emotion (Gestures + contextual menus - video)
2006-2014, Alfredo Snchez[Magallanes, Molina, Mndez& Snchez 2012]
Heuristic evaluationProblems observed in actual use 2006-2014, Alfredo SnchezIssues identified by expertsIDIssueOccurrencesP1Movement of objects using two fingers.13P2Size of the keyboard.8P3Finding help for the user.5P4Finding the toolbar associated with each object.4P5Navigating using the fan menu.3P6Navigating between nodes.3P7Identifying the icon for linking to new pages 2P8Incomplete or inaccessible menus.1
IDTaskIssue1Insert two images, change their position and size.P12Enter a text next to the images as a caption.P23Obtain the toolbar for an image.P3, P44Pop a menu and discover how it works.P55Build a hierarchy of linked workspaces.P6, P76Get the menu in different positions. Cause the menu to go off the screen.P87Insert two images, change their position and size.P18Enter a text next to the images as a caption.P2
Additional feedback on emotions, based on Product Emotion Measurement Tool (PrEmo)
[Desmet 2002]Participants were asked to select an animated representation in correspondence with their emotional reactions
2006-2014, Alfredo Snchez[In collaboration with Universidad del Cauca, Colombia]
Positive emotions in spite of problems! 2006-2016, Alfredo Snchez
Positive emotions associated with novelty?Negative emotions associated with known interface elements?[Magallanes, Molina, Mndez& Snchez 2012]
ImplicationsTechniques for evaluation of usability and user experience are needed that take affect into account
Framework developed at UniCauca, Colombia[Mndez, Collazos, Granollers,Snchez 2014; Mndez 2015] 2006-2014, Alfredo Snchez
EmoValA framework for emotional evaluation designed to enrich existing UX evaluation methodsThree major componentsEmoVal-C: Behavioral evaluation (facial expressions, gaze, posture, movements)EmoVal-S: Self-report of emotions while using a systemEmoVal-F: Physiological sensing, mainly brain activity during system usageSupporting website (http://artemisa.unicauca.edu.co/~ymendal/EmoVal) 2006-2016, Alfredo Snchez
[Mndez 2015]
EmoVal: Detailed support / facial expressions 2006-2016, Alfredo Snchez
[Mndez 2015]
EmoVal: Detailed support / posture and movement 2006-2016, Alfredo Snchez
[Mndez 2015]
Building capabilities for affect detection
2006-2014, Alfredo Snchez
Current scenario: Extending an app to include more techniques is not straightforward
Application domainTechniqueTechnologies 2006-2014, Alfredo Snchez
Goal: Integration platform
2006-2014, Alfredo Snchez[Snchez et al. 2015]
Architecture for implementation
2006-2014, Alfredo Snchez[Snchez et al. 2015]
Simple data model and messaging protocol
2006-2014, Alfredo Snchez[Snchez et al. 2015]
Prototypical apps: Kinect-based emotion detection 2006-2014, Alfredo Snchez
14 sorpresa[Snchez et al. 2015]
Prototypical apps: Kinect-based emotion detection 2006-2014, Alfredo Snchez[Snchez et al. 2015]
Prototype evaluation
2006-2014, Alfredo Snchez[Snchez et al. 2015]
Prototypical apps: Android-based emotion self reporter 2006-2014, Alfredo Snchez
[Snchez et al. 2015]
Prototype:Monitoringservices
2006-2014, Alfredo Snchez[Snchez et al. 2015]
BCIs for UXhttp://www.gadgetify.com/wp-content/uploads/2014/09/15/epoc.png
Experimenting with EPOC+ (video)
What about sentiment?
2006-2014, Alfredo Snchez
Sentiment analysisBy definition,The use of NLP and computational linguistics to identify attitudes in speakers or writers with respect to specific subjects 2006-2014, Alfredo Snchez
http://datameer2.datameer.com/blog/wp-content/uploads/2012/10/22nd.png
2006-2014, Alfredo Snchez
Applications of sentiment analysisDetecting / studying:
Responses to marketing/political campaigns User satisfactionBias in newsAppropriateness of contents for different audiences 2006-2014, Alfredo Snchez
Some techniquesMostly heuristic methodsSocial media analysis
Keyword-basedSemantic approachesAnnotated datasetsSupport Vector Machines (SVMs) for classification
2006-2014, Alfredo Snchez
Expression ArchitectureA flexible architecture enables Expression to listen from any data source and have an independent component-based front-end.Expression Sentiment Analysis Platform at UDLAPArchitecture[Cervantes, Gutirrez, Gutirrez, Castillo,Snchez, Wan 2015]
The front end of Expression is based on sentiments cards that represent topics of interest.Sentiment cardshttps://github.com/Innova4D/expression
Resaltar el concepto de REACTIVO
A Sentiment Card graphically summarizes opinion trends in social media
Sentiment Card
Aqu es importante mencionar que se pueden visualizar diferentes fuentes de datos, solamente debe conectarse al social data manager y es una de las cosas interesantes del sentiment card. Incluso los usuarios pueden escribir comentarios desde la aplicacin para contribuir.
Some challengesLanguage subtleties Irony, sarcasm, abbreviations, misspellings, idiomsRepresentativeness Dissatisfied users tweet more activelyContext dependent semantics SpamClassification beyond positive or negative 2006-2014, Alfredo Snchez
Affect mediation
2006-2014, Alfredo Snchez
Mediation: Conveying affect through ICTsVideoconferencing
Immersive environments
Asynchronous communicationEmail, forums, blogs 2006-2014, Alfredo Snchez
Some issues and challengesAudio and video quality over InternetQuality of multimedia experience(http://i-guide.org/)Metrics, assessment methods
Gaze awareness
3D modelingIssues shared with affect synthesis
2006-2014, Alfredo Snchez
Emotion over low data rates:Sending codes instead of full videoIn collaboration with University of Agder, Norwayhttp://www.paulekman.com/facs/
SensingEncodingTransmittingReconstructing[Lamei, Li, Snchez 2014]
Conveying mood
2006-2014, Alfredo Snchez
Instant messagingEmoticons work for emotions, instant reactions associated to specific statements
Mood is longer lasting, overarchingIdentifying mood can be a challenge, even for the person in that affective state 2006-2014, Alfredo Snchez
Russells Circumplex Model can help identify mood
Two-dimensional, valence-arousal model
Consistent spatial location in self-report experiments across individuals and across cultures/languages 2006-2014, Alfredo Snchez
[Russell 1983]
Extending Russells model - Intensity 2006-2014, Alfredo Snchez
Extended, discrete model 2006-2014, Alfredo SnchezUser testing showedaccuracy in affect identification
But using the model directlywould result in a cluttered interface
Gradual revelationannoyedhappysadcalm
Representative affectives states per quadrant (according to user testing)sleepyarousedupsetsatisfied 2006-2014, Alfredo Snchez
78
Emoticon design 2006-2014, Alfredo Snchez
Prototype (Russkman)
[Snchez et al. 2006] 2006-2014, Alfredo Snchez
80
Prototype
[Snchez et al. 2006] 2006-2014, Alfredo Snchez
81
Prototype
[Snchez et al. 2006] 2006-2014, Alfredo Snchez
82
EvaluationChat sessions among 2-3 subjectsHypothetical situations (upcoming exams, good grades, recent party, lost wallet )Subjects were able to specify their mood rapidlySubjects were able to identify mood of other users (high correlation)Persistent emoticons helped in keeping awareness of moods 2006-2014, Alfredo Snchez
Affect synthesis: Modeling and displaying emotion and mood
Synthesizing emotionsIf an emotion (or mood or sentiment) is recognized in the user, the computer (or an adaptive environment) could (should?) respond accordingly
Approaches to emotion synthesisReasoning about emotionsgiven a set of conditions, produce specific reactionsCharacters with personality (temperament, sentiment) 2006-2014, Alfredo Snchez
85
Emotion synthesis
Characters with animated facial expressions(facial cues for sensing also apply here)[CSLU] 2006-2014, Alfredo Snchez
86
Responding to emotion
Affective CompanionText from automatic speech recognizer and EmoVoice(for the user)Natural language understanding[Cavazza et al. 2010] 2006-2014, Alfredo Snchez
FACS applied to emotion synthesis
FACSGen [Roesch et al. 2010] 2006-2014, Alfredo Snchez
Affective animated avatarsDesign using popular tools and a reduced set of FACS AUs
Available at http://chopis.com
Testing in Mxico and China
[Snchez et al. 2014] 2006-2014, Alfredo Snchez
Some challengesPrototypical vs. non-prototypical emotions Pure vs. combined emotionsIntegrating facial, posture and voice expressions Offering service platforms for app construction 2006-2014, Alfredo Snchez
Perspectives of affective UXAffect is key in UX design and evaluation
Opportunities for research and development to improve detection, synthesis and mediation
Work needed in mood and sentiment analysis
Opportunities for inter-disciplinary collaboration 2006-2014, Alfredo Snchez
Affective computing becoming mainstream? 2006-2014, Alfredo Snchez
Pepper, SoftBanks affect-aware robot, 1,000 units sold out, December 2015, for about USD 1,800Involves facial recognition, video and audio processing, sensors, machine learning,
But waitForward-looking statements are by their nature subject to various risks and uncertainties, including, without limitation, a decline in general economic conditions, general market conditions, technological developments, changes in customer demand for products and services, increased competition, risks associated with international operations, and other important factors, each of which may cause actual results and future developments to differ materially from those expressed or implied in any forward-looking statement. [] Use of or reliance on the information in this presentation is at your own risk.
http://webcast.softbankmobile.co.jp/en/conference/20140605/index.html 2006-2014, Alfredo Snchez
Affective UXChallenges in UX design involving affective computing
Alfredo SnchezUniversidad de las Amricas Pueblahttp://ict.udlap.mx
01WORLD IA DAY 2016PRESENTATION TITLE HEREHEADER OPTIONSub head or short descriptionSome kind of explanatory text, reference or footnote can go here and wrap to two lines, if needed.Some kind of illustration or image?
ReferencesCervantes, O., Gutirrez, F., Gutirrez, E., Castillo E., Snchez, J. A., Wan, W. 2015. Expression: Understanding affective content in social media. 7th Latin American Conference on Human-computer Interaction (CLIHC 2015), Cordoba, Argentina, Nov. 18-23. Available from the ACM Digital Library (http://dl.acm.org/citation.cfm?doid=2824893.2824903).Magallanes, Y., Molina, A., Mndez, Y., Snchez, J. A. 2012. Combining gestures and graphical elements for collaboration using multitouch surfaces. Proceedings of the 22nd International Conference on Electronics, Communications and Computers (CONIELECOMP 2012, Feb.). IEEE Computer Society Press, 173-178.Magallanes, Y., Molina, A., Mndez, Y., Snchez, J. A. 2012. Towards an emotional validation of heuristic approaches for usability evaluation. Acta Universitaria 22, 3, 119-125. (http://www.actauniversitaria.ugto.mx/index.php/acta/article/view/351). Mndez, Y. A., Collazos, C. A., Snchez, J. A., Merchn Paredes, L. 2012. Criterios para la evaluacin de superficies multitctiles y sus aplicaciones. Lmpsakos 7 (Jan.), 64-68. (disponible en http://www.funlam.edu.co/revistas/index.php/lampsakos/article/view/847). Mndez, Y. A. 2015. Marco de referencia para la captura semi-invasiva y analisis de las emociones en el proceso de evaluacion con usuarios de sistemas interactivos. Tesis doctoral. Universidad del Cauca, Colombia. Disponible en http://www.unicauca.edu.co/doctoradoce/publicaciones/Monografia_Mendez.pdf
2006-2014, Alfredo Snchez
ReferencesSnchez, J. A., Hernndez, N. P., Penagos, J. C., Ostrvskaya, Y. 2006. Conveying mood and emotion in instant messaging by using a two-dimensional model for affective states. Proceedings of the Symposium on Human Factors in Computer Systems (IHC 2006, Natal, Brazil), 66-72. Available from the ACM Digital Library (http://dl.acm.org/citation.cfm?id=1298033&CFID=582956970&CFTOKEN=61490837). Mndez, Y. A., Collazos, C. A., Granollers, T., Snchez, J. A. 2014. Evaluating interactive systems from an emotional perspective. Revista Cientfica Guillermo de Ockham 12(1), 43-49. (disponible en http://www.redalyc.org/pdf/1053/105332478005.pdf). Snchez, J. A., Corts, X., Starostenko, O., Cervantes, O., Wan, W. 2015. An Extensible Platform for Seamless Integration and Management of Applications for Emotion Sensing and Interpretation. Journal of Ambient Intelligence and Smart Environments 7(1), 5-19. (available from http://iospress.metapress.com/content/y1g64422t78107v7). Starostenko, O., Corts, X., Snchez, J. A. 2015. Unobtrusive emotion sensing and interpretation in smart environments. Journal of Ambient Intelligence and Smart Environments 7, 59-83. (avalilable from http://iospress.metapress.com/content/w2681p2827l2u22p/).
2006-2014, Alfredo Snchez
top related