VICTEC and Empathic Interfaces

Ruth Aylett

Ana Paiva

V: Virtual

ICT: Information Communication

Technology

E: Empathic

C: Characters

What does VICTEC stand for?

Inspired by theatre

Use of performance to dramatise the problem Theatre in Education

Linked workshops But is not a cheap option

In every group some will be bullying others Interaction is limited in conventional performance

Forum theatre (Boal)

An interactive form of theatre Sections of audience ‘own’ a character The actor meets them ‘in role’ and discusses

actionsCan only act ‘in character’

Episodic performanceActor can halt an episode to ask for more advice

Hard to manage in school environment

Implementing a Virtual Forum Theatre

3D interactive graphics and virtual actors - synthetic characters

Individual interaction based on creating empathy with the characters

Computer game technology - but NOT a game! Provide a safe environment for children to explore

different perspectives on bullying behaviour.

What are Intelligent SyntheticCharacters?

A synthetic character is a digital or physical entity with personality that can interact with people in real time. Computer games characters are not very intelligent Do not remember anything about interaction Hard to believe in them as personalities Thus hard to create empathy with them

How realistic should they be?

We want them to be believable Is believability the same as realism? We believe in Mickey Mouse…

Characters that are NEARLY realistic may be ‘creepy’ We have high expectations And then they clash with them

Empathic Characters

Already much experience in animation

Mediating Empathy

(1) via the situation- the observer concludes the emotional state of the target from the situation the target is dealing with.

(2) via emotional expressions - the observer interprets the behaviour of the target: for example, when a target cries this means that he/she is probably sad.

Empathic Synthetic Agents

Users experience Empathic emotions

Characters show empathy towards users and other

characters

""empathic synthetic agent" as empathic synthetic agent" as "an agent that is able"an agent that is able

to perceive and have an to perceive and have an internal representation of the internal representation of the other agent's emotions and/or other agent's emotions and/or experiencing an appropriate experiencing an appropriate emotion as a consequenceemotion as a consequence

"empathy triggering synthetic agent" empathy triggering synthetic agent" as "an agent that is able to, by itsas "an agent that is able to, by itsbehaviour and features, allow the behaviour and features, allow the

users to build an empathicusers to build an empathicrelation with it"relation with it"

Mediation by situation

Depends on narrative engagement Collection of narratives from children

Using Kar2ouche story-boarding tool Observation of dramatic performance Scripting of dramas by drama students

Performances by Uni Herts students

Narratives should be: believable, dramatic, somewhat unpredictable

Mediation by emotional expression

Expressive behaviour linked to internal state modelling character’s affect

Facial Expression (expressive facial expressions that include emotions and communication signals)

Voice Body expression, posture and gestures

Designing Characters in VICTEC: Mediating empathy through facial expression

Designing Characters in VICTEC: Mediating empathy though body expression

Designing Characters in VICTEC: Mediating empathy through exaggeration

Through body expression and animation

Through posture and facial expression

Voice:

• Current Text-to-Speech technology is inadequate

•Real Voice must be used to emphasize emotional content

Engaging the user

The character must be believable No rerun - what happens counts Different scenarios and characters

Emerging story as in improvisational drama, not pre-scripted

Acting as an ‘invisible friend’. Tamagotchis seemed to evoke empathy Give advice about dealing with the situation and

see what happens

A Trailer

Not the finished software, but how we think it is going to look

Capturing affective responses

Creating an affective loop FearNot! Attempts to evoke empathy in the user How could one produce an empathic response to

the user? Requires ability to detect affective state of the

userNon-invasively

Interaction using Sentoy

Intelligent Physical InterfaceAims at making the bridge between the real and virtual worlds, extending and enhancing the child-system interactions

Hardware of SenToy

Magnet (on off)

Pressure sensors(4 positions each)

Accelerometer(x, y, z angles)

Embodiment

-Restrictions:

• Sensors size / number• Sensors capabilities• Battery vs Autonomy• Has to handled by children (not too heavy but also robust)

Validation of Embodiment

A Teddy, a puppet and a Barbie were tested - They had to express emotions and actions with

every dolls: - The subjects preferred a neutral expression (Barbie and Teddy)

- The doll had to be malleable (Teddy)

The teddy was easier to handle, more flexible and allowed the free expression of emotions / actions

Built-in patterns for emotion recognition

Gestures Emotions Reference

Put the toy's hands in front of it's eyes or moving the toy backwards

vigorouslyFEAR

According to Lazarus fear is associated with avoidance.

Moving the toy slightly backwards (squeezing it slightly)

DISGUSTAccording to Lazarus action tendencies for disgust include “move away”, nausea and even vomiting.

Swinging the toy (putting it dancing) and/or playing with its arms

HAPPINESSJoy is portrayed with open arms, movements such as clapping or rhythmic movement according to Darwin.

Bend down its neck or bend down all the toy's trunk

SADNESSAccording to Scherer sadness is expressed through slow movement inwards and head down.

To place its arms crosswise or shake the toy vigorously ANGER

According to Lazarus anger is associated with “tendency to attack”

Open its arms backwards inclining its trunk slightly backwards too

SURPRISE

According to Laban surprise is associated with attention and with a sudden event and inclination of trunk backwards. “A surprise person often raises his opened hands high above his head, or by bending his arms only to the level of his face”. 

Built-in patterns

Gestures Actions

Swing the legs forward and backward alternately or moving the toy forward with

small jumpsWALK

A bouncing jump in the vertical JUMP

To bend down and move the arm like it was picking something or moving the toy

like it was divingPICK

Software

The Data from the sensors is passed through radio into a receiver connected with the computer.

Then some signal processing occurs that filters the information, detects patterns and send it to a specific application integration module that provides the semantic.

SenToy Hardware

Signal ProcessingProcedure

SenToy’s Integration

Module

Sentoy in Victec

Roles of SenToy in the Interaction: Out-of-scenario interaction

As an embodiment of the user (toy as best friend). Number of toys is big restriction for evaluation

In-scenario As a form of transparently enriching the experience.

As inspirational input to the character (eg. cheering) As a non-intrusive recording device

Virtually friends

Event for 400 children in June 2004 at University of Hertfordshire: Virtually friends: Social role-play with robots and

agents Questionnaire and interaction with the software Activities with the robots used in research

Challenges - 1

There IS no easy answer No strategy always works Tension between realism and a straightforward message Do we have to present a happy ending?

Producing believable characters and varying scenarios Pushing the technology here Especially language for unscripted interaction

Challenges - 2 Cultural differences

There is no word exactly equivalent to ‘bullying’ in German or Portuguese

School uniform and other differences Where bullying takes place

No school dinners in GermanyChildren unaccompanied to school

Now extending work to China...