An Affective Agent Playing Tic-Tac-Toe as Part of a

Healing Environment

Matthijs Pontier & Ghazanfar F. Siddiqui

Overview of this presentation

● Background● About the models incorporated in the agent● Application● Conclusions & Future research

Background

● Growing belief that not only healthcare, but also environment affects healing process

● Previous research:

Physical and social changes in environment can positively influence healing process Enhance recovery Reduce length of stay in hospital

Background

● Previous research: Poor design Negative effects:

● Anxiety● Delirium● Elevated blood pressure levels● Increased intake pain drugs

Unfamiliar environments Psychological stress that can negatively affect healing / wellness.

Background

● Loneliness is important contributing factor to Psychosocial stress

● Loneliness is common problem in long-term care facilities

● Previous research: Animal-shaped toys can be useful as a tool for

occupational therapy Paro is used for therapy at eldercare facilities, and

improves users’ moods, activeness and communicativeness

Animal-assisted therapy with AIBO robot dog helped just as good as with living dog

Background

AIBO Paro

Background

● Banks et al.: AIBO dog was not used to its full capacity, and results might be further enhanced

● If artificial companion demonstrates human-like emotional behavior, this might increase ability to reduce loneliness patients

● We present affective virtual agent that can play tic-tac-toe as a pilot application for this purpose Equipped with Silicon Coppélia: Integration of three affect-

related models. Therefore able to show human-like emotional behavior Simulated emotions: Joy, Distress, Hope, Fear, Surprise

The Models Incorporated in the Agent

● I-PEFiCADM: Model to let agents trade rational for affective choices, based on theory Frijda

The Models Incorporated in the Agent● EMA: Model to let agents exhibit and cope with

(negative) affect based on Smith & Lazarus’ theory

The Models Incorporated in the Agent

● CoMERG: Can simulate different emotion regulation strategies explained by Gross

Combined model: Silicon Coppélia

● Simulate affective decision making process: decisions based not only on rationality, but also on affective influences

● Simulate emotions based on beliefs about world-states, and how these affect goals

● Emotion regulation strategies can be applied to regulate (simulated) emotions

ENCODE COMPARE RESPOND

SITUATIONS FEATURES APPRAISAL APPRAISAL RESPONSEDOMAINS PROCESS COVERT OVERT

Inte

grat

ed m

ode

l

currentvalence

weightedfeatures

ethicsaffordances

aestheticsepistemics

currentstate

predicates

futurestate

predicates

affe

ctiv

e s

tate

s

“emotions”“mood”

Features are matched against goals, concerns, beliefs, intentions, etc. of self and others (allows taking perspectives)

involvementdistance

satisfa

ction

situationmodification

futurevalence

relevance

affe

ctive de

cision m

akin

g

situation selection

po

sitive ap

proach

neg

ative

app

roach

chan

ge

avoid

experiential behavioral

physiological(e.g., arousal, heart-beat, ……………., sweat, blush)

response modulation

cognitive change

attentional deployment

similarity

Relevance and valence (gray area) are the variables in the appraisal frames

use intentions

app

rais

al

fram

es

User feature

Affords Robot goal state

Positive / Negative

Facilitates / Inhibits

Desired / Undesired

Determining which action to take

● Agent has beliefs that actions facilitate goals (winning / losing) based on status tic-tac-toe game

● Agent can have desire to win, but also desire to lose● For action that facilitates desired goal, a high Action_Tendency

is generated● Positivity action based on belief action facilitates losing● Negativity action based on belief action facilitates winning● ExpectedSatisfaction(Action) =

wat * Action_Tendency + wpos * (1 - abs(positivity – biasI * Involvement)) + wneg * (1 - abs(negativity – biasD * Distance))

Calculating Hope and Fear

● Hope and fear are based on perceived likelihood [0, 1] of winning and losing (which are based on status tic-tac-toe game)

● Higher hope_for_goal if likelihood close to ‘f’● ‘f’ is set to 0.5● ‘Hope’ for negative ambition_level is fear

IF f >= likelihood THEN hope_for_goal =

-0,25 * ( cos( 1 / f * p * likelihood(goal) ) -1,5) * ambition_level(goal)

IF f < likelihood THEN hope_for_goal =

-0.25 * ( cos( 1 / (1-f) * p * (1-likelihood(goal)) ) -1.5) * ambition_level(goal)

Calculating Hope and Fear

● The following algorithm is performed:1. Sort hope_for_goal values in two lists: [01] and [0-1]

2. Start with 0 and take the mean of the value you have and the next value in the list. Continue until the list is finished. Do this for both the negative and the positive list.

3. Hope = Outcome positive list.

Fear = abs(Outcome negative list).

● This way, multiple hope_for_goals increase hope. ● However, with the more hope_for_goals there are, the

less impact each extra hope_for_goal has on the level of hope

Calculating Joy and Distress

● Joy and Distress are based on (not) reaching (un)desired goal-states (i.e., winning / losing):

● Reaching desired goal increases joy, decreases distress● Reaching undesired goal decreases joy, increases distress● Higher ambition level goal Bigger impact on emotions

IF ambition_level(goal) >= 0 THEN:new_joy = old_joy + mf_joy * ambition_level(goal) * (1-old_joy)

new_distress = old_distress + mf_distress * -ambition_level(goal) * old_distress

IF ambition_level(goal) < 0 THEN:new_joy = old_joy + mf_joy * ambition_level(goal) * old_joynew_distress = old_distress + mf_distress * -ambition_level(goal) * (1-

old_distress) 

Calculating Surprise

● Agent generates expectations about move human or outcome game, based on status tic-tac-toe game

● Surprise = 1 – likelihood(move)● Surprise = 1 – likelihood(outcome_game)

Emotion expression by agent

● All 5 emotions are simulated in parallel, and are shown by the facial expression of the agent

● After each human move, the agent speaks a message depending on the level of surprise

● Satisfaction is calculated based on the outcome of the game and amount of money played for

● Relief = Satisfaction * Surprise● After finishing game, agent speaks message

based on Satisfaction and Relief

Agent 1: The agent tries to win

The ambition level for winning of the agent is set to 1

The ambition level for losing is set to -1

The weight of the affective influences in the decision making process is set to 0.

The agent always tries to win

Winning increases joy, and decreases distress

Losing decreases joy, and increases distress

The agent gets hope from a high likelihood to win

The agent gets fear from a high likelihood to lose

The agent becomes surprised from unexpected moves

Agent 2: The agent deliberately tries to lose

The agent has an ambition level for winning of -1

The agent has an ambition level for losing of 1

The weight of the affective influences in the decision making process is set to 0.

The agent always tries to lose.

Losing increases joy and decreases distress

Winning decreases joy and increases distress

The agent gets hope from a high likelihood to lose

The agent gets fear from a high likelihood to win

Agent 3: The agent decides whether it wants to win based on its money

The agent determines its ambition level for winning and losing on whether it has more money than the user or not.

If the user has more money than the agent, it will deliberately try to lose, but otherwise it will try to win.

Importance is determined by dividing the amount played for by the amount the agent has left

Importance determines the impact on joy and distress

If the outcome of the game is desired (agent wants to win, and wins, or wants to lose, and loses), joy is increased and distress is decreased

If the outcome of the game is undesired, joy is decreased and distress is increased

Agent 4: The agent is too involved with the user to win

The agent bases its decisions only on emotions

The agent is designed to be very involved with the user.

The agent performs actions that facilitate losing.

The agent has the ambition to win, but because it bases its decisions on emotions it tries to lose anyway

Losing decreases joy, and increases distress

Winning increases joy, and decreases distress

Agent 5: The agent is balanced, and wins sometimes, and loses sometimes

The agent bases its decisions partly on emotions, and partly on rationality.

The agent has always more ambition to win than to lose. How big this difference in ambition is, depends on the amount of money that is played for.

When playing for a high amount, the agent tries to win

When playing for a low amount, the agent is too involved with the human user to try to win

Losing decreases joy, and increases distress

Winning increases joy, and decreases distress

Conclusions

● We presented an agent equipped with a combination of emotion models, that can play tic-tac-toe

● Experimenting with various parameter settings indicates the affective agent shows realistic behavior

● This agent can be seen as a pilot application for an artificial interaction partner that can reduce loneliness

● In future research we intend to perform user studies to show whether this really is the case

● A link to the application can be found on my website:● http://www.few.vu.nl/~mpontier

Questions?