Bob MarinierAdvisor: John Laird

Functional Contributions of Emotion to Artificial

Intelligence

Introduction

2

Folk psychology considers emotions to be a distraction from logical thought

People tend to think that emotion is unknowable, indefinable

Psychological work in the last several decades has demonstrated that emotion plays a critical role in effective functioning and learning

Introduction

3

Research GoalsBring the functionality of emotion to AICreate a precise computational definition of emotion

ApproachIntegrate emotion with a complete agent frameworkComputationally distinguish emotion, mood and

feelingWeight feeling’s importance by computing its

intensityUse feeling as intrinsic reward signal to drive

reinforcement learning

Appraisal Theories of Emotion

4

A situation is evaluated along a number of appraisal dimensions,many of which relate the situation to current goalsNovelty, goal relevance, goal conduciveness, expectedness, causal

agency, etc.Result of appraisals determines emotionThe emotion is combined with mood, which is an “average” over

recent emotions, to form a feeling, which is actually perceived with some intensity

The feeling can then be coped with (via internal or external actions)

SituationGoals

Appraisal

Emotion, Mood, Feeling

Coping

Appraisals to Emotions(Scherer 2001)

5

Joy Fear Anger

Suddenness High/medium High High

Unpredictability High High High

Intrinsic pleasantness Low

Goal/need relevance High High High

Cause: agent Other/nature Other

Cause: motive Chance/intentional Intentional

Outcome probability Very high High Very high

Discrepancy from expectation

High High

Conduciveness Very high Low Low

Control High

Power Very low HighWhy these dimensions?What is the functional purpose?

Functions of Emotion

6

Situation summary: Appraisals and emotion provide abstract interpretation

Decouples stimulus/response: Can react to interpretation instead of stimulus

Attention: Some appraisals help prioritize processingHistorical context: Mood provides a context for current

interpretationsLearning: Feeling may provide an intrinsic reward signalMemoryDecision makingAction preparationCommunication

Outline

7

Integrate emotion with a complete agent framework

Computationally distinguish emotion, mood and feeling

Weight feeling’s importance by computing its intensity

Use feeling as intrinsic reward signal to drive reinforcement learning

Discussion & Conclusion

SituationGoals

Appraisal

Emotion, Mood, Feeling

Coping

Newell’s Abstract Functional Operations(Newell 1990)

8

Allen Newell defined a set of computational Abstract Functional Operations that are necessary and sufficient for immediate behavior in humans and complete agents

Perceive Obtain raw perception

Encode Create domain-independent representation

Attend Choose stimulus to process

Comprehend

Generate structures that relate stimulus to tasks and can be used to inform behavior

Task Perform task maintenance

Intend Choose an action, create prediction

Decode Decompose action into motor commands

Motor Execute motor commands

Newell’s Abstract Functional Operations(Newell 1990)

9

…but how these actually work was not clear.

Perceive What information is generated?

Encode What information is generated?

Attend What information is required?

Comprehend

What information is required and generated?

Task What information is required?

Intend What information is required?

NAFO and Appraisal(Marinier & Laird 2006)

10

Generated By Required By

Suddenness Perceive

AttendUnpredictability

EncodeIntrinsic pleasantness

Goal relevance

Causal agent

ComprehendComprehend,Task,Intend

Causal motive

Outcome probability

Discrepancy from expectationGoal/need conducivenessControl

Power

Integrate emotion with a complete agent framework

Computationally distinguish emotion, mood and feeling

Weight feeling’s importance by computing its intensity

Use feeling as intrinsic reward signal to drive reinforcement learning

Discussion & Conclusion

Outline

11

SituationGoals

Appraisal

Emotion, Mood, Feeling

Coping

Body

Symbolic Long-Term Memories

Procedural

Short-Term Memory

Situation, Goals

Decision Procedure

Chunking

Reinforcement

Learning

Semantic

SemanticLearning

Episodic

EpisodicLearning

Perception ActionVisual

Imagery

Extending Soar with Emotion(Marinier & Laird 2007)

12

Soar is a cognitive architecture

A cognitive architecture is a set of task-independent mechanisms that interact to give rise to behavior

Cognitive architectures are general agent frameworks

Feelin

g

Genera

tion

Reinforcement

Learning

Emotion.5,.7,0,-.4,.3,

…

Extending Soar with Emotion(Marinier & Laird 2007)

13

Body

Decision Procedure

Perception Action

Appraisals

Feelings Short-Term Memory

Situation, Goals

Mood.7,-.2,.8,.3,.6,

…

Feelings

Knowledge

Architecture

Symbolic Long-Term Memories

Procedural

Chunking

Semantic

SemanticLearning

Episodic

EpisodicLearning

+/- In

tensity

Feeling.9,.6,.5,-.1,.8,

…

VisualImagery

Computing Feeling from Emotion and Mood(Marinier & Laird 2007)

14

Assumption: Appraisal dimensions are independentLimited Range: Inputs and outputs are in [0,1] or [-1,1]Distinguishability: Very different inputs should lead to

very different outputsNon-linear: Linearity would violate limited range and

distinguishability

Example

16

Maze Task

Start

Goal

17

Feeling Dynamics Results

18

very easy

Computing Feeling Intensity(Marinier & Laird 2007)

19

Motivation: Intensity gives a summary of how important (i.e., how good or bad) the situation is

Limited range: Should map onto [0,1]No dominant appraisal: No single value should drown out

all the othersCan’t just multiply values, because if any are 0, then

intensity is 0Realization principle: Expected events should be less

intense than unexpected events

Example

21

Outline

22

Integrate emotion with a complete agent framework

Computationally distinguish emotion, mood and feeling

Weight feeling’s importance by computing its intensity

Use feeling as intrinsic reward signal to drive reinforcement learning

Discussion & Conclusion

Intrinsically MotivatedReinforcement Learning(Sutton & Barto 1998; Singh et al. 2004)

23

Environment

Critic

Agent

Actions

StatesRewar

ds

External Environment

Internal Environment

Agent

Critic

Actions

StatesRewar

dsDecisions

Sensations

Appraisal

Process+/-

Feeling Intensity

“Organism”

Learning Task

Start

Goal

24

Learning Results

25

Discussion & Conclusion

26

DiscussionAgent learns fast

Gets frequent reward signalsMood accelerates learning

Provides reward during those steps in which the agent has no emotion

ConclusionDeveloped an initial computational model of

emotionIntegrated model with complete agent frameworkDemonstrated some functional advantages of

integration