robots and agents that care: social, cognitive & affective ... · theories on social, cognitive...

Mark Neerincx

Gert Jan van der Burg

Robots and Agents that Care: Social, Cognitive & Affective Support

http://www.pal4u.eu/

Method

Situated Cognitive EngineeringSituated Cognitive EngineeringSituated Cognitive EngineeringSituated Cognitive EngineeringF

ou

nd

ati

on

Operational

Demands

Theories on social,

cognitive & affective

processes

Technological

constraints &

opportunities

Effect /

Claim

Function /

Requirement

Use Case

Sp

eci

fica

tio

n Integrated

prototype

Ev

alu

atio

n

Part1 Part1 …

Needs Analysis

Diabetes Camps

Situated Needs Analysis: Photo Elicitation

Diabetes Camps

Draw-Write-Tell

Story-Telling

Image Theatre

Situated Needs Analysis: Creative Methods

Diabetes Camps

5

Calculation SupportHelp for MeasuringFriendship

Play-fellow

Support

...

Creative methods• Draw-write-tell• Image theatre• Story-telling

• Positive: Support of friends • Negative: Diabetes regime • Have fun activities like

others• ...

Photo-elicitation

Situated Needs Analysis: Outcomes

Diabetes Camps

Comparing Italian and Dutch Children’s Experiences Comparing Italian and Dutch Children’s Experiences Comparing Italian and Dutch Children’s Experiences Comparing Italian and Dutch Children’s Experiences

with the Robot at the Campswith the Robot at the Campswith the Robot at the Campswith the Robot at the Camps

Italian camp:

• learning to live with diabetes & needs analysis

• 34 children, age 10-14

Dutch camp:

• sharing leisure experiences & needs analysis

• 21 children, age 8-11

Child-robot activities (quiz & sorting game):

• Sentiment Analyses on child’s textual answers)

• Observations of speech, gestures and posture

Results

• Children were lenient &

positive

• Emotional expressions differ

per activity

• Italians more expressive, less

eye contact

• User requirements for culture-

aware personalization

Method

Situated Cognitive EngineeringSituated Cognitive EngineeringSituated Cognitive EngineeringSituated Cognitive EngineeringF

ou

nd

ati

on

Operational

Demands

Theories on social,

cognitive & affective

processes

Technological

constraints &

opportunities

Effect /

Claim

Function /

Requirement

Use Case

Sp

eci

fica

tio

n Integrated

prototype

Ev

alu

atio

n

Part1 Part1 …

Needs Analysis

Needs Synthesis

• Use cases

• Values

• Requirements

• Claims

8

• Personas

• Journey maps

• Storyboards

Effect /

Claim

Function /

Requirement

Use Case

• Diabetes camps

• Focus groups

• Interviews

• Stakeholder analysis

Deriving Design SpecificationsDeriving Design SpecificationsDeriving Design SpecificationsDeriving Design Specifications

Involving Parents and Caregivers as PAL -Users

9

PAL Control & Monitor• Child profile• Goal setting• Authoring• Monitoring• …

“Blended Care”• Caregivers are present during

child-robot activities• Policies and agreements on

• Goals• Information sharing (privacy)

Method

Situated Cognitive EngineeringSituated Cognitive EngineeringSituated Cognitive EngineeringSituated Cognitive EngineeringF

ou

nd

ati

on

Operational

Demands

Theories on social,

cognitive & affective

processes

Technological

constraints &

opportunities

Effect /

Claim

Function /

Requirement

Use Case

Sp

eci

fica

tio

n Integrated

prototype

Ev

alu

atio

n

Part1 Part1 …

Needs AnalysisSelf-Determination

Theory

PAL prototype version 0.1

12

Requirements Baseline

HighHighHighHigh----Level RequirementsLevel RequirementsLevel RequirementsLevel Requirements

Autonomy Competence Relatedness

Timeline create (timeline

related) goals

edit activities (e.g.

school, sport, meal)

and measurements

add photo’s, mood and

thoughts

Avatar guides the children

through the selection

process

praises adherence

and progress

discusses activities, matches

child mood, shares stories &

photos of itself

Self-Determination Theory

Method

Situated Cognitive EngineeringSituated Cognitive EngineeringSituated Cognitive EngineeringSituated Cognitive EngineeringF

ou

nd

ati

on

Operational

Demands

Theories on social,

cognitive & affective

processes

Technological

constraints &

opportunities

Effect /

Claim

Function /

Requirement

Use Case

Sp

eci

fica

tio

n Integrated

prototype

Ev

alu

atio

n

Part1 Part1 …

Needs AnalysisSelf-Determination

Theory

Evaluation at Diabetes Camp

Formative Cooperative EvaluationFormative Cooperative EvaluationFormative Cooperative EvaluationFormative Cooperative Evaluation

MyPal (timeline)

PAL Avatar

Recommendations for

• Functions

• Ease of Use

• Attractiveness

• …

Recommendations for

• Personalization

• Behaviours

• Appearances

• …

Evaluation in Hospital and Home

Formative Usage EvaluationFormative Usage EvaluationFormative Usage EvaluationFormative Usage Evaluation

13 participants (age: 7-12; 4 girls, 9 boys)

Patients at Ziekenhuis Gelderse Vallei

Three parts:

• Introduction session at hospital

• Two-week myPAL use at home

• Evaluation session at hospital

Evaluation in Hospital and Home

OutcomesOutcomesOutcomesOutcomes

“Overachievers”

PAL prototype version 1.0

.

4 week experiment period (Home)

Use of PAL system (Quiz, timeline)

1st Appointment (Hospital)

• QOL questionnaire• Interview• Introduction MyPal• Set goals in MyPal• Nao introduction • Play with Nao

3rd Appointment

(Hospital)

• Interview• Play with NAO

2nd Appointment

(Telephone)

• General check-up• Interview

Experiment in Italy and NetherlandsExperiment in Italy and NetherlandsExperiment in Italy and NetherlandsExperiment in Italy and Netherlands

Design and procedureDesign and procedureDesign and procedureDesign and procedure

PAL Year 1: First analysis of individual system usage and knowledge developement.

3 trends

19

PAL Year 1: First analysis of individual system usage and knowledge developement.

3 trends

• Minimal use during whole period

PAL Year 1: First analysis of individual system usage and knowledge developement.

3 trends

• Minimal use during whole period

• Initial use with decrease over time

PAL Year 1: First analysis of individual system usage and knowledge developement.

3 trends

• Minimal use during whole period

• Initial use with decrease over time

• Consistent use during whole period

Evolving Empirical Foundation

Method

Situated Cognitive EngineeringSituated Cognitive EngineeringSituated Cognitive EngineeringSituated Cognitive EngineeringF

ou

nd

ati

on

Operational

Demands

Theories on social,

cognitive & affective

processes

Technological

constraints &

opportunities

Effect /

Claim

Function /

Requirement

Use Case

Sp

eci

fica

tio

n Integrated

prototype

Ev

alu

atio

n

Part1 Part1 …

Needs AnalysisSelf-Determination

Theory

BDI-agent

Ontological Reasoning

Percepts:

• Performance

• Facial expressions

• Heart rate

• …

Actions:

• Choose answer

• Motivational feedback

• Propose activity (switch)

• …Goals:

• Play 4 quiz sessions

• Establish level X

• Share emotions within

boundaries

• …Beliefs:

• Child’s emotion

• Child’s skill level

• Child’s preference

• …

Emotions:

• Valence

• Arousal

Child-Agent Interaction

We kunnen een quiz spelen, dagboekje

invullen en je diabetesdoelen

bekijken

“No machines with self-sustaining long-term goals and intent have been developed, nor are they likely to be developed in the near future”

Stone et al. (Sept 2016). Artificial Intelligence and Life in 2030. One Hundred Year Study on Artificial Intelligence: Report of the 2015-2016 Study Panel, Stanford University, Stanford, CA

Model-based

Data-driven

KnowledgeBase

learnin

g

interactive thing

effectorssens

ors

interactive thing

ontologypolicies simulations

sens

ors effectorsse

nsor

s

effectors

sens

ors

effectors

BC-Strategies

DM-Protocols RelationshipsEmotion

PrivacyGoal-Setting

http://www.pal4u.eu/

Human Values Situated Evolutionary

Partners of the project

Co-funded by the Horizon 2020 Framework Programme of the European Union under grant agreement no 643783

Thank you for your attentionThank you for your attentionThank you for your attentionThank you for your attention