Effect of Shared-attention for Human-Robot Interaction

Junji Yamato

jy@acm.org 　

NTT Communication Science Labs., NTT Corp. Japan

Kazuhiko Shinozawa, Futoshi Naya

ATR Intelligent Robot and Communication Labs.

Aim

To build Social Robot/Agent

Sub goalTo establish

Evaluation methodsDesign guidelines

for communication of human-robot/agent

• To measure the influence of Agent/Robot on users

• Acceptance ratio of agent/robot recommendation

Method

Color name selection taskBlue or Green? Cobalt green or emerald green?Skin color or KARE-IRO?SUMIRE-IRO or AYAME-IRO?--------

Total:30 questions.(from color name text book)

• No “correct” answer• Easy to be influenced

Four experiments1. Compared agent and robot2. Compared agent and robot in physical

world3. Measured the effect of eye contact4. Measured the effect of shared-attention

Yamato, J., Shinozawa, K., Brooks, R., and Naya, F. Human-Robot Dynamic Social Interaction. NTT Technical Review 1, 6(2003), 37-43.

Detailed description of Experiment 1 and 2

Available on-linehttp://www.ntt.co.jp/tr/Back number -> Sep. 2003

Shinozawa, K., Naya, F., Yamato, J., and Kogure, K. Differences in Effect of Robot and Screen Agent Recommendations on Human Decision-Making , IJHCS (to appear)

Experiment 1, 2, and description of K4(robot)

Experiment 1 ：Compare Agent and Robot

• Conditions: 30 questions, 30 subjects in each group‐ Same question sequences, same voice, similar gesture• Measurement: acceptance ratio, questionnaire

Agent RobotAgent Robot

Experiment 1: Robot

QuickTime˛ Ç∆MPEG-4 ÉrÉfÉI êLí£ÉvÉçÉOÉâÉÄ

ǙDZÇÃÉsÉNÉ`ÉÉÇ å©ÇÈÇΩÇflÇ…ÇÕïKóvÇ≈Ç∑ÅB

Experiment 1: Result

• Acceptance ： agent > robot (p<.01)

• Familiarity ： independent

Initial expectationRobot has more influence because it lives in 3D world,same as subjects.

ag

en

tro

bot

○

×

GapGap Gap

Experiment 2: Compare in physical world

Color plate

Button box

Button box

• No recommendation (30 subjects)• Recommendation by robot （ 31 subjects ）• Recommendation by agent (30 subjects)

Experiments

Experiment 2: Result

• selection ratio ： robot > agent ( p < 0.05) 　　　　　 robot>> no recommendation ( p < 0.01)

Experiment 1 and 2: Results

Media world Physical world

agen

tro

bot

Consistency matters.

○

× ○

×

Embodiment and communication

Why robot is better?

• Easy to detect gaze– Eye contact– Shared attention/joint attention

Measure the effect of eye contact and shared-attention

• Eye contact was established by face tracking

• Eye contact time: period that subject looked at robot and robot looked at subject

• Eye contact time and selection ratio?

• Two groups (14 subjects each)

– Eye contact, and NO eye contact

Experiment 3: Effect of eye contact (mutual gaze)

Robots

• Higher selection ratio for eye contact group

• K4: No E.C. < E.C. (p=0.012)

• Rabbit: No E.C. < E.C. (p=0.003)

Selection ratio

• Shared attention:– Period that robot looks at an object and

subject looks at the same object. (color

plate, button box)• SA time and selection ratio

– Is there correlation?

Experiment 4: Effect of shared-attention

• Robot looks at color plate and button box by prepared program

• Eye contact established by face tracking

Establishing shared-attention

QuickTime˛ Ç∆ÉtÉHÉg - JPEG êLí£ÉvÉçÉOÉâÉÄǙDZÇÃÉsÉNÉ`ÉÉÇ å©ÇÈÇΩÇflÇ…ÇÕïKóvÇ≈Ç∑ÅB

QuickTime˛ Ç∆ÉtÉHÉg - JPEG êLí£ÉvÉçÉOÉâÉÄǙDZÇÃÉsÉNÉ`ÉÉÇ å©ÇÈÇΩÇflÇ…ÇÕïKóvÇ≈Ç∑ÅB

QuickTime˛ Ç∆ÉtÉHÉg - JPEG êLí£ÉvÉçÉOÉâÉÄǙDZÇÃÉsÉNÉ`ÉÉÇ å©ÇÈÇΩÇflÇ…ÇÕïKóvÇ≈Ç∑ÅB

Example: video

• 28 subjects• SA time = 51.7 sec (total for 30 questions)

– (Longer than in Experiment 3 )• Selection ratio. Average: 0.57 S.D.= 0.14

• Some subjects were positive, and others were not. Clear contrast, from the questionnaire.

　　 Example: Robot is prompting wrong choice. I feel the robot forced me to select his recommendation (negative).

Experimental conditions

• No correlation

SA time and selection ratio

Shared-Attention time (count) 50count=1sec.

Selection ratio

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 500 1000 1500 2000 2500 3000 3500 4000

ånóÒ1

• Ego-gram based on transactional analysis

• Measure three ego-states by questionnaire– CP, NP (critical parent, nurturing

parent)– A (adult)– FC, AC (free child, adapted child)

• TEG (Tokyo Univ. Egogram ） is common in Japan

Clustering subjects by TEG(Ego-gram)

• Strong correlation in SA time and acceptance ratio for high

AC (Adapted Child) group

High/Low TEG measurement and SA time.

CP NP A FC AC

Threshold (threshold) 10 17 12 14 12

Selection ratio High group 0.548717949 0.564444444 0.564285714 0.617948718 0.597777778

Low group 0.586666667 0.574358974 0.573809524 0.526666667 0.535897436

SAtime2553.461538 2581.933333 2575.285714 2587.615385 2510.866667

2612.266667 2588.461538 2594.642857 2582.666667 2670.461538

Correlation fo High -0.046541377 -0.035889649 -0.03145821 0.136152737 0.405908084

SAtime and sel ratio Low 0.054648939 0.10996751 0.055147415 -0.078258186 -0.461514856

• Positive correlation(Speaman’s r=0.51,p=0.051).

SA time and selection ratio (high AC & low CP group)

• High-SA group = high selection ratio (p<0.05)

SA time and selection ratio

（ high AC group)

• High AC subject (obedient type) showed

positive correlation between SA time and

selection ratio.

• No significant difference between SA time itself

and selection ratios for high AC and low AC

groups

• Eye contact and shared-attention promote

close communication. Some people like such

intimate relation, but others don’t. It depends

on the character.

• SA is effective. Even SA was not “actually”

realized. We do not need to develop image

understanding technology; we just have to fake

it.

Result and Discussion