Savvy software agents can encourage the use of second-order theory of mind

by negotiators

Harmen de Weerd, Eveline Broers, Rineke VerbruggeInstitute of Artificial Intelligence, University of Groningen

인지과학 방법론발제일 : 2015. 9. 16

발제자 : 이윤진 ( 석사 2 학기 )

Contents

• Big Question

• Brief Introduction

• Theoretical Backgrounds• Theory of Mind• Mixed Motive Situation • Colored Trails

• Experiment• 선행연구• Experiment Design• Results

• Discussion

• Q&A

Big Question

Artificially Intelligent Agent

“Savvy software agents can encourage the use of second-order theory of mind by negotiators“

in Mixed-motive Situation (Colored Trails game)

Software

agent with Theory of

Mind

Human with

Theory of Mind

Theory of Mind [ 마음이론 ]

• What is ‘Theory of Mind’?

“People use this theory of mind (1) to understand why other people behave in a certain way, (2) to predict their future behaviour, and (3) to distinguish between intentional and accidental behav-iour. People also take this ability one step further, (4) and consider that others have a theory of mind as well. This second-order theory of mind allows people to consider and even expect that others will understand why they behave the way that they do.” (Premack & Woodruff, 1978)

Human with

Theory of Mind

Theory of Mind [ 마음이론 ]

• Second-order theory of mind (Premack & Woodruff, 1978)

“allows people to reason explicitly about belief attributions made by others. For example, in the sentence “Alice knows that Bob knows that Carol is throwing him a surprise party”, a second-order knowledge attribution is made to Alice, in which she attributes knowledge to Bob.

The human ability to make use of higher-order (i.e. at least second-order) theory of mind is well-established, both through tasks that require explicit reasoning about second-order belief attributions, as well as in strategic games.”

https://cdn.psychologytoday.com/sites/default/files/blogs/276/2008/05/739-74445.jpg

Theory of Mind [ 마음이론 ]

• Theory of Mind as a system

• Agent-based computational modeling• a research tool

• how behavioral patterns emerge from the interactions between individuals

• allows precise control and monitors of the mental content of agents, including application of theory of mind

• zero-order Theory of Mind (ToM0)first-order Theory of Mind (ToM1)second-order Theory of Mind (ToM2)

http://www.ieet.org/images/uploads/p20130730b.jpg

Software agent with Theory of

Mind

Mixed Motive Situation [ 동기 갈등 상황 ]

• as the task of sharing a pie

partially cooperative - pie 늘리기 &partially competitive - pie 분배하기

• Colored Trails

http://peachypalate.com/wp-content/uploads/2014/06/DeepDishBerryCrumblePie15.jpg

Mixed Motive Situation [ 동기 갈등 상황 ]

• Colored Trails• a board game introduced by Grosz, Kraus, and colleagues

(Lin et al., 2008; Gal et al., 2010)

• negotiation setting• a research test-bed

to study decision making in humans and computer agents

• Interactions in mixed-motive settings• trading chips ( 협력 ) & win the game ( 경쟁 )

Colored Trails출발

목표지점 (goal location)

Negotia-tion

Colored Trails

is-sue

is-sue

is-sue

is-sue

Multi-Issue Bargaining Situa-tion

is-sue

is-sue

is-sue

is-sue

is-sue

is-sue

Colored Trails

• https://coloredtrails.atlassian.net/wiki/display/coloredtrailshome/

Experiment

in Mixed-motive Situation (Colored Trails game)

Software agent with Theory of

Mind

Human with

Theory of Mind

Experiment - Methods

• Participant: Human-software agent • Human (27 students of the University of Groningen, 10 female, 17 male)• Software agent (ToM0, ToM1, ToM2)

• 24 games from randomly generated games• 목표점은 chip 8 개로 도달가능 하도록 ,• Computational agent 가 피험자가 사용하는 마음이론에 따라 다른 결과를 내도록 ,• 제안 횟수에 한계가 있도록 (2 회 ~6 회 ),

• Familiarization phase ( Experimental phase)• Cover story: 회사 법적대리인으로서 다양한 고객과 협상하는 상상하라• 게임 상대방은 computer player(Alex) 라 고지• 게임 원칙 확인

Experiment - Methods

• 3 blocks (ToM0, ToM1, ToM2 condition, random order) X 8 games/block = 24 games• 각 block 에서 상대방의 전략이 바뀐다는 정보 없이 다른 고객을 상대할 것이라 고지• 누가 먼저 제안할지 (initiator) 는 무작위적으로 배당 , 그 후 번갈아 가면서 협상 제안

[ToM0]

Hu-man

8 games

[ToM1]

Hu-man

8 games

[ToM2]

Hu-man

8 games

Experiment - MethodsTheory of Mind in Software Agents • adapted from De Weerd et al. (2013)

• software agent {ToM0, ToM1, ToM2} X {ToM0, ToM1, ToM2}• competitive and cooperative aspects 각각 살펴봄

• The use of first-order and second-order theory of mind allows software agents to balance competitive and cooperative aspects of the game.

• the use of theory of mind prevents negotiations from breaking down

• {software agent} X {human} 연구 to what extent human participants reason at higher orders of theory of mindin response to each level of theory of mind

• More than 100 offers now only six turns

Experiment – Methods Colored Trails

• 5X5 tiles with four patterns• Chips and Goal location randomized• + Uncertainty

상대방의 goal location 을 알 수 없음

• Theory of Mindsoftware agent- 각 block 마다 고정human participant – 고정 X, ToM3 로 추정

Reason and update my belief in partner’s goal lo-

cation (and use it in negotiation)by using theory of mind

Experiment - MethodsTheory of Mind in Software Agents

Zero-order Theory of Mind

(ToM0)

First-order Theory of Mind

(ToM1)

Second-order Theory of Mind

(ToM2)

mental content (goal location) 추론X

O O

Zero-order belief: 제안이 수용될 likelihood

Parameter: learning speed (lambda 0~1)

First-order belief: what its own decision would have been if it had been in the position of its partner

Second-order belief: knows the fact that ‘ 상대방이 나의 제안을

분석하여 나의 목표점을 추론해낼 수 있다’ 상대방이 받아들일 만한 제안으로만 추론예 . 적은 수의 chip 보단 많은 수의 chip 제안

• positional bargaining (Fisher & Ury, 1981)

the partner has belief similar to my own 이라 추론

상대방이 제안한 내용에서 상대방의 goal location 추론

counter-offer use Zero-order belief & First-order

belief

• partner could be ToM1• 제안에 따라 상대방이 갖고 있는

‘나의 goal location ＇에 대한 믿음이 바뀐다는 것을 추론

• 상대방에게 나의 목표점에 대한 정보를 제공하여 원하는 chip 을 얻어내고자 함

• use Zero, First, Second order be-lief 사용

• interest-based negotiation (Fisher & Ury, 1981)

Experiment - MethodsTheory of Mind in Software Agents

Zero-order Theory of Mind

(ToM0)

First-order Theory of Mind

(ToM1)

Second-order Theory of Mind

(ToM2)

mental content (goal location) 추론X

O O

Zero-order belief: 제안이 수용될 likelihood

Parameter: learning speed (lambda 0~1)

First-order belief: what its own decision would have been if it had been in the position of its partner

Second-order belief: knows the fact that ‘ 상대방이 나의 제안을

분석하여 나의 목표점을 추론해낼 수 있다’ 상대방이 받아들일 만한 제안으로만 추론예 . 적은 수의 chip 보단 많은 수의 chip 제안

• positional bargaining (Fisher & Ury, 1981)

the partner has belief similar to my own 이라 추론

상대방이 제안한 내용에서 상대방의 goal location 추론

counter-offer use Zero-order belief & First-order

belief

• partner could be ToM1• 상대방도 incomplete information

갖고 있음을 알고 있음 • 제안에 따라 상대방이 갖고 있는

‘나의 goal location ＇에 대한 믿음이 바뀐다는 것을 추론

• 상대방에게 나의 목표지점에 대한 정보를 제공하여 원하는 chip 을 얻어내고자 함

• use Zero, First, Second order be-lief 사용

[ToM2]

Hu-man

ResultsHow the score of agents and participantschanged as a result of negotiation for each participant and each block

• 점선 : zero performance line 직선 : Pareto efficient outcomesallocation of resource in which it is possible to make any one individual better off without making at least on individual worse off

• Performance Ranking#1: Second-order Theory of Mind#2: Zero-order Theory of Mind#3: First-order Theory of Mind

ResultsTo what extent participants make use of theory of mind while playing Colored Trails(how similar participant’s offers were to ToM0, ToM1, ToM2 agents)

• ToM3 ‘spectator agent’: 유사성 측정: whether the offers of a participant are most consistent with zero-,first-,or second theory of mind reasoning : 각 마음이론의 confidence 측정 - which order of theory of mind would yield the best outcome

• Level of

Theory of Mind Chi-square Results

Zero-order (X^2 (2) =0:52, ns)

First-order (X^2 (2) =2:67, ns)

Second-order (X^2 (2) =24.89, p<0.001)

Results

Effects of opening bid of a negotiation

• 사전연구 : 협상 과정에 영향을 미침 (Raiffa et al., 2002)

• Average Outcome:

[initiator] software agent software agent & 피험자 모두 평균 +15 점

• Software agent ToM2 조건

[initiator] ToM2 피험자 – ----- [responder] ToM2 software agent 피험자 > agent

Discussion 1– perceived agency problem

• Experimental setting: informed participants that they were interacting with ‘client’

Would their use of Theory of Mind have changedif they knew it was not ‘client’, but ‘software’?

Discussion 2– Colored trails as a method to study human-computer interaction

• Colored Trails: cooperation X competition setting

사회적으로 있을 법한 mixed motive situatio 에서 사용하는 마음 이론의 유용성을 따져보는 데는 유용해 보이지만이를 인간 - 컴퓨터 상호작용 연구에 적용하는 것이 적합할까 ?

Discussion 3- Augmenting Human Intelligence vs. Building Artificial Intelligence

https://youtu.be/D156TfHpE1Qhttps://youtu.be/sNExF5WYMaA• Both are autonomous agent and would have ‘Theory of Mind’

Should the software program with Theory of Mind be modified to be more human-like (AI)

or to make an outcome closer to Pareto Optimal Line(IA)?

• Artificial Intelligence (AI) building a human-like intelligence in the form of an autonomous technological system

• Intelligence amplification (IA) the effective use of information technology in augmenting human intelligence Douglas Engelbart

Reference

• de Weerd, H., Verbrugge, R., & Verheij, B. (2013). Higher-order theory of mind in negotiations un-der incomplete information. In PRIMA 2013: Principles and Practice of Multi-Agent Systems (pp. 101-116). Springer Berlin Heidelberg.

• de Weerd, H., Broers, E., & Verbrugge, L. (2015). Savvy software agents can encourage the use of second-order theory of mind by negotiators. In Proceedings of the 37th Annual Conference of the Cognitive Science Society. Pasadena: Cognitive Science Society.

• Premack, D., & Woodruff, G. (1978). Does the chimpanzee have a theory of mind?. Behavioral and brain sciences, 1(04), 515-526.

• https://coloredtrails.atlassian.net/wiki/display/coloredtrailshome/Colored+Trails+Homepage

• https://en.wikipedia.org/wiki/Intelligence_amplification#Douglas_Engelbart:_Augmenting_Human_Intellect

Thank you for your attention

Q & A