distributed search by agents with personal preferences

Ben-Gurion University of the Negev

Department of Computer Science

Distributed Search by Agents with Personal Preferences

Alon Grubshtein

Lessons learnt from applying distributed constraint

reasoning to “realistic” agents with personal preferences

Ben-Gurion University of the NegevDepartment of Computer Science

Before we begin…


Multi Agent Systems

Constraint Reasoning

Distributed Computing

Distributed

Constraint

Reasoning

In this talk:


Sometime back in 2006

Check out this great phone I gotI can use it to work on my calendar!!! Who needs a

computer with such phones?

You can even write programs for it…

Lets write a distributed agent to automate meeting coordination


Constraint reasoning (centralized)

A Constraint Reasoning problem:

• Variables• Domains• Constraints (relations)

A solution concept (target objective)


Examples


What’s in a constraint?

Two important classes of problems:

• Constraint Satisfaction (CSP)

• Constraint Optimization (COP)

A satisfying assignment

A minimal cost assignment


Constraint algorithms

How do we find a solution?

• Enumerate feasible outcomes• Backtracking / Branch and Bound• Intelligent backtracking• Pre processing, forward checking and

heuristics• Local search algorithms


From centralized to distributed

The problem itself is distributed across computational nodes – agents:• Privacy• “Difficulty”

http://research.microsoft.com/en-us/um/people/mbj/mars_pathfinder/mars_pathfinder.html


Constraint reasoning (distributed)

Distributed Constraint Reasoning (DCR) problem:

• Agents• Variables• Domains• Constraints (relations)

DCSP /

DCOP


From centralized to distributed

• Computation on separate entities• Communication via messages• Each agent knows only a small

portion of the problem

• Allows for parallel computation

DISTRIBUTED =/= PARALLEL


DCR algorithms


Local Search for “real” problems

• Computationally hard• Simplistic myopic algorithms

(“local search”/“adaptive heuristics”)

• Example, DSA:1. Pick a random assignment2. While (stop condition):

a. Send assignment to all neighbors (receive)b. If can improve local state by changing assignment:

change with probability p


A simple MAS example

Coordinating a meeting (e.g., seminar):

• Two alternatives: Morning or Evening• More participants – better• Prof. Lynn does not care when• If students disagree - morning• Alice prefers morning• Anna prefers evening

Prof. Lynn

Alice

Anna

5 1

0 2

M

M

E

E

AliceAnna

3 0

2 4

M

M

E

E

AliceAnna

5 , 3

1 , 0

0 , 2

2 , 4

M

M

E

E

AliceAnna


Solving as a DCOP

Alice Anna

5 , 3 1 , 0

0 , 2 2 , 4

M

M

E

E

AliceBob

8 1

2 6

M

M

E

E

AliceBob

What if students can’t/won’t

communicate preferences?

Alice

Anna

M

M M

E

EE

Cost:

8 1 2 6


Standard model solutions

• Easiest solution: Disclose preferences

• An alternate approach:Add unary constraints

• Problem: Can prove that this approach will fail on some instances


How its done these days

The PEAV formulation:

A1 A2

x1 x2

x21 x1

2x y

a 3 6

b 7 5 x y

a 4 1

b 2 8mirror variables

hard constraints

x12

x2

x1

x21

• Modified search space Can’t be used with many local search algorithm!

• Requires more space


Introducing ADCOPs

Different preferences on outcomes are not part of the standard model…

Asymmetric constraintsFormally:

Captures the idea that each agent has a personal “table” with costs/gains of each outcome


ADCOPs

• ADCOPs:• At least as expressive as existing model• Succinct representation• Used with existing local search algorithms

Search can be improved by introducing cooperation/coordination


ADCOP Local Search (quality)

0 20 40 60 80 100 120 140 160 180 2002000

3000

4000

5000

6000

7000

8000

9000

10000

11000

12000

MCS-MGMGCA-MGMACLSMGM2MGMDSA

Cycles

Solu

tion

Cost

0 20 40 60 80 100 120 140 160 180 200

Cycles

Solu

tion

Cost

0 20 40 60 80 100 120 140 160 180 200

Cycles

Solu

tion

Cost

DCOP ADCOP


Multi Agent Systems



Distributed Constraint Reasoning


Rethinking agents joint objective

11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 3010

20

30

40

50

60

70

Normalized

Number of meetings

% Q

uality

Utilitarian

Difference in best and worst gains – Meeting Scheduling Problem


Agreeing on an outcome(what is a fair solution?)

• Game Theory defines stable points:

• Assumptions:1. Self interested2. Rational (some knowledge)


Graphical Games

• ADCOPs are Games played on a Graph• Closely related to Graphical Games• ADCOPs:

No knowledge assumed Agents are cooperative An even more succinct representation

• Can use DCR techniques to solve a game theoretic multi agent problem!

5 , 3

1 , 0

0 , 2

2 , 4

M

M

E

E

AliceAnna


Asynchronous Nash BackTracking (ANT)

• Transform a MAS to a Distributed Constraint Problem

• A distributed, asynchronous, non-binary, asymmetric search

Two symmetric constraints

Three asymmetric constraints


ANT

• A satisfaction problem• Inspired by ABT (ABT-1ph)• A solution always exists• Guaranteed to find an epsilon NE:

• More efficient than other distributed GG solvers


Multi Agent Systems





The quality of a stable solution

• A stable solution is not necessarily a good one…

• Why is that?• Competitive solution for cooperative

agents?

A2 \ A1 Cooperate Defect

Cooperate 4,4 6,0Defect 0,6 1,1


Agreeing on an outcome(what is a fair solution?)

Cooperativ

e

Competitive

Utilitarian, Egalitarian, Leximin,…

Stable points: Nash (pure/mixed), Bayesian, Strong, Correlated, …


A different approach

assume cooperation but try to incentivize agents by examining personal goals

• “Cost of Cooperation”• Baseline search


The Cost of Cooperation (CoC) criteria:

The difference in an agent’s gain from the worst equilibrium (from its point of view) outcome and from cooperatively solving the problem

Possible solutions

U2(x)

U1(x)

Pareto front

Optimal solution (max sum)

Nash equilibrium solutions

Non positive CoC solutions


u1=med

C2

A simple P2P example

a1 a8

a2

a3

a4

a5

a6

a7

C1

FS F

u1=low

u2=high

F S

S

• Agents only interact with neighbors (unknown topology)

• An agent’s gain is lowered when exerting resources on sharing (S)

• Gain is maximized if an agent can free ride the efforts of other agents (F)

• Gain is lowest if no one shares


Competitive and Cooperative solutions

a1

a8

a2

a3

a4

a5

a6

a7

S S

F F

FF

F F

0.3 0.3

1 1

1

1 1

1

Cooperative Solution

a1 a8

a2

a3

a4

a5

a6

a7

S S

F F

FF

F F

0.3 0.31 1

0

0 0

0

A Bayesian stable solution (possible)


Cost of Cooperation solution

• An improvement can be guaranteed (proved) for a set of interactions!

a1 a8

a2

a3

a4

a5

a6

a7

S S

F

F

F F

0.3 0.31 1

0

0S

S

0.3

0.3

35


Applied to network games

ADCOP (CoC)

Maximizing utilities


Multi Agent Systems





Limits of the CoC approach

• So far we have seen several solutions:Fully cooperative (Utilitarian)Stable (Epsilon Nash Equilibrium)A combination:

Non positive Cost of Cooperation

• However…

A2 \ A1 Left Right

Up 2,5 4,1Down 6,1 0,3

Mixed NE: (1/2,1/3)Gain: (3, 7/3)

NO Cost of Cooperation solution!


A framework for partial cooperation

• Agents gain is different • Do not “improve cooperatively”• Define cooperation with respect to

some baseline solution• Agents must agree on the baseline

(may need to apply a simple search algorithm).


Modes of cooperation

• Define modes of cooperation within an Interaction Process: Non-Cooperative (NC) – agents are driven by

their own goals and act rationally. Can serve as a baseline solution

Guaranteed Personal Benefit (GPB) – agents seek an agreement and may take irrational steps. Guarantees a Pareto improvement

λ-cooperation – agents agree to a bounded loss from their NC gain (up to some predefined λ)


Local Search and Partial Cooperation

Maintain threshold/guarantee:

1. Incorporate with distributed “anytime”

Can use any LS algorithm Focus on exploration

2. Tailor an algorithm maintain invariant (begins in a “legal”

state)


Evaluation

Three key parameters:1. Compromise levels

(lambda)2. Agents’ degree3. Costs distribution

0 200 400 600 800 10001200140016001800200016000

17000

18000

19000

20000

21000

22000

23000

24000

Goods-MGMAGCMGMMGM2MCS-MGMGCA-MGM

Cycles

Solu

tion

Cost


Multi Agent Systems





SUMMARY & CONCLUSIONS


Summary

Multi Agent Problem

DCSP/DCOP

Utilitarian(Minimal sum of costs)

Asymmetric Constraints

Stableε-Nash Equilibrium

Non positive Cost of Cooperation

Partial Cooperation

Represent

ation

Algorithm

Objecti

ve


Conclusions

• Three points (‘up and down the ladder of abstraction’):

1. How to model the problem2. How does the model effect the means to

find a solution 3. What is a solution?

• Rethinking basic assumptions• Applying well established models to

simple realistic settings can reveal many of its shortcoming


Journal publications:• Arnon Netzer, Alon Grubshtein and Amnon Meisels, “Concurrent Forward Bounding”, Artificial Intelligence, Vol. 193, pp. 186-216,

2012.• Roie Zivan, Alon Grubshtein and Amnon Meisels, “Hybrid Search for Dynamically changing CSPs”, Constraints, special issue on

constraint satisfaction for planning and Scheduling, Vol. 16, num. 3, pp. 228-249, 2011.• Alon Grubshtein and Amnon Meisels, “Cost of Cooperation for Scheduling Meetings”, Journal of Computer Science and Information

System (ComSIS), Vol. 7, num. 3, pp. 551-567, 2010.Conference and workshops publications :• Alon Grubshtein and Amnon Meisels, “Finding a Nash Equilibrium by Asynchronous Backtracking”, 18th Intl. Conf. on Principles and

Practice of Constraint Programming (CP’12), pp. 925-940, Quebec city, Canada, Oct. 2012.• Alon Grubshtein, Roie Zivan and Amnon Meisels, “Partial Cooperation in Multi Agent Local Search”, 20th European Conf. on Artificial

Intelligence,pp.378-383, Montpellier France, Aug. 2012• Roie Zivan, Alon Grubshtein, Michal Friedman and Amnon Meisels, “Partial Cooperation in Multi Agent Search”, (Extended Abstract)

Proc. 11th intern. Conf. Autonom. Agents Multi agent Sys. (AAMAS’12), Valencia, Spain.• Alon Grubshtein and Amnon Meisels, “A Distributed Cooperative Approach for Optimizing a Family of Network Games”, Proc. of the 5th

Intern. Symp. on Intelligent Distributed Computing (IDC’11), Delft, the Netherlands, pp. 49-62, October 2011.• Alon Grubshtein and Amnon Meisels, “A Distributed Cooperative Approach for Optimizing a Network Game”, Proc. 13th Intern.

Workshop on Dist. Constraints Reasoning (DCR’11), Barcelona, Spain, June 2011.• Alon Grubshtein, Nir Herschorn, Arnon Netzer, Guy Rapaport, Guy Yaffe and Amnon Meisels, “The Distributed Constraints (DisCo)

Simulation Tool”, Proc. 13th Intern. Workshop on Dist. Constraints Reasoning (DCR’11), Barcelona, Spain, June 2011.• Alon Grubshtein and Amnon Meisels, “Cooperation Mechanism for a Network Game”, Proc. 3rd Intern. Conf. Agents and AI

(ICAART’11), Rome, Italy, pp. 336-341, January 2011.• Alon Grubshtein, Tal Grinshpoun, Amnon Meisels and Roie Zivan, “Local Search for Distributed Asymmetric Optimization” , Proc. 9th

intern. Conf. Autonom. Agents Multi agent Sys. (AAMAS’10), Toronto, Canada, pp. 1015-1022, May 2010.• Arnon Netzer, Amnon Meisels and Alon Grubshtein, “Concurrent Forward Bounding for DCOPs”, Proc. 12th Intern. Workshop on Dist.

Constraints Reasoning (DCR’10) at AAMAS’10, Toronto, May 2010.• Alon Grubshtein, Nurit Gal-Oz, Tal Grinshpoun, Amnon Meisels and Roie Zivan, “Manipulating Recommendation Lists by Global

Considerations”, Proc. 2nd Intern. Conf. Agents and AI (ICAART’10),pp. 135-142, Valencia, Spain, January 2010.• Alon Grubshtein and Amnon Meisels, “Cost of Cooperation for Scheduling Meetings”, Proc. 3rdIntern.Symp. Intell. Dist. Comp. (IDC’09),

Vol. 237, pp. 227-236, Ayia Napa, Cyprus, October 2009.• Alon Grubshtein, Tal Grinshpoun, Amnon Meisels and Roie Zivan, “Asymmetric Distributed Constraint Optimization”, Proc. 11th Intern.

Workshop on Dist. Constraints Reasoning (DCR’09) at IJCAI-09, Pasadena CA, July 2009.• Ehud Gudes, Nurit Gal-Oz and Alon Grubshtein, “Methods for Computing Trust and Reputation While Preserving Privacy”, Proc. Data

and App. Security XXIII, 23rd Ann. IFIP WG 11.3 Working Conf. (DBSEC’09), Vol. 5645, pp. 291-298, Montreal, Canada, July 2009.• Amir Gershman, Alon Grubshtein, Amnon Meisels and Roie Zivan, “Scheduling Meetings by Agents”, Proc.7thintern. Conf. Practice and

Theory Auto. Timetabling (PATAT’08), Montreal, August 2008.

Thank you!

distributed search by agents with personal preferences

Documents

parallelbengurion university

distributed agent

realistic agents

distributedthe problem

computational nodes

messageseach agent

local state

whenif students