aamas-19 tutorial on: multi-agent distributed …fferdinando3/cfp/aamas19/tutorial-p… · 5 cost =...

83
AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED CONSTRAINED OPTIMIZATION Hands on pyDCOP Pierre Rust Gauthier Picard

Upload: others

Post on 25-Apr-2020

7 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

AAMAS-19 Tutorial on:

MULTI-AGENT DISTRIBUTEDCONSTRAINED OPTIMIZATION

Hands on pyDCOP

Pierre Rust Gauthier Picard

Page 2: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Menu

Hands on PyDCOP I

Hands on PyDCOP II

Focus on Smart Environment Con�guration Problems

Distributing Computations

Hands on PyDCOP III

Dynamic DCOPs

Conclusion

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 2

Page 3: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Menu

Hands on PyDCOP I

Hands on PyDCOP II

Focus on Smart Environment Con�guration Problems

Distributing Computations

Hands on PyDCOP III

Dynamic DCOPs

Conclusion

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 3

Page 4: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP I

Install VirtualBoxImport the pyDCOP Virtual Machine (http://bit.ly/pyDCOP)I It’s a Debian image with everything preinstalled:I python3, pyDCOP, matplotlib, glpk, etc.

Alternatively, followhttps://pydcop.readthedocs.io/en/latest/installation.html

1. https://pydcop.readthedocs.io/en/latest/tutorials/getting_started.html

2. https://pydcop.readthedocs.io/en/latest/tutorials/analysing_results.html

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 4

Page 5: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IVirtual machine Setup

Before starting the VM:

"Bridged adapter" mode

Select wi� network adapter

Reset MAC Address

Then

Start the VM

login: dcop / pyDCOP

Launch a terminal

Note down the IP with ipaddress

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 5

Page 6: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IVirtual machine Setup

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 6

Page 7: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IFiles for the tutorials are in /home/dcop/tutorials.

$ cd /home/dcop/tutorials/hands-on_1

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 7

Page 8: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IDCOP - Graph Coloring

v2

v1 v3

(a) constraints graph

p2 v2

d1,2

v1

d2,3

v3p1 p3

(b) factor graph

Objective: minimizeDomain: 2 colors R and BVariables: V1, V2, V3

Constraints: neighbors must have di�erent colors + preferencesAgents: 3 agents

Yaml representation

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 8

Page 9: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IpyDCOP yaml format

graph_coloring.yaml

name: graph coloringobjective: min

domains:colors:values: [R, G]

variables:v1:domain: colorsv2:domain: colorsv3:domain: colors

constraints:pref_1:type: extensionalvariables: v1values:-0.1: R0.1: G

pref_2:type: extensionalvariables: v2values:-0.1: G0.1: R

pref_3:type: extensionalvariables: v3values:-0.1: G0.1: R

diff_1_2:type: intentionfunction: 10 if v1 == v2 else 0

diff_2_3:type: intentionfunction: 10 if v3 == v2 else 0

agents: [a1, a2, a3, a4, a5]

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 9

Page 10: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP ISolving the Graph Coloring DCOP

Command:

$ pydcop solve --algo dpop graph_coloring.yaml

Output:

..."assignment": {"v1": "R","v2": "G","v3": "R"},"cost": -0.1,...

With other algorithms:

$ pydcop --timeout 2 solve --algo dsa graph_coloring.yaml$ pydcop solve --algo mgm --algo_params stop_cycle:20 \

graph_coloring.yaml

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 10

Page 11: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IResults

Full results :

{"agt_metrics": {...},"assignment": {"v1": "R","v2": "G","v3": "R"},"cost": -0.1,"cycle": 20,"msg_count": 158,"msg_size": 158,"status": "FINISHED","time": 0.03201029699994251,"violation": 0}

Look at results from mgm and dsa, compared to dpop’s results !

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 11

Page 12: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP ILogs

Simple:use -v 0..3

$ pydcop -v 3 solve --algo dsa --algo_params stop_cycle:20 graph_coloring.yaml

Precise :use -log <log.conf>

$ pydcop --log log.conf solve --algo dsa --algo_params stop_cycle:10graph_coloring.yaml

Now, look at algo.log

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 12

Page 13: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IRun-time metrics

periodic: "--collect_on period --period <p>"

$ pydcop --log log.conf -t 10 solve \--collect_on period --period 1 --run_metric ./metrics.csv \--algo dsa graph_coloring.yaml

cycle: "--collect_on cycle_change"Only supported with synchronous algorithms !

$ pydcop solve --algo mgm --algo_params stop_cycle:20 \--collect_on cycle_change --run_metric ./metrics.csv \graph_coloring_50.yaml

value: "--collect_on value_change"

$ pydcop -t 5 solve --algo mgm --collect_on value_change \--run_metric ./metrics_on_value.csv \graph_coloring_50.yaml

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 13

Page 14: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IRun-time metrics

With a bigger graph coloring problem

$ pydcop solve --algo mgm --algo_params stop_cycle:20 \--collect_on cycle_change \--run_metric ./metrics.csv \graph_coloring_50.yaml

Plotting with matplotlib

$ python3 plot_cost.py ./metrics.csv

Do the same thing with DSA, look at the result, what do you see ?Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 14

Page 15: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IRun-time metrics

MGM (1720) and DSA (1647) , both with 30 cycles

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 15

Page 16: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IWeb-ui

Web-base agent graphical interface:

Run the web application

$ cd ~/pydcop-ui$ python3 -m http.server

Launch a browser on http://127.0.0.1:8000

Solve the dcop with the option --uiport <port> (also, use --delay <delay>)

$ pydcop -v 3 solve -a mgm -d adhoc --delay 2 --uiport 10000./graph_coloring_3agts_10vars.yaml

Each agent exposes a web-socket, the web application connects to thesewebsockets and display the agents’ state.

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 16

Page 17: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IWeb-ui

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 17

Page 18: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IWeb-ui

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 18

Page 19: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Menu

Hands on PyDCOP I

Hands on PyDCOP II

Focus on Smart Environment Con�guration Problems

Distributing Computations

Hands on PyDCOP III

Dynamic DCOPs

Conclusion

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 19

Page 20: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IIDevelopping with pyDCOP

pyDCOP is designed to make it easy to implement new DCOP algorithmsAll the infrastructure is provided:I agents,I messaging,I metrics,I etc.

Base classes and utility functions forI constraints,I variables,I domains,I etc.

Plugin mechanism to de�ne new algorithms for DCOP, distribution and replication.

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 20

Page 21: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Implementing a DCOP algorithm with pyDCOPSimple DSA implementation

Create a new python module in pydcop.algorithms

De�ne a constant indicating the graphical representation used by your algorithm :GRAPH_TYPE = ’constraints_hypergraph’

De�ne your message(s):message_type(<name>, [fields]);

Subclass VariableComputation:I Annotate your message handler(s) with @register(<name>)I Send messages to your neighbors using self.post_msg orself.post_to_all_neighbors

I Select a new value with self.value_selectionI Start a new cycle with self.new_cycle()

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 21

Page 22: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IISimple DSA implementation

One class, 3 main methods:

1 GRAPH_TYPE = ’constraints_hypergraph’2 algo_params = []3

4 DsaMessage = message_type("dsa_value", ["value"])5

6 class DsaTutoComputation(VariableComputation):7

8 def __init__(self, computation_definition):9 ...

10

11 def on_start(self):12 ...13

14 @register("dsa_value")15 def on_value_msg(self, variable_name, recv_msg, t):16 ...17

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 22

Page 23: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IISimple DSA implementation

Creating the computation instance.a computation_definition contains: constraints, variable, neighbors, parameters,etc.

1 class DsaTutoComputation(VariableComputation):2

3 def __init__(self, variable, constraints, computation_definition):4 super() .__init__(computation_definition.node.variable,5 computation_definition)6

7

8 self.constraints = computation_definition.node.constraints9 self.current_cycle = {}

10 self.next_cycle = {}11

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 23

Page 24: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IISimple DSA implementation

On startup, select a value at random and send it to all neighbors.

1 def on_start(self):2 self.random_value_selection()3 self.post_to_all_neighbors(DsaMessage(self.current_value))4 self.evaluate_cycle()5

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 24

Page 25: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IISimple DSA implementation

Receiving values from our neighborsDSA is a synchronous algorithm !

1 @register("dsa_value")2 def on_value_msg(self, variable_name, recv_msg, t):3

4 if variable_name not in self.current_cycle:5 self.current_cycle[variable_name] = recv_msg.value6 self.evaluate_cycle()7

8 else: # The message is for the next cycle9 self.next_cycle[variable_name] = recv_msg.value

10

11

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 25

Page 26: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IISimple DSA implementation

The actual DSA implementation: select a new value if the gain is positive.

1 def evaluate_cycle(self):2

3 self.current_cycle[self.variable.name] = self.current_value4 current_cost = assignment_cost(self.current_cycle, self.constraints)5 arg_min, min_cost = self.compute_best_value()6

7 if current_cost - min_cost > 0 and 0.5 > random.random():8 self.value_selection(arg_min)9

10 self.new_cycle()11 self.current_cycle, self.next_cycle = self.next_cycle, {}12 self.post_to_all_neighbors(DsaMessage(self.current_value))13

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 26

Page 27: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IISimple DSA implementation

Last utility method: �nd the best value given the value from the neighbors.

1 def compute_best_value(self):2 arg_min, min_cost = None, float(’inf’)3 for value in self.variable.domain:4 self.current_cycle[self.variable.name] = value5 cost = assignment_cost(self.current_cycle, self.constraints)6 if cost < min_cost:7 min_cost, arg_min = cost, value8 return arg_min, min_cost9

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 27

Page 28: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IISimple DSA implementation

We can now use this new algorithm directly through the command line interface(except for stop_cycle:20):

$ pydcop --log log.conf -t 20 solve --algo dsatuto \--collect_on value_change \--run_metric ./metrics_tuto.csv \graph_coloring_50.yaml

Of course, it also works with the metrics, web-ui, etc.

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 28

Page 29: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Menu

Hands on PyDCOP I

Hands on PyDCOP II

Focus on Smart Environment Con�guration Problems

Distributing Computations

Hands on PyDCOP III

Dynamic DCOPs

Conclusion

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 29

Page 30: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

SECP modelSmart Environment Con�guration Problem [Rust et al., 2016]

Example of applying DCOPs to a "real"problem

Coordinate objects in the buildingModelI objectsI relations between objects and

environmentI user objectives and requirements

Formulate the problem as anoptimization problem

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 30

Page 31: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

SECP modelSmart Environment Con�guration Problem [Rust et al., 2016]

Focus on smart lighting use cases

Objects: anything that can produce light: light bulbs, windows with rolling shutter,etc.

User preferences: having a prede�ned luminosity level in a room, under someconditions

Energy e�ciency

Linking objects and user preferences:

How to model the luminosity in a room ? variable

How to model the dependency between the light sources and the luminosity ?function / constraint

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 31

Page 32: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

SECP modelExample application to ambient intelligence scenario

ActuatorsI Connected light bulbs, TV, Rolling shutters, ...

SensorsI Presence detector, Luminosity Sensor, etc.

Physical Dependency ModelsI E.g. Living-room light model

User PreferencesI Expressed as rules :

IF presence_living_room = 1

AND light_sensor_living_room < 60

THEN light_level_living_room ← 60

AND shutter_living_room ← 0

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 32

Page 33: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

SECP modelExample application to ambient intelligence scenario

ActuatorsI Decision variable xi, domain DxiI Cost function ci : Dxi → R

SensorsI Read-only variable sl, domain Dsl

Physical Dependency Models 〈yj , φj〉I Give the expected state of the environment from a set of

actuator-variables in�uencing this modelI Variable yj representing the expected state of the

environmentI Function φj :

∏ς∈σ(φj)Dς → Dyj

User PreferencesI Utility function ukI Distance from the current expected state to the target state

of the environment

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 32

Page 34: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Formulating SECP as a DCOP

Multi-objective optimization problem

minxi∈ν(A)

∑i∈A

ci and maxxi∈ν(A)

yj∈ν(Φ)

∑k∈R

uk

s.t. φj(x1j , . . . , x

φj

j ) = yj ∀yj ∈ ν(Φ)

Then mono-objective DCOP formulation

maxxi∈ν(A)

yj∈ν(Φ)

ωu∑k∈R

uk − ωc∑i∈A

ci +∑ϕj∈Φ

ϕj

with reformulation of hard constraints φj into soft ones:

ϕj(x1j , . . . , x

|σ(φj)|j , yj) =

{0 if φj(x

1j , . . . , x

|σ(φj)|j ) = yj

−∞ otherwise

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 33

Page 35: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Formulating SECP as a DCOPRepresenting a DCOP as a factor graph

x1

x2

x3

c1

c2

c3

ϕ1 y1 u1

s1

Factor graph: Bipartite graph with nodes for variables and constraints

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 34

Page 36: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

SECP Factor Graphin a house (without rules)

Desk

LivingRoom

TV

Kitchen

Entrance

Stairs

ld1

ld2llv3

llv2

llv1ltv1

ltv2 ltv3

lk1 lk2

lk3

le1

ls1

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 35

Page 37: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Menu

Hands on PyDCOP I

Hands on PyDCOP II

Focus on Smart Environment Con�guration Problems

Distributing Computations

Hands on PyDCOP III

Dynamic DCOPs

Conclusion

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 36

Page 38: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Distribution of computationsAllocating computations to agents

DCOP: 〈A,X ,D, C, µ〉µ : function mapping variables to their associated agent

Why is distribution needed ?

Common assumptions:

computation ≡ variable

each agent controls exactly onevariable (bijection)

binary constraints

Real distributed problems:

agents must be hosted on realdevices

the set of devices might be given bythe problem

for some variables the relation withan agent is obvious, but not always

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 37

Page 39: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Distribution of computations

Several graph representations for the same DCOP.

Nodes in the graph = computations

x1

x2 x3

x4

(a) Simple constraint graph

x1

x2 x3

x4

(b) Factor graph

x1

f123x2 x3

x4f24

(c) Factor graph

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 38

Page 40: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Distribution computations

Computations

belong to an agent :"natural" link,problem characteristics

shared decisions

factors, in a factor graph:

a1

a2 a3

a4

x1

x2 x3

x4

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 39

Page 41: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Distribution computations

Computations

belong to an agent

shared decisions :modeling artifact, with no obvious agentrelation (e.g. distributed meeting scheduling,SECP, etc.)

factors, in a factor graph:

a1

a2

a4

x1

x2 x3

x4

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 39

Page 42: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Distribution computations

Computations

belong to an agent

shared decisions :modeling artifact, with no obvious agentrelation (e.g. distributed meeting scheduling,SECP, etc.)

factors, in a factor graph:

a2

a1

a4

x1

x2 x3

x4

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 39

Page 43: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Distribution computations

Computations

belong to an agent

shared decisions

factors, in a factor graph:not representing a decision variable

a1

a2a3

a4

x1

f123x2 x3

x4f24

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 39

Page 44: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Distribution computations

Computations

belong to an agent

shared decisions

factors, in a factor graph:not representing a decision variable

a1

a2 a3

a4

x1

f123x2 x3

x4f24

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 39

Page 45: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Distribution of computationsAllocating computations to agents

Distributing computationsI computations depends on the graph model used by the algorithmI variables and / or factors

Distribution impacts the system characteristicsI speed,I communication load,I hosting costs, etc.

Computing a distributionI heuristicsI optimal ?

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 40

Page 46: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Distribution of computationsOptimal de�nition

Optimal distribution ?Problem dependentOptimization problem :�nd the best distribution, for your problem’s criteriaOptimal distribution ≡ graph partitioning,NP-hard in general [Boulle, 2004]

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 41

Page 47: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Distribution of computationsBetter de�nition

SECP distribution problem

Devices have limited memory

Communication is expensive and has limited bandwidth

Variable related to an actuator are hosted by it

Objective : minimize overall communication between agents

Optimization problem : de�ne an ILP for it !

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 42

Page 48: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Binary ILP for computation distribution

xki , binary variables that map computations to agents and αmnij = xmi · fnj

∀xi ∈ X,∑am∈A

xmi = 1 (1)

Message’s size between variable xi and factor fj : msg(i, j)

minimizexmi

∑(i,j)∈D

∑(m,n)∈A2

msg(i, j) · αmnij (2)

Memory footprint of a computation: weight(e) , and memory capacity for adevice: cap(ak)

∀am ∈ A,∑xi∈D

weight(xi) · xmi ≤ cap(am) (3)

and a few linearization constraints

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 43

Page 49: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Binary ILP for computation distribution

More generic case:

Add route cost: com(i, j,m, n)

∀xi, xj ∈ X, ∀am, an ∈ A,

com(i, j,m, n) =

{msg(i, j) · route(m,n) if (i, j) ∈ D,m 6= n0 otherwise (4)

minimizexmi

∑(i,j)∈D

∑(m,n)∈A2

com(i, j,m, n) · αmnij (5)

Add hosting costs : host(am, xi)

minimizexmi

∑(xi,am)∈X×A

xmi · host(am, xi) (6)

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 44

Page 50: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Binary ILP for computation distribution

minimizexmi

ωcom ·∑

(i,j)∈D

∑(m,n)∈A2

com(i, j,m, n) · αmnij

+ ωhost ·∑

(xi,am)∈X×A

xmi · host(am, xi) (7)

subject to

∀am ∈ A,∑xi∈D

weight(xi) · xmi ≤ cap(am) (8)

∀xi ∈ X,∑am∈A

xmi = 1 (9)

∀xi ∈ X, αmnij ≤ xmi (10)

∀xj ∈ X, αmnij ≤ xmj (11)

∀xi, xj ∈ X, am ∈ A, αmnij ≥ xmi + xnj − 1 (12)

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 45

Page 51: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Solving the ILP for computation deployment

NP-hard, but can be solved with branch-and-cutLP solvers are very good at thisYet, only possible for small instancesGives us a reference for optimality: benchmarkingWhen not solvable, still gives us a metrics to compare heuristics

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 46

Page 52: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Menu

Hands on PyDCOP I

Hands on PyDCOP II

Focus on Smart Environment Con�guration Problems

Distributing Computations

Hands on PyDCOP III

Dynamic DCOPs

Conclusion

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 47

Page 53: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP III

Distribution and deployment

1. Deploy on several machineshttps:

//pydcop.readthedocs.io/en/latest/tutorials/deploying_on_machines.html

2. Running a single agenthttps://pydcop.readthedocs.io/en/latest/usage/cli/run.html

3. Distributing computations / taskshttps://pydcop.readthedocs.io/en/latest/usage/cli/distribute.html

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 48

Page 54: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IIISECP

A Very simple SECP: single room

3 light bulbs, 1 model and 3 rules

/tutorials/hands-on_3/single_room.yaml

Solve with

pydcop --log log.conf -t 10 solve \--algo maxsum --algo_params damping:0.8 \--dist adhoc single_room.yaml

Result : "cost": 702.3000000000004, ...I not that good ...I Look at the yaml de�nitionI the rules contradict each other !

Change the yaml de�nitionI comment out rules to keep only one activeI could be done with ’read-only’ variablesI solve it again

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 49

Page 55: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IIISECP - Running on several machines

We used solveI great for testingI everything run locally, in the same process

Launching several agents:I One agent for each light bulb a1, a2 and a3 (change port for each agent)

pydcop -v3 agent -n a1 -p 9001 \--orchestrator 127.0.0.1:9000

I an orchestrator

pydcop --log log.conf -t 10 orchestrator \--algo maxsum --algo_params damping:0.8 \--dist adhoc single_room.yaml

I run the agents on di�erent Virtual machines, di�erent computers

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 50

Page 56: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IIIA bigger SECP

in /tutorials/hands-on_3/SimpleHouse.yml13 light bulbs, 6 models

Desk

LivingRoom

TV

Kitchen

Entrance

Stairs

ld1

ld2llv3

llv2

llv1ltv1

ltv2 ltv3

lk1 lk2

lk3

le1

ls1

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 51

Page 57: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IIIDistributing a SECP

$ pydcop --output dist_house_fg_ilp.yaml distribute -d ilp_compref \-a maxsum SimpleHouse.yml

Need to specify the algorithm, used to deduce:

the computation graph

the computations’ weight

the size of computations’ messages

On such a small system, we can compute the optimal distribution !

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 52

Page 58: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Hands on PyDCOP IIIDistributing a SECP

cost: 8725.0distribution:a_d1: [mv_desk, mc_desk, l_d1, r_work, mc_livingroom, mv_livingroom]a_d2: [l_d2]a_e1: [mv_entry, r_entry, mv_stairs, l_e1, mc_entry, mc_stairs]a_e2: [l_e2]a_k1: [l_k1]a_k2: [l_k2]a_k3: [l_k3]a_lv1: [l_lv1]a_lv2: [mc_kitchen, l_lv2]a_lv3: [l_lv3]a_tv1: [l_tv1]a_tv2: [l_tv2]a_tv3: [r_lunch, l_tv3, mv_tv, r_cooking, r_homecinema, mc_tv, mv_kitchen

]inputs:algo: maxsumdcop: [SimpleHouse.yml]dist_algo: ilp_comprefgraph: factor_graph

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 53

Page 59: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Menu

Hands on PyDCOP I

Hands on PyDCOP II

Focus on Smart Environment Con�guration Problems

Distributing Computations

Hands on PyDCOP III

Dynamic DCOPs

Conclusion

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 54

Page 60: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

SECP and DCOP

So far we have:

Designed a model for SECP

Formulated this model as a DCOP

Distributed the computation of the DCOP on devices / agents (bootstrap)

Run our system to get self-con�gured devices

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 55

Page 61: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

But what happensin dynamic environments ?

if objects appear and disappear?

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 56

Page 62: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

SECP is a dynamic problem

Dynamics in the infrastructure

Devices can disappear

New devices can be added to thesystem

At runtime..

No powerful device available to solvethe ILP

The deployment must be repaired:self-adaptation

Only consider a portion of the factorgraph: the neighborhood

c1 x1 u1

a1

c2 x2 ϕ1 y1

a2

c3 x3 ϕ2

a3

c4 x4 y2 u2

a4

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 57

Page 63: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

k-resilienceDynamics in the infrastruture

De�nition (k-resiliency)k-resiliency is the capacity for a system to repair itself and operate correctly even inthe case of the disappearance of up to k agents

Two parts:I Do not loose the de�nition of the computations: replicationI Migrate the orphaned computations to another agent: selection / activation

Apply to any graph of computations, not only DCOP

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 58

Page 64: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Replication of computations

Replica distribution

For each computation, place k replica on k other agentsreplica equiv de�nition of the computation

Must be distributed !

Optimal replication ? impact the set of available agents when repairingwhich criteria ? too hard (quadratic multiple knapsack problem)...

Distributed Replica Placement Method (DRPM)

Heuristic : place replica on agents close (network) the active computation, whilerespecting capacity

Distributed version of iterative lengthening (aka uniform cost search based onpath costs)

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 59

Page 65: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Replication of computationsiterative lengthening on route and hosting costs

a1

a2 a3

a4

route(a1, a2) = 1

route(a2, a3) = 3

route(a2, a4) = 1

route(a1, a4) = 1

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60

Page 66: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Replication of computationsiterative lengthening on route and hosting costs

a1

a2 a3

a4

a2 a3

a4

host(a2, xi) = 1 host(a3, xi) = 1

host(a4, xi) = 5

route(a1, a2) = 1

route(a2, a3) = 3

route(a2, a4) = 1

route(a1, a4) = 1

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60

Page 67: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Replication of computationsiterative lengthening on route and hosting costs

a1

a2 a3

a4

a2 a3

a4

host(a2, xi) = 1 host(a3, xi) = 1

host(a4, xi) = 5

route(a1, a2) = 1

route(a2, a3) = 3

route(a2, a4) = 1

route(a1, a4) = 1

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60

Page 68: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Replication of computationsiterative lengthening on route and hosting costs

a1

a2 a3

a4

a2 a3

a4

host(a2, xi) = 1 host(a3, xi) = 1

host(a4, xi) = 5

route(a1, a2) = 1

route(a2, a3) = 3

route(a2, a4) = 1

route(a1, a4) = 1

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60

Page 69: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Replication of computationsiterative lengthening on route and hosting costs

a1

a2 a3

a4

a2 a3

a4

host(a2, xi) = 1 host(a3, xi) = 1

host(a4, xi) = 5

route(a1, a2) = 1

route(a2, a3) = 3

route(a2, a4) = 1

route(a1, a4) = 1

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60

Page 70: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Replication of computationsiterative lengthening on route and hosting costs

a1

a2 a3

a4

a2 a3

a4

host(a2, xi) = 1 host(a3, xi) = 1

host(a4, xi) = 5

route(a1, a2) = 1

route(a2, a3) = 3

route(a2, a4) = 1

route(a1, a4) = 1

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60

Page 71: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Replication of computationsiterative lengthening on route and hosting costs

a1

a2 a3

a4

a2 a3

a4

host(a2, xi) = 1 host(a3, xi) = 1

host(a4, xi) = 5

route(a1, a2) = 1

route(a2, a3) = 3

route(a2, a4) = 1

route(a1, a4) = 1

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60

Page 72: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Replication of computationsiterative lengthening on route and hosting costs

a1

a2 a3

a4

a2 a3

a4

host(a2, xi) = 1 host(a3, xi) = 1

host(a4, xi) = 5

route(a1, a2) = 1

route(a2, a3) = 3

route(a2, a4) = 1

route(a1, a4) = 1

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60

Page 73: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Migrating computationsSelecting an agent

Migrating a set of xi computations Xc

set of candidate agents Acmigrating the computation must not exceed agent’s capacity

for each computation, select the agent that minimize hosting and communicationcost

Same optimization problem than for initial distribution, but on a subset of the of thegraph

Distributed process !

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 61

Page 74: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Migrating computationsSelecting an agent

Distributed optimization problem ⇒ let’s use a DCOP!A is the set of candidate agents AcX are the binary decision variables xmiC are the constraints ensuring that all computations are hosted, agent’s capacitiesare respected and hosting and communication costs are minimized

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 62

Page 75: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Migrating computationsSelecting an agent

∑am∈Ai

c

xmi = 1 (13)

∑xi∈Xm

c

weight(xi) · xmi +∑

xj∈µ−1(am)\Xc

weight(xj) ≤ cap(am) (14)

∑xi∈Xm

c

host(am, xi) · xmi (15)

∑(xi,xj)∈Xm

c ×Ni\Xc

xmi · com(i, j,m, µ−1(xj))

+∑

(xi,xj)∈Xmc ×Ni∩Xc

xmi ·∑

an∈Ajc

xnj · com(i, j,m, n)) (16)

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 63

Page 76: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Decentralized reparation

When agents are removed:

computation to migrate = computation that were hosted on these agents

candidate agents = remaining agents that posses a replica of these orphanedcomputation

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 64

Page 77: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Solving the migration DCOPWhich algorithm should we use ?

Criteria:

lightweight

fast (even if not optimal !)

monotonic : mix of hard and soft constraints

MGM-2 : like MGM, with 2-coordination

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 65

Page 78: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

How does it behave, experimentally?

400

500

600

700

800

900

1000

colo

ring

scal

efre

e

uniform

400

500

600

700

800

900

1000problem dependent

5000

5500

6000

6500

7000

colo

ring

rand

om

5000

5500

6000

6500

7000

0 25 50 75 100 125 150 175time (s)

125

100

75

50

25

0

25

ising

0 25 50 75 100 125 150 175time (s)

120100806040200

20

single instance cost with pertubation average cost with pertubation average cost without pertubation

DSA

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 66

Page 79: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

How does it behave, experimentally? (cont.)

80

100

120

140

160

180

colo

ring

scal

efre

e

uniform

80

100

120

140

160

problem dependent

250

300

350

400

450

colo

ring

rand

om

250

300

350

400

0 25 50 75 100 125 150 175time (s)

30

25

20

15

10

5

0

ising

0 25 50 75 100 125 150 175time (s)

30

20

10

0

single instance cost with pertubation average cost with pertubation average cost without pertubation

MaxSum

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 67

Page 80: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

Menu

Hands on PyDCOP I

Hands on PyDCOP II

Focus on Smart Environment Con�guration Problems

Distributing Computations

Hands on PyDCOP III

Dynamic DCOPs

Conclusion

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 68

Page 81: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

To sum up

What we’ve seen today:

Some generic conceptsI How to model coordination problems using DCOP formalismI Some solution methods (complete and incomplete) to solve DCOP

Some speci�cities of IoT-based appsI How to model a speci�c smart environment con�guration problem as a DCOPI How to use PyDCOP to model, run, solve, and distribute DCOPI How to equip a system with resilience using replication and DCOP-based reparation

Want to go deeper into DCOPs→ OPTMAS-DCR workshop series (AAMAS/IJCAI),other tutorials at AAMAS/IJCAI

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 69

Page 82: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

The End of Hands on PyDCOP

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 70

Page 83: AAMAS-19 Tutorial on: MULTI-AGENT DISTRIBUTED …fferdinando3/cfp/AAMAS19/tutorial-P… · 5 cost = assignment_cost(self.current_cycle, self.constraints) 6 ifcost < min_cost: 7 min_cost,

References

Boulle, M. (2004). “Compact Mathematical Formulation for Graph Partitioning”. In: Optimization and Engineering 5.3,pp. 315–333. issn: 1573-2924. doi: 10.1023/B:OPTE.0000038889.84284.c7. url:http://dx.doi.org/10.1023/B:OPTE.0000038889.84284.c7.

Rust, P., G. Picard, and F. Ramparany (2016). “Using Message-passing DCOP Algorithms to Solve Energy-e�cient SmartEnvironment Con�guration Problems”. In: International Joint Conference on Arti�cial Intelligence (IJCAI). AAAI Press.

Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 71