aamas-19 tutorial on: multi-agent distributed …fferdinando3/cfp/aamas19/tutorial-p… · 5 cost =...
TRANSCRIPT
AAMAS-19 Tutorial on:
MULTI-AGENT DISTRIBUTEDCONSTRAINED OPTIMIZATION
Hands on pyDCOP
Pierre Rust Gauthier Picard
Menu
Hands on PyDCOP I
Hands on PyDCOP II
Focus on Smart Environment Con�guration Problems
Distributing Computations
Hands on PyDCOP III
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 2
Menu
Hands on PyDCOP I
Hands on PyDCOP II
Focus on Smart Environment Con�guration Problems
Distributing Computations
Hands on PyDCOP III
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 3
Hands on PyDCOP I
Install VirtualBoxImport the pyDCOP Virtual Machine (http://bit.ly/pyDCOP)I It’s a Debian image with everything preinstalled:I python3, pyDCOP, matplotlib, glpk, etc.
Alternatively, followhttps://pydcop.readthedocs.io/en/latest/installation.html
1. https://pydcop.readthedocs.io/en/latest/tutorials/getting_started.html
2. https://pydcop.readthedocs.io/en/latest/tutorials/analysing_results.html
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 4
Hands on PyDCOP IVirtual machine Setup
Before starting the VM:
"Bridged adapter" mode
Select wi� network adapter
Reset MAC Address
Then
Start the VM
login: dcop / pyDCOP
Launch a terminal
Note down the IP with ipaddress
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 5
Hands on PyDCOP IVirtual machine Setup
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 6
Hands on PyDCOP IFiles for the tutorials are in /home/dcop/tutorials.
$ cd /home/dcop/tutorials/hands-on_1
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 7
Hands on PyDCOP IDCOP - Graph Coloring
v2
v1 v3
(a) constraints graph
p2 v2
d1,2
v1
d2,3
v3p1 p3
(b) factor graph
Objective: minimizeDomain: 2 colors R and BVariables: V1, V2, V3
Constraints: neighbors must have di�erent colors + preferencesAgents: 3 agents
Yaml representation
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 8
Hands on PyDCOP IpyDCOP yaml format
graph_coloring.yaml
name: graph coloringobjective: min
domains:colors:values: [R, G]
variables:v1:domain: colorsv2:domain: colorsv3:domain: colors
constraints:pref_1:type: extensionalvariables: v1values:-0.1: R0.1: G
pref_2:type: extensionalvariables: v2values:-0.1: G0.1: R
pref_3:type: extensionalvariables: v3values:-0.1: G0.1: R
diff_1_2:type: intentionfunction: 10 if v1 == v2 else 0
diff_2_3:type: intentionfunction: 10 if v3 == v2 else 0
agents: [a1, a2, a3, a4, a5]
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 9
Hands on PyDCOP ISolving the Graph Coloring DCOP
Command:
$ pydcop solve --algo dpop graph_coloring.yaml
Output:
..."assignment": {"v1": "R","v2": "G","v3": "R"},"cost": -0.1,...
With other algorithms:
$ pydcop --timeout 2 solve --algo dsa graph_coloring.yaml$ pydcop solve --algo mgm --algo_params stop_cycle:20 \
graph_coloring.yaml
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 10
Hands on PyDCOP IResults
Full results :
{"agt_metrics": {...},"assignment": {"v1": "R","v2": "G","v3": "R"},"cost": -0.1,"cycle": 20,"msg_count": 158,"msg_size": 158,"status": "FINISHED","time": 0.03201029699994251,"violation": 0}
Look at results from mgm and dsa, compared to dpop’s results !
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 11
Hands on PyDCOP ILogs
Simple:use -v 0..3
$ pydcop -v 3 solve --algo dsa --algo_params stop_cycle:20 graph_coloring.yaml
Precise :use -log <log.conf>
$ pydcop --log log.conf solve --algo dsa --algo_params stop_cycle:10graph_coloring.yaml
Now, look at algo.log
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 12
Hands on PyDCOP IRun-time metrics
periodic: "--collect_on period --period <p>"
$ pydcop --log log.conf -t 10 solve \--collect_on period --period 1 --run_metric ./metrics.csv \--algo dsa graph_coloring.yaml
cycle: "--collect_on cycle_change"Only supported with synchronous algorithms !
$ pydcop solve --algo mgm --algo_params stop_cycle:20 \--collect_on cycle_change --run_metric ./metrics.csv \graph_coloring_50.yaml
value: "--collect_on value_change"
$ pydcop -t 5 solve --algo mgm --collect_on value_change \--run_metric ./metrics_on_value.csv \graph_coloring_50.yaml
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 13
Hands on PyDCOP IRun-time metrics
With a bigger graph coloring problem
$ pydcop solve --algo mgm --algo_params stop_cycle:20 \--collect_on cycle_change \--run_metric ./metrics.csv \graph_coloring_50.yaml
Plotting with matplotlib
$ python3 plot_cost.py ./metrics.csv
Do the same thing with DSA, look at the result, what do you see ?Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 14
Hands on PyDCOP IRun-time metrics
MGM (1720) and DSA (1647) , both with 30 cycles
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 15
Hands on PyDCOP IWeb-ui
Web-base agent graphical interface:
Run the web application
$ cd ~/pydcop-ui$ python3 -m http.server
Launch a browser on http://127.0.0.1:8000
Solve the dcop with the option --uiport <port> (also, use --delay <delay>)
$ pydcop -v 3 solve -a mgm -d adhoc --delay 2 --uiport 10000./graph_coloring_3agts_10vars.yaml
Each agent exposes a web-socket, the web application connects to thesewebsockets and display the agents’ state.
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 16
Hands on PyDCOP IWeb-ui
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 17
Hands on PyDCOP IWeb-ui
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 18
Menu
Hands on PyDCOP I
Hands on PyDCOP II
Focus on Smart Environment Con�guration Problems
Distributing Computations
Hands on PyDCOP III
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 19
Hands on PyDCOP IIDevelopping with pyDCOP
pyDCOP is designed to make it easy to implement new DCOP algorithmsAll the infrastructure is provided:I agents,I messaging,I metrics,I etc.
Base classes and utility functions forI constraints,I variables,I domains,I etc.
Plugin mechanism to de�ne new algorithms for DCOP, distribution and replication.
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 20
Implementing a DCOP algorithm with pyDCOPSimple DSA implementation
Create a new python module in pydcop.algorithms
De�ne a constant indicating the graphical representation used by your algorithm :GRAPH_TYPE = ’constraints_hypergraph’
De�ne your message(s):message_type(<name>, [fields]);
Subclass VariableComputation:I Annotate your message handler(s) with @register(<name>)I Send messages to your neighbors using self.post_msg orself.post_to_all_neighbors
I Select a new value with self.value_selectionI Start a new cycle with self.new_cycle()
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 21
Hands on PyDCOP IISimple DSA implementation
One class, 3 main methods:
1 GRAPH_TYPE = ’constraints_hypergraph’2 algo_params = []3
4 DsaMessage = message_type("dsa_value", ["value"])5
6 class DsaTutoComputation(VariableComputation):7
8 def __init__(self, computation_definition):9 ...
10
11 def on_start(self):12 ...13
14 @register("dsa_value")15 def on_value_msg(self, variable_name, recv_msg, t):16 ...17
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 22
Hands on PyDCOP IISimple DSA implementation
Creating the computation instance.a computation_definition contains: constraints, variable, neighbors, parameters,etc.
1 class DsaTutoComputation(VariableComputation):2
3 def __init__(self, variable, constraints, computation_definition):4 super() .__init__(computation_definition.node.variable,5 computation_definition)6
7
8 self.constraints = computation_definition.node.constraints9 self.current_cycle = {}
10 self.next_cycle = {}11
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 23
Hands on PyDCOP IISimple DSA implementation
On startup, select a value at random and send it to all neighbors.
1 def on_start(self):2 self.random_value_selection()3 self.post_to_all_neighbors(DsaMessage(self.current_value))4 self.evaluate_cycle()5
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 24
Hands on PyDCOP IISimple DSA implementation
Receiving values from our neighborsDSA is a synchronous algorithm !
1 @register("dsa_value")2 def on_value_msg(self, variable_name, recv_msg, t):3
4 if variable_name not in self.current_cycle:5 self.current_cycle[variable_name] = recv_msg.value6 self.evaluate_cycle()7
8 else: # The message is for the next cycle9 self.next_cycle[variable_name] = recv_msg.value
10
11
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 25
Hands on PyDCOP IISimple DSA implementation
The actual DSA implementation: select a new value if the gain is positive.
1 def evaluate_cycle(self):2
3 self.current_cycle[self.variable.name] = self.current_value4 current_cost = assignment_cost(self.current_cycle, self.constraints)5 arg_min, min_cost = self.compute_best_value()6
7 if current_cost - min_cost > 0 and 0.5 > random.random():8 self.value_selection(arg_min)9
10 self.new_cycle()11 self.current_cycle, self.next_cycle = self.next_cycle, {}12 self.post_to_all_neighbors(DsaMessage(self.current_value))13
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 26
Hands on PyDCOP IISimple DSA implementation
Last utility method: �nd the best value given the value from the neighbors.
1 def compute_best_value(self):2 arg_min, min_cost = None, float(’inf’)3 for value in self.variable.domain:4 self.current_cycle[self.variable.name] = value5 cost = assignment_cost(self.current_cycle, self.constraints)6 if cost < min_cost:7 min_cost, arg_min = cost, value8 return arg_min, min_cost9
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 27
Hands on PyDCOP IISimple DSA implementation
We can now use this new algorithm directly through the command line interface(except for stop_cycle:20):
$ pydcop --log log.conf -t 20 solve --algo dsatuto \--collect_on value_change \--run_metric ./metrics_tuto.csv \graph_coloring_50.yaml
Of course, it also works with the metrics, web-ui, etc.
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 28
Menu
Hands on PyDCOP I
Hands on PyDCOP II
Focus on Smart Environment Con�guration Problems
Distributing Computations
Hands on PyDCOP III
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 29
SECP modelSmart Environment Con�guration Problem [Rust et al., 2016]
Example of applying DCOPs to a "real"problem
Coordinate objects in the buildingModelI objectsI relations between objects and
environmentI user objectives and requirements
Formulate the problem as anoptimization problem
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 30
SECP modelSmart Environment Con�guration Problem [Rust et al., 2016]
Focus on smart lighting use cases
Objects: anything that can produce light: light bulbs, windows with rolling shutter,etc.
User preferences: having a prede�ned luminosity level in a room, under someconditions
Energy e�ciency
Linking objects and user preferences:
How to model the luminosity in a room ? variable
How to model the dependency between the light sources and the luminosity ?function / constraint
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 31
SECP modelExample application to ambient intelligence scenario
ActuatorsI Connected light bulbs, TV, Rolling shutters, ...
SensorsI Presence detector, Luminosity Sensor, etc.
Physical Dependency ModelsI E.g. Living-room light model
User PreferencesI Expressed as rules :
IF presence_living_room = 1
AND light_sensor_living_room < 60
THEN light_level_living_room ← 60
AND shutter_living_room ← 0
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 32
SECP modelExample application to ambient intelligence scenario
ActuatorsI Decision variable xi, domain DxiI Cost function ci : Dxi → R
SensorsI Read-only variable sl, domain Dsl
Physical Dependency Models 〈yj , φj〉I Give the expected state of the environment from a set of
actuator-variables in�uencing this modelI Variable yj representing the expected state of the
environmentI Function φj :
∏ς∈σ(φj)Dς → Dyj
User PreferencesI Utility function ukI Distance from the current expected state to the target state
of the environment
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 32
Formulating SECP as a DCOP
Multi-objective optimization problem
minxi∈ν(A)
∑i∈A
ci and maxxi∈ν(A)
yj∈ν(Φ)
∑k∈R
uk
s.t. φj(x1j , . . . , x
φj
j ) = yj ∀yj ∈ ν(Φ)
Then mono-objective DCOP formulation
maxxi∈ν(A)
yj∈ν(Φ)
ωu∑k∈R
uk − ωc∑i∈A
ci +∑ϕj∈Φ
ϕj
with reformulation of hard constraints φj into soft ones:
ϕj(x1j , . . . , x
|σ(φj)|j , yj) =
{0 if φj(x
1j , . . . , x
|σ(φj)|j ) = yj
−∞ otherwise
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 33
Formulating SECP as a DCOPRepresenting a DCOP as a factor graph
x1
x2
x3
c1
c2
c3
ϕ1 y1 u1
s1
Factor graph: Bipartite graph with nodes for variables and constraints
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 34
SECP Factor Graphin a house (without rules)
Desk
LivingRoom
TV
Kitchen
Entrance
Stairs
ld1
ld2llv3
llv2
llv1ltv1
ltv2 ltv3
lk1 lk2
lk3
le1
ls1
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 35
Menu
Hands on PyDCOP I
Hands on PyDCOP II
Focus on Smart Environment Con�guration Problems
Distributing Computations
Hands on PyDCOP III
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 36
Distribution of computationsAllocating computations to agents
DCOP: 〈A,X ,D, C, µ〉µ : function mapping variables to their associated agent
Why is distribution needed ?
Common assumptions:
computation ≡ variable
each agent controls exactly onevariable (bijection)
binary constraints
Real distributed problems:
agents must be hosted on realdevices
the set of devices might be given bythe problem
for some variables the relation withan agent is obvious, but not always
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 37
Distribution of computations
Several graph representations for the same DCOP.
Nodes in the graph = computations
x1
x2 x3
x4
(a) Simple constraint graph
x1
x2 x3
x4
(b) Factor graph
x1
f123x2 x3
x4f24
(c) Factor graph
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 38
Distribution computations
Computations
belong to an agent :"natural" link,problem characteristics
shared decisions
factors, in a factor graph:
a1
a2 a3
a4
x1
x2 x3
x4
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 39
Distribution computations
Computations
belong to an agent
shared decisions :modeling artifact, with no obvious agentrelation (e.g. distributed meeting scheduling,SECP, etc.)
factors, in a factor graph:
a1
a2
a4
x1
x2 x3
x4
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 39
Distribution computations
Computations
belong to an agent
shared decisions :modeling artifact, with no obvious agentrelation (e.g. distributed meeting scheduling,SECP, etc.)
factors, in a factor graph:
a2
a1
a4
x1
x2 x3
x4
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 39
Distribution computations
Computations
belong to an agent
shared decisions
factors, in a factor graph:not representing a decision variable
a1
a2a3
a4
x1
f123x2 x3
x4f24
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 39
Distribution computations
Computations
belong to an agent
shared decisions
factors, in a factor graph:not representing a decision variable
a1
a2 a3
a4
x1
f123x2 x3
x4f24
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 39
Distribution of computationsAllocating computations to agents
Distributing computationsI computations depends on the graph model used by the algorithmI variables and / or factors
Distribution impacts the system characteristicsI speed,I communication load,I hosting costs, etc.
Computing a distributionI heuristicsI optimal ?
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 40
Distribution of computationsOptimal de�nition
Optimal distribution ?Problem dependentOptimization problem :�nd the best distribution, for your problem’s criteriaOptimal distribution ≡ graph partitioning,NP-hard in general [Boulle, 2004]
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 41
Distribution of computationsBetter de�nition
SECP distribution problem
Devices have limited memory
Communication is expensive and has limited bandwidth
Variable related to an actuator are hosted by it
Objective : minimize overall communication between agents
Optimization problem : de�ne an ILP for it !
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 42
Binary ILP for computation distribution
xki , binary variables that map computations to agents and αmnij = xmi · fnj
∀xi ∈ X,∑am∈A
xmi = 1 (1)
Message’s size between variable xi and factor fj : msg(i, j)
minimizexmi
∑(i,j)∈D
∑(m,n)∈A2
msg(i, j) · αmnij (2)
Memory footprint of a computation: weight(e) , and memory capacity for adevice: cap(ak)
∀am ∈ A,∑xi∈D
weight(xi) · xmi ≤ cap(am) (3)
and a few linearization constraints
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 43
Binary ILP for computation distribution
More generic case:
Add route cost: com(i, j,m, n)
∀xi, xj ∈ X, ∀am, an ∈ A,
com(i, j,m, n) =
{msg(i, j) · route(m,n) if (i, j) ∈ D,m 6= n0 otherwise (4)
minimizexmi
∑(i,j)∈D
∑(m,n)∈A2
com(i, j,m, n) · αmnij (5)
Add hosting costs : host(am, xi)
minimizexmi
∑(xi,am)∈X×A
xmi · host(am, xi) (6)
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 44
Binary ILP for computation distribution
minimizexmi
ωcom ·∑
(i,j)∈D
∑(m,n)∈A2
com(i, j,m, n) · αmnij
+ ωhost ·∑
(xi,am)∈X×A
xmi · host(am, xi) (7)
subject to
∀am ∈ A,∑xi∈D
weight(xi) · xmi ≤ cap(am) (8)
∀xi ∈ X,∑am∈A
xmi = 1 (9)
∀xi ∈ X, αmnij ≤ xmi (10)
∀xj ∈ X, αmnij ≤ xmj (11)
∀xi, xj ∈ X, am ∈ A, αmnij ≥ xmi + xnj − 1 (12)
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 45
Solving the ILP for computation deployment
NP-hard, but can be solved with branch-and-cutLP solvers are very good at thisYet, only possible for small instancesGives us a reference for optimality: benchmarkingWhen not solvable, still gives us a metrics to compare heuristics
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 46
Menu
Hands on PyDCOP I
Hands on PyDCOP II
Focus on Smart Environment Con�guration Problems
Distributing Computations
Hands on PyDCOP III
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 47
Hands on PyDCOP III
Distribution and deployment
1. Deploy on several machineshttps:
//pydcop.readthedocs.io/en/latest/tutorials/deploying_on_machines.html
2. Running a single agenthttps://pydcop.readthedocs.io/en/latest/usage/cli/run.html
3. Distributing computations / taskshttps://pydcop.readthedocs.io/en/latest/usage/cli/distribute.html
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 48
Hands on PyDCOP IIISECP
A Very simple SECP: single room
3 light bulbs, 1 model and 3 rules
/tutorials/hands-on_3/single_room.yaml
Solve with
pydcop --log log.conf -t 10 solve \--algo maxsum --algo_params damping:0.8 \--dist adhoc single_room.yaml
Result : "cost": 702.3000000000004, ...I not that good ...I Look at the yaml de�nitionI the rules contradict each other !
Change the yaml de�nitionI comment out rules to keep only one activeI could be done with ’read-only’ variablesI solve it again
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 49
Hands on PyDCOP IIISECP - Running on several machines
We used solveI great for testingI everything run locally, in the same process
Launching several agents:I One agent for each light bulb a1, a2 and a3 (change port for each agent)
pydcop -v3 agent -n a1 -p 9001 \--orchestrator 127.0.0.1:9000
I an orchestrator
pydcop --log log.conf -t 10 orchestrator \--algo maxsum --algo_params damping:0.8 \--dist adhoc single_room.yaml
I run the agents on di�erent Virtual machines, di�erent computers
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 50
Hands on PyDCOP IIIA bigger SECP
in /tutorials/hands-on_3/SimpleHouse.yml13 light bulbs, 6 models
Desk
LivingRoom
TV
Kitchen
Entrance
Stairs
ld1
ld2llv3
llv2
llv1ltv1
ltv2 ltv3
lk1 lk2
lk3
le1
ls1
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 51
Hands on PyDCOP IIIDistributing a SECP
$ pydcop --output dist_house_fg_ilp.yaml distribute -d ilp_compref \-a maxsum SimpleHouse.yml
Need to specify the algorithm, used to deduce:
the computation graph
the computations’ weight
the size of computations’ messages
On such a small system, we can compute the optimal distribution !
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 52
Hands on PyDCOP IIIDistributing a SECP
cost: 8725.0distribution:a_d1: [mv_desk, mc_desk, l_d1, r_work, mc_livingroom, mv_livingroom]a_d2: [l_d2]a_e1: [mv_entry, r_entry, mv_stairs, l_e1, mc_entry, mc_stairs]a_e2: [l_e2]a_k1: [l_k1]a_k2: [l_k2]a_k3: [l_k3]a_lv1: [l_lv1]a_lv2: [mc_kitchen, l_lv2]a_lv3: [l_lv3]a_tv1: [l_tv1]a_tv2: [l_tv2]a_tv3: [r_lunch, l_tv3, mv_tv, r_cooking, r_homecinema, mc_tv, mv_kitchen
]inputs:algo: maxsumdcop: [SimpleHouse.yml]dist_algo: ilp_comprefgraph: factor_graph
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 53
Menu
Hands on PyDCOP I
Hands on PyDCOP II
Focus on Smart Environment Con�guration Problems
Distributing Computations
Hands on PyDCOP III
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 54
SECP and DCOP
So far we have:
Designed a model for SECP
Formulated this model as a DCOP
Distributed the computation of the DCOP on devices / agents (bootstrap)
Run our system to get self-con�gured devices
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 55
But what happensin dynamic environments ?
if objects appear and disappear?
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 56
SECP is a dynamic problem
Dynamics in the infrastructure
Devices can disappear
New devices can be added to thesystem
At runtime..
No powerful device available to solvethe ILP
The deployment must be repaired:self-adaptation
Only consider a portion of the factorgraph: the neighborhood
c1 x1 u1
a1
c2 x2 ϕ1 y1
a2
c3 x3 ϕ2
a3
c4 x4 y2 u2
a4
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 57
k-resilienceDynamics in the infrastruture
De�nition (k-resiliency)k-resiliency is the capacity for a system to repair itself and operate correctly even inthe case of the disappearance of up to k agents
Two parts:I Do not loose the de�nition of the computations: replicationI Migrate the orphaned computations to another agent: selection / activation
Apply to any graph of computations, not only DCOP
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 58
Replication of computations
Replica distribution
For each computation, place k replica on k other agentsreplica equiv de�nition of the computation
Must be distributed !
Optimal replication ? impact the set of available agents when repairingwhich criteria ? too hard (quadratic multiple knapsack problem)...
Distributed Replica Placement Method (DRPM)
Heuristic : place replica on agents close (network) the active computation, whilerespecting capacity
Distributed version of iterative lengthening (aka uniform cost search based onpath costs)
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 59
Replication of computationsiterative lengthening on route and hosting costs
a1
a2 a3
a4
route(a1, a2) = 1
route(a2, a3) = 3
route(a2, a4) = 1
route(a1, a4) = 1
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60
Replication of computationsiterative lengthening on route and hosting costs
a1
a2 a3
a4
a2 a3
a4
host(a2, xi) = 1 host(a3, xi) = 1
host(a4, xi) = 5
route(a1, a2) = 1
route(a2, a3) = 3
route(a2, a4) = 1
route(a1, a4) = 1
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60
Replication of computationsiterative lengthening on route and hosting costs
a1
a2 a3
a4
a2 a3
a4
host(a2, xi) = 1 host(a3, xi) = 1
host(a4, xi) = 5
route(a1, a2) = 1
route(a2, a3) = 3
route(a2, a4) = 1
route(a1, a4) = 1
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60
Replication of computationsiterative lengthening on route and hosting costs
a1
a2 a3
a4
a2 a3
a4
host(a2, xi) = 1 host(a3, xi) = 1
host(a4, xi) = 5
route(a1, a2) = 1
route(a2, a3) = 3
route(a2, a4) = 1
route(a1, a4) = 1
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60
Replication of computationsiterative lengthening on route and hosting costs
a1
a2 a3
a4
a2 a3
a4
host(a2, xi) = 1 host(a3, xi) = 1
host(a4, xi) = 5
route(a1, a2) = 1
route(a2, a3) = 3
route(a2, a4) = 1
route(a1, a4) = 1
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60
Replication of computationsiterative lengthening on route and hosting costs
a1
a2 a3
a4
a2 a3
a4
host(a2, xi) = 1 host(a3, xi) = 1
host(a4, xi) = 5
route(a1, a2) = 1
route(a2, a3) = 3
route(a2, a4) = 1
route(a1, a4) = 1
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60
Replication of computationsiterative lengthening on route and hosting costs
a1
a2 a3
a4
a2 a3
a4
host(a2, xi) = 1 host(a3, xi) = 1
host(a4, xi) = 5
route(a1, a2) = 1
route(a2, a3) = 3
route(a2, a4) = 1
route(a1, a4) = 1
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60
Replication of computationsiterative lengthening on route and hosting costs
a1
a2 a3
a4
a2 a3
a4
host(a2, xi) = 1 host(a3, xi) = 1
host(a4, xi) = 5
route(a1, a2) = 1
route(a2, a3) = 3
route(a2, a4) = 1
route(a1, a4) = 1
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 60
Migrating computationsSelecting an agent
Migrating a set of xi computations Xc
set of candidate agents Acmigrating the computation must not exceed agent’s capacity
for each computation, select the agent that minimize hosting and communicationcost
Same optimization problem than for initial distribution, but on a subset of the of thegraph
Distributed process !
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 61
Migrating computationsSelecting an agent
Distributed optimization problem ⇒ let’s use a DCOP!A is the set of candidate agents AcX are the binary decision variables xmiC are the constraints ensuring that all computations are hosted, agent’s capacitiesare respected and hosting and communication costs are minimized
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 62
Migrating computationsSelecting an agent
∑am∈Ai
c
xmi = 1 (13)
∑xi∈Xm
c
weight(xi) · xmi +∑
xj∈µ−1(am)\Xc
weight(xj) ≤ cap(am) (14)
∑xi∈Xm
c
host(am, xi) · xmi (15)
∑(xi,xj)∈Xm
c ×Ni\Xc
xmi · com(i, j,m, µ−1(xj))
+∑
(xi,xj)∈Xmc ×Ni∩Xc
xmi ·∑
an∈Ajc
xnj · com(i, j,m, n)) (16)
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 63
Decentralized reparation
When agents are removed:
computation to migrate = computation that were hosted on these agents
candidate agents = remaining agents that posses a replica of these orphanedcomputation
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 64
Solving the migration DCOPWhich algorithm should we use ?
Criteria:
lightweight
fast (even if not optimal !)
monotonic : mix of hard and soft constraints
MGM-2 : like MGM, with 2-coordination
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 65
How does it behave, experimentally?
400
500
600
700
800
900
1000
colo
ring
scal
efre
e
uniform
400
500
600
700
800
900
1000problem dependent
5000
5500
6000
6500
7000
colo
ring
rand
om
5000
5500
6000
6500
7000
0 25 50 75 100 125 150 175time (s)
125
100
75
50
25
0
25
ising
0 25 50 75 100 125 150 175time (s)
120100806040200
20
single instance cost with pertubation average cost with pertubation average cost without pertubation
DSA
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 66
How does it behave, experimentally? (cont.)
80
100
120
140
160
180
colo
ring
scal
efre
e
uniform
80
100
120
140
160
problem dependent
250
300
350
400
450
colo
ring
rand
om
250
300
350
400
0 25 50 75 100 125 150 175time (s)
30
25
20
15
10
5
0
ising
0 25 50 75 100 125 150 175time (s)
30
20
10
0
single instance cost with pertubation average cost with pertubation average cost without pertubation
MaxSum
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 67
Menu
Hands on PyDCOP I
Hands on PyDCOP II
Focus on Smart Environment Con�guration Problems
Distributing Computations
Hands on PyDCOP III
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 68
To sum up
What we’ve seen today:
Some generic conceptsI How to model coordination problems using DCOP formalismI Some solution methods (complete and incomplete) to solve DCOP
Some speci�cities of IoT-based appsI How to model a speci�c smart environment con�guration problem as a DCOPI How to use PyDCOP to model, run, solve, and distribute DCOPI How to equip a system with resilience using replication and DCOP-based reparation
Want to go deeper into DCOPs→ OPTMAS-DCR workshop series (AAMAS/IJCAI),other tutorials at AAMAS/IJCAI
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 69
The End of Hands on PyDCOP
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 70
References
Boulle, M. (2004). “Compact Mathematical Formulation for Graph Partitioning”. In: Optimization and Engineering 5.3,pp. 315–333. issn: 1573-2924. doi: 10.1023/B:OPTE.0000038889.84284.c7. url:http://dx.doi.org/10.1023/B:OPTE.0000038889.84284.c7.
Rust, P., G. Picard, and F. Ramparany (2016). “Using Message-passing DCOP Algorithms to Solve Energy-e�cient SmartEnvironment Con�guration Problems”. In: International Joint Conference on Arti�cial Intelligence (IJCAI). AAAI Press.
Pierre Rust, Gauthier Picard Multi-Agent Distributed Constrained Optimization 71