distributed cooperation and coordination using the max-sum algorithm
DESCRIPTION
Distributed cooperation and coordination using the Max-Sum algorithm. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/1.jpg)
Distributed cooperation and coordination using the Max-Sum
algorithm
Farinelli, A., Rogers, A., Petcu, A. and Jennings, N. R. (2008) Decentralised Coordination of Low-Power Embedded Devices Using the Max-Sum Algorithm. In: Seventh International Conference on Autonomous Agents and Multi-Agent
Systems (AAMAS-08), 12-16 May 2008, Estoril, Portugal.
Waldock, A., Nicholson, D. and Rogers, A. (2008) Cooperative Control using the Max-Sum Algorithm. In: Second International Workshop on Agent
Technology for Sensor Networks, Estoril, Portugal.
![Page 2: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/2.jpg)
What is Max-Sum?
• Max-sum is a derivative of the sum-product algorithm, formed by transforming the max-product algorithm into log-space.
![Page 3: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/3.jpg)
What is Sum-Product?
• General form of the forward-backward algorithm
• Iterative algorithm to compute the marginals of functions
• a.k.a Belief Propagation (when applied to probabilistic graphical models)– Information Theory– Machine Learning– Vision
![Page 4: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/4.jpg)
An abstract problem
Consider a function, F, that is dependent on N variables, x = {x1 . . . xN}, and is defined as a product of M factors such that:
where each of the factors fm(xm) is a function of a subset xm of the variables that make up x.
![Page 5: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/5.jpg)
An abstract problem
Find the marginal function zn(xn) that describes the dependency of F(x) on variable xn:
The marginal zn(a) is defined to be the sum of F(x) over all configurations of variables where
xn = a.
![Page 6: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/6.jpg)
Factor graph representation
• Bipartite graph consisting of two types of nodes– Variable nodes– Function nodes
![Page 7: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/7.jpg)
Factor graph representation
• Example: F = f1(x1,x2)f2(x2,x3)
![Page 8: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/8.jpg)
What is Sum-Product?
• Given a factor graph
• Apply a local message passing procedure
• Compute the marginals of functions
![Page 9: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/9.jpg)
Two local message types
• From variable to function:
• From function to variable:
![Page 10: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/10.jpg)
Canonical update procedure
• Graph is cycle-free (a tree)
• Leaf nodes send identity messages to parents
• When a node receives a message on some edge, it computes and transmits the outgoing message for all other edges
• After variable node has received a message from every neighbor, it can compute its exact marginal value:
![Page 11: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/11.jpg)
Message Passing
![Page 12: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/12.jpg)
Message Passing
![Page 13: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/13.jpg)
Message Passing
![Page 14: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/14.jpg)
“Loopy” Belief Propagation
• Sum-product algorithm is only guaranteed to converge on graphs containing up to one loop
• Could oscillate, or fail to converge on cyclic graphs
• Empirically, it often converges pretty well on many types of graphs
![Page 15: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/15.jpg)
“Loopy” Belief Propagation
• All variable messages initialized to 1
• All messages updated at every iteration
• Normalize variable to function message:
Choose αnm such that
![Page 16: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/16.jpg)
What is Max-Product?
• Instead of computing the marginal functions, we can calculate arg maxx F(x)
• Replace function-to-variable message:
![Page 17: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/17.jpg)
What is Max-Product?
• Now, when we take the product of the messages:
• For arg maxx F(x), each component is given by:
• A.k.a: Viterbi algorithm
![Page 18: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/18.jpg)
What is Max-Sum?
• Suppose we have an F(x) that is the sum of function nodes, e.g.
• Use max-product in logarithm space
![Page 19: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/19.jpg)
Two new local message types
• From variable to function:
• From function to variable:
![Page 20: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/20.jpg)
What is Max-Sum?
• Now, when we take the sum of the messages:
![Page 21: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/21.jpg)
Advantages
• Existing theoretical results• Simple, scalable local algorithm
– Exponential only in # of neighbors in graph
• Already decentralized and asynchronous– Partition nodes as desired– Only exchange local messages
• Can estimate variables before convergence
![Page 22: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/22.jpg)
Caveats
• No strong theoretical results for loopy BP– Convergence not guaranteed
• Must exchange local messages between related factor and variable nodes
• Not necessarily easy to evaluate functions and compute messages
![Page 23: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/23.jpg)
How to use Max-Sum to solve your distributed problem:
Construct utility function
![Page 24: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/24.jpg)
How to use Max-Sum to solve your distributed problem:
Construct utility function
Turn it into factor
graph
![Page 25: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/25.jpg)
Application #1:Decentralized Coordination
Farinelli, A., Rogers, A., Petcu, A. and Jennings, N. R. (2008) Decentralised Coordination of Low-Power Embedded Devices Using the Max-Sum
Algorithm. In: Seventh International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-08), 12-16 May 2008,
Estoril, Portugal.
![Page 26: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/26.jpg)
Problem
• Solve graph coloring on sensor network
• Utility can be formulated for each agent:
Penalty for same color:
A priori color preference:
![Page 27: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/27.jpg)
Factor-graph Formulation
• Utility function is only dependent on neighbors• Let each sensor maintain its state and utility
nodes:
• Exchange local messages as per Max-Sum
![Page 28: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/28.jpg)
Comparison Algorithms• BR - Best Response
– Each agent chooses the “best” state for its variable (i.e. the one that minimizes the conflicts) according to the current states of its neighbors
– When a state change occurs, the agent sends the updated state to all of its neighbors
– Represents a lower bound on the performance of any algorithm since it uses the minimum computation and communication possible
• DSA - Distributed Stochastic Algorithm– Same as best response, except that an agent only actually performs the state
change according to a predefined probability (called the activation probability)– Whenever an agent’s preferred state actually changes, it sends a message to its
neighbors informing them of this fact– Set the activation probability to 0.6 (additional experiments indicate little
sensitivity to the exact value of this parameter in our setting)• DPOP - Dynamic Programming Optimization Protocol
– A complete algorithm that maintains optimality by preprocessing the constraint graph to produce a pseudo-tree, and then performs local message passing on this tree
![Page 29: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/29.jpg)
Test Conditions
• Known colorable– 3-color random graphs – 10, 20, 30, 40, 50 nodes– Link density of 3– 50 instances
• ADOPT graph repository– Random graphs – 10, 14, 18, 25 nodes– Link density of 3– Not generally colorable with 3-colors
• Lattice graphs– 4-color colorable– 36, 49, 64, 81, 100 nodes– Nodes connected to eight neighbors
![Page 30: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/30.jpg)
Conflicts over time
![Page 31: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/31.jpg)
Conflicts over time
![Page 32: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/32.jpg)
Conflicts over time
![Page 33: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/33.jpg)
What happened here?
• Combination of small and large irregular loops
• Cycling causes poor convergence
• Modify utility function:
![Page 34: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/34.jpg)
Summary of conflict results
![Page 35: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/35.jpg)
Message Size
![Page 36: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/36.jpg)
Message Loss
![Page 37: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/37.jpg)
Hardware Implementation
• http://www.youtube.com/v/T6H1AwQ2gXw
• http://www.youtube.com/v/D6vWvs3Lsj0
![Page 38: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/38.jpg)
Application #2:Cooperative Control
Waldock, A., Nicholson, D. and Rogers, A. (2008) Cooperative Control using the Max-Sum Algorithm. In: Second International Workshop on
Agent Technology for Sensor Networks, Estoril, Portugal.
![Page 39: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/39.jpg)
Problem
• Sensor network doing distributed tracking• Sensors share information using
decentralized data fusion (DDF)
![Page 40: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/40.jpg)
Problem
• Select control parameters for each sensor– Maximize sum of global info for all targets
![Page 41: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/41.jpg)
Problem
• Define U as a summation of target factors Uj over only sensors that can observe target, e.g.
where Ub is defined as:
![Page 42: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/42.jpg)
Problem
• Unfortunately, we can’t break into smaller factors:
≠
![Page 43: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/43.jpg)
Factor-graph Formulation
• Each sensor maintains variable node for its own control parameter
• Define a function node for each target– Maintained by the sensor that first saw target
• Use DDF to determine edges of graph– Add/remove edges as target moves in and out of
sensor range
• Exchange local messages as per Max-Sum
![Page 44: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/44.jpg)
Factor-graph Formulation
Sensor 3
θ2
Sensor 4
θ4
Sensor 2
θ2 UB
Sensor 1
θ1 UA
Sensor 5
θ5 UC
![Page 45: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/45.jpg)
Test conditions
• Sensors initialized with weak priors and target responsibilities
• Performance evaluated using global information from one sensor
![Page 46: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/46.jpg)
Global Information
![Page 47: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/47.jpg)
Adjusting negotiation time
![Page 48: Distributed cooperation and coordination using the Max-Sum algorithm](https://reader036.vdocuments.site/reader036/viewer/2022062315/56814e80550346895dbc1e63/html5/thumbnails/48.jpg)
How to use Max-Sum to solve your distributed problem:
Construct utility function
Turn into factor graph