cooperative control of multi-agent systemswatanabe-...1 cooperative control of multi-agent systems...
TRANSCRIPT
![Page 1: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/1.jpg)
1
Cooperative Control of
Multi-Agent Systems
Hideaki Ishii
Dept. Computational Intelligence & Systems Science
Advanced Topics in Mathematical Information Sciences I
Jul 10th, 2015
![Page 2: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/2.jpg)
2
Control of multi-agent systems Active research in the area of systems control (2000~)
Keywords: Distributed control/algorithms, Communication
networks, Remote control over networks,…
Introduction
2
![Page 3: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/3.jpg)
3
Consensus problem One of the basic problems for multi-agent systems
Initiated the research trend in this area
Systems control approach: Theory-based with applications
In this lecture Basics of multi-agent consensus
Introduction
3
![Page 4: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/4.jpg)
44Sensor networks
Flocks of fish/birds
Formation of autonomous robots
Load balancing among servers
What is consensus?
![Page 5: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/5.jpg)
5
Cluster of small robots for planetary exploration High flexibility and reliability at low cost
Communication is limited by on-board power
Array antenna Multiple antennas coupled for directed transmission
Formation of robots based on distributed control laws
Example 1: Autonomous robots
5
Formation
![Page 6: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/6.jpg)
6
Spatially distributed autonomous sensors with wireless communication capability
Problem: When each sensor measures unknown parameter + noise, want to find the average of all measurements.
Example 2: Sensor networks
6
Centralized scheme Distributed scheme
Fusion center
![Page 7: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/7.jpg)
7
Consensus problem
Network of agents without a leader
Each agent communicates with others and updates its state
All agents should arrive at the same (unspecified) state
7Achieve global objectives through local interaction!
![Page 8: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/8.jpg)
8
Flocking of birds: Formation flying without a leader
What are the simple control laws for each bird?
Simulation-based study by Raynolds
Some history (1): Boids
8Raynolds (1987)
Three rules
Separation
Alignment
Cohesion
![Page 9: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/9.jpg)
9
Proposed a mathematical model of agents’ dynamics Each agent moves on a plane at constant speed
Align with the directions of neighboring agents
Flocking behavior was observed by simulation
Some history (2): Model by Vicek et al.
9Jadbabaie, Lin, & Morse (2003), Tsitsiklis & Bertsekas (1989)
Vicek et al. (1995)
Analytic results by Jadbabaie et al.
Proved that all agents converge to the same direction if there is sufficient connectivity structure Motivated control researchers to study multi-robot problems
![Page 10: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/10.jpg)
10
Network of agents
10
![Page 11: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/11.jpg)
11
Network of agents
11
1
2 6
3
4
10
5
9
8
7Info can be sent
from 4 to 2
![Page 12: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/12.jpg)
12
Connectivity in multi-agent systems
12
1
2 6
3
4
10
5
9
8
7
Represented as a graph Node set ⇒ Indices for the agents Edge set ⇒ Communication among
the agents
Info can be sent from 4 to 2⇔
![Page 13: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/13.jpg)
Neighbor set ⇒ Indices of agents that can send info to agent i
Example: For agent 2
13
Connectivity in multi-agent systems
1
2 6
3
4
10
5
9
8
7neighbor
![Page 14: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/14.jpg)
14
Basics of graphs
Types: Directed/Undirected Nodes i and j are connected
⇔ Agent j is reachable from i by following edges Graph is (strongly) connected
⇔ Any two nodes are connected 14
1
2 6
3
4
10
5
9
8
7
Agents 2 and 9The whole graph
⇒ Connected!
![Page 15: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/15.jpg)
15
At time k, agent i does the following:
1. Sends its value to the neighbor agents
2. Updates its value based on the received info and
obtains
15
Protocol for distributed algorithms
1
2 6
3
4
10
5
9
8
7
![Page 16: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/16.jpg)
16
Average consensus
Problem: Find a distributed algorithm satisfying the two
conditions:
1. All agents converge to the same value.
2. The value is the average of the initial values.
16
![Page 17: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/17.jpg)
17
Algorithms in this lecture
Two classes of consensus problems1. Real-valued2. Integer-valued (Quantized)
Algorithms may be deterministic or probabilistic
Graph structure: Undirected and connected
17
![Page 18: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/18.jpg)
18
Average consensus (1)
Real-valued case
18
![Page 19: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/19.jpg)
19
Each agent has a real value
Average consensus
Example
19
Real-valued average consensus
Average of initial values
2 31
Initial values 1 2 2
Ave=1.666
![Page 20: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/20.jpg)
20
Update scheme for agent i:
where
Can be implemented in a distributed manner
20
Distributed algorithm
Number of neighbors for agent i
if
if
Otherwise
Xiao, Boyd, Lall (2005)
![Page 21: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/21.jpg)
2121
Example
2 31
Init. values 1 2 2
Ave=1.666
Update scheme for agent 1:
Update scheme for agent 2:
2 31
![Page 22: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/22.jpg)
2222
Example
2 31
Init. values 1 2 2
Ave=1.666
Distributed algorithm:
![Page 23: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/23.jpg)
2323
Example
2 31
Init. values 1 2 2
Ave=1.666
Consensus!
![Page 24: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/24.jpg)
2424
Example
2 31
Init. values 1 2 2
Ave=1.666
Distributed algorithm in vector form:
Each element is nonnegative, and
Sum of elements in each row = 1 ⇒ Row stochastic
Sum of elements in each column = 1 ⇒ Column stochastic
![Page 25: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/25.jpg)
25
Property 1Because W is row stochastic,
The matrix has eigenvalue 1
Corresponding eigenvector is a (scalar multiple of)
vector 1:
25
General form of the algorithm
Stochastic matrix (Row and column)
where
![Page 26: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/26.jpg)
26
Property 2Because W is column stochastic,
Thus
26
General form of the algorithm
Stochastic matrix (Row and column)
where
Sum of all elements is invariant!
![Page 27: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/27.jpg)
27
By properties 1 and 2,
For eigenvalue 1, the eigenvector is in the form
and satisfies
Hence
27
Average vector
The desired average!
However, there may be other vectors as the eigenvector.
If the graph is connected, then it is unique.
(by the Perron-Frobenius theorem)
![Page 28: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/28.jpg)
28
Computation via power method
The state converges to the eigenvector
Result: If the network of agents forms a connected graph,
then average consensus is achieved:
28
Convergence of the algorithm
![Page 29: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/29.jpg)
10 agents
Random graph:
Initial positions are uniformly distributed
Neigbors are agents within radius r
Autonomous mobile robots: Randezvous
29
Radius r
![Page 30: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/30.jpg)
Radius r=0.8 # of edges 35
30
![Page 31: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/31.jpg)
Radius r=0.6 # of edges 27
The graph is a subgraph of the previous one.
31
![Page 32: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/32.jpg)
Radius r=0.38 # of edges 11
Disconnected !
32
![Page 33: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/33.jpg)
33
Recap
Average consensus: Real-valued case True average
Connected graph
Matrix theory
33
![Page 34: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/34.jpg)
34
Average consensus (2)
Integer-valued (quantized) case
34
![Page 35: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/35.jpg)
35
Each agent’s value is an integer
What’s different:
True average of N integers ≠ integer
Approximation of the average is not unique
Convergence in finite time is possible (i.e., not asymptotic)
35Kashap, Basar, Srikant (2007)
Quantized average consensus
2 31
Init. values 1 2 2 Ave=1.666
1 or 2 ?
![Page 36: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/36.jpg)
36
Gossip algorithm Agents decide to communicate at a random time with
randomly chosen neighbor. To each edge, assign a probability to be chosen.
No need of a common clock.
(asynchronous communication)
36
Boyd, Ghosh, Prabhakar, Shah (2006)
Probabilistic communication
1
2 6
3
4
5
7
![Page 37: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/37.jpg)
37
Problem:
Find a distributed algorithm such that
1. Each agent’s value is always an integer
2. Sum of all agents’ value is constant
3. For sufficiently large k, the agents achieve average
consensus, that is,
37Kashap, Basar, Srikant (2007)
Quantized average consensus
or
![Page 38: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/38.jpg)
38
At time k, one edge is randomly chosen.
38
Quantized gossip algorithm
1
2 6
3
4
5
7
Agents update their values to by
If , then the values stay the same.
If , then exchange the values(Swapping)
Otherwise, if , then let
1. Sum of both values remains the same
2. Their difference is reduced
![Page 39: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/39.jpg)
39
Result:
The algorithm achieves quantized average consensus
with probability 1 in finite time.
39
Quantized gossip algorithm
Two important properties:
Swapping
Probabilistic algorithm
![Page 40: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/40.jpg)
40
For each edge, the difference in values is at most 1.
The average is unknown from local info.
By swapping, consensus is possible.
Agents with values 1 and 3 become neighbors (with prob. 1).
40
Example 1 (Swapping)
1 2 2 3 Average = 2Init. values 2 1 12 22Consensus!
![Page 41: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/41.jpg)
4141
Example 2 (Probabilistic algorithm)
Example of a deterministic algorithm: Periodic comm.1
2 3
3
2 1
3
1 2
2
1 3
Only swapping occurs, thus no consensus.
Under probabilistic comm., convergence in a few steps.
1
2 3
Average = 2
Init. values
・・・
![Page 42: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/42.jpg)
42
Recap
Average consensus: Quantized-valued case Approximate average
Gossip algorithm – Probabilistic but always correct
Theory of Markov chain
Performance at the order of
42
![Page 43: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/43.jpg)
43
Summary
Multi-agent systems and consensus problems
Graph representation of network structures
Distributed algorithms: Deterministic vs Probabilistic
Update schemes for different agent values
(real, quantized, and binary)
43
New challenges Performance
Communication (time delay, data rate, graph,…)
Dynamics of the agents (high dim., nonlinear,…)
![Page 44: Cooperative Control of Multi-Agent Systemswatanabe-...1 Cooperative Control of Multi-Agent Systems Hideaki Ishii Dept. Computational Intelligence & Systems Science ishii@dis.titech.ac.jp](https://reader035.vdocuments.site/reader035/viewer/2022062602/5ecedfec306e27216e53356c/html5/thumbnails/44.jpg)
44
Consensus problem
Network of agents without a leader
Each agent communicates with others and updates its state
All agents should arrive at the same (unspecified) state
44Achieve global objectives through local interaction!