lecture vi: collective behavior of multi- agent systems ii: intervention zhixin liu complex systems...

Lecture VI:Collective Behavior of Multi-

Agent Systems II: Intervention

Zhixin Liu

Complex Systems Research Center, Complex Systems Research Center, Academy of Mathematics and Systems Academy of Mathematics and Systems

Sciences, CASSciences, CAS

In the last lecture, we talked about

Collective Behavior of Multi-Agent Systems I: Analysis

Introduction Model: Vicsek model


Multi-Agent System (MAS)

MAS Many agents Local interactions between agents Collective behavior in the population level

More is different.---Philp Anderson, 1972 e.g., small-world, swarm intelligence, panic, phase transition, coordination, s

ynchronization, consensus, clustering, aggregation, ……

Examples:Physical systemsBiological systemsSocial and economic systems Engineering systems… …

Autonomy: capable of autonomous action Interactions: capable of interacting with other agents

Vicsek Model (T. Vicsek et al. , PRL, 1995)

http://angel.elte.hu/~vicsek/http://angel.elte.hu/~vicsek/

r

A bird’s neighborhoodAlignment: steer towards the average heading of neighbors

)(ti

v: the constant speed of birdsr: radius of neighborhood

: heading of agent ixi(t) : position of agent i in the plane

Vicsek Model

r

A bird’s neighborhood

Alignment: steer towards the average heading of neighbors

http://angel.elte.hu/~vicsek/

})()(:{)( rtxtxjtN jii Neighbors:

))1(sin),1((cos)()1( ttvtxtx iiii Position:

Synchronization: There exists θ, such that

Heading:

)()(cos

)()(sin

arctan)1(

ti

Njt

j

ti

Njt

j

ti

Introduction Model Theoretical analysis Concluding remarks


.))1(sin),1((cos)()1(

,)()(

1)1(

)(

ttvtxtx

ttN

t

iiii

tNjj

ii

i

.))1(sin),1((cos)()1(

),()()1(

ttvtxtx

ttPt

otherwise

jiiftNtp

tptP

iij

ij

0

~|)(|

1)(

)},({)(

The Linearized Vicsek Model

A. Jadbabaie , J. Lin, and S. Morse, IEEE Trans. Auto. Control, 2003.

Related result: J.N.Tsitsiklis, et al., IEEE TAC, 1984

Joint connectivity of the neighbor graphs on each time interval [th, (t+1)h] with h >0

Synchronization of the linearized Vicsek model

Theorem 2 (Jadbabaie et al. , 2003)

Random Framework

Random initial states:

1) The initial positions of all agents are uniformly and independently distributed in the unit square;

2) The initial headings of all agents are uniformly and independently distributed in [-+ε, -ε] with ε∈ (0, ). The initial headings and positions are independent.

For any given system parameters

and when the number of agnets n

is large, the Vicsek model will synchronize almost surely.

0v,0r

Theorem 7

High Density Implies Synchronization

This theorem is consistent with the simulation result.

Let and the velocity

satisfy

Then for large population, the MAS will synchronize almost surely.

),(log

),1(61

nn ron

nor

.

log 2/3

6

n

nrOv n

n

Theorem 8High density with short distance interaction

Three Categories of Research on Collective Behavior

Three Categories of Research on Collective Behavior

J.Han, M.Li, L.guo, JSSC,2006

AnalysisGiven the local rules of the agents, what is the collective behavior of the overall system ? (Bottom Up)

DesignGiven the desired collective behavior, what are the local rules for agents ? (Top Down)

InterventionGiven the local rule of the agents, how we intervene the collective behavior?

Example 1: Synchronization

r

A bird’s Neighborhood


Simulation Result

Q: Under what conditions such a system can reach consensus?

Example 2: Escape Panic

Normal, no panic Fire, panic

D. Helbing, et al., Nature, Vol. 407, 2000

Three Categories of Research on Collective Behaviors





How we design the control law of each plane to maintain the form ?

Example 1: Formation control

Example 2: Swarm Intelligence (Marco Dorigo et al., 2001-2004)

www.answers.com/topic/s-bot-mobile-robot

Example 3:Distributed Control in Boid Model

Each agent is described by a double integrator (Newton's second law of motion ):

i i

i i

x v

v u

R. Olfati-Saber, IEEE Trans. Auto. Control ,2006.

where xi, vi and ui represent the position, velocity and the control input of the agent i.

What information can be used to design the controller?The position and velocity of neighbors

Goal: 1) Avoid collision 2) Alignment 3) Cohension

Algorithm

where A=[aij(q)] is the adjacency matrix,

Theorem 1:

If the neighbor graphs are connected at each time instant. Then

1) The group will form cohesion.

2) All agents asymptotically move with the same velocity.

3) No interagent collisions occur.

Neighbor graph

Controller design:

(·) is the action function, )(

isσ-norm, and

Three Categories of Research on Collective Behaviors





Example 1: Can we guide the birds’ flight if we know how they fly ?

Intervention

Example 2: Leadership by Numbers

Couzin, et al., Nature, Vol. 433, 2005

The larger the group is, the smaller the leaders are needed.

Example 3: Cockroach

J.Halloy, et al., Science, November 2007

III. Intervention Given the local rule of the agents, how we intervene the collective behavior?

The current control theory can not be applied directly, because It is a many-body self-organized system. The purpose of control aims to collective behavior. Not allowed to change the local rules of the existing

agents; Distributed Control: special task of formation, Pinning Control: Networked system, imposed

controllers on selected nodes

Intervention Via

Soft Control

Soft Control

The multi-agent system: Many agents Each agent follows the local rules

Autonomous, distributed Agents are connected, the local effect will affect the whole.

From Jing Han’s PPT

Soft Control

y(t)u(t)

The “Control”: No global parameter to adjust Not to change the local rule of the existing agents;

Put a few “shill” agents to guide (seduce) Shill: is controlled by us, not following the local rules, is treated as an ordinary agent by other ordinary agents The power of shill seems limited

The ‘control’ is soft and seems weak


an associate of a person selling goods or services or a political group, who pretends no association to the seller/group and assumes the air of an enthusiastic customer.

Soft Control

y(t)U(t)

Key points:Key points: Different from distributed control approach.

Intervention to the distributed system Not to change the local rule of the existing agents Add one (or a few) special agent – called “shill” based on the syst

em state information, to intervene the collective behavior; The “ shill” is controlled by us, but is treated as an ordinary agent

by all other agents. Shill is not leader, not leader-follower type. Feedback intervention by shill(s).

From Jing Han’s PPTThis page is very important!

There Are Lots of Questions …

What is the purpose/task of control here? Synchronization/consensus Group connected / Dissolve a group Turning (Minimal Circling) Lead to a destination (in a shortest time) Avoid hitting an object Tracking …

In what degree we can control the shill? (heading, position, speed, …)

How much information the shill can observe ? (positions, headings, …)

…


A Case Study

Problem statement: System: A group of n agents with initial headings i(0)[0,

); Goal: all agents move to the direction of eventually. Soft control: Design one shill agent based on the agents’ state

information.

Assumptions: The local rule about the ordinary agents is known The position x0(t) and heading 0(t) of the spy can be controlled

at any time step t The state information (headings and positions) of all ordinary

agents are observable at any time step


Vicsek Model

r

A bird’s Neighborhood


http://angel.elte.hu/~vicsek/


))(sin),((cos)()1( ttvtxtx iiii Position:

Heading:

)()(cos

)()(sin

arctan)1(

ti

Njt

j

ti

Njt

j

ti

Synchronization: There exists θ, such that

Problem statement: System: A group of n agents with initial headings i(0)[0, ); Goal: all agents move to the direction of eventually. Soft control: Design one shill agent based on the agents’ state information.

Assumptions: The local rule about the ordinary agents is known

The position x0(t) and heading 0(t) of the shill can be controlled at any time step t

The state information (headings and positions) of all ordinary agents are observable at any time step


A Case Study

The Control Law u

Control the Shill agent


Theorem 4: For any initial headings and positions

i(0)[0, ), xi(0)R2, 1 i n, the updat

e rule and the control law uβ will lead to th

e asymptotic synchronization of the group.

Control the Shill agent

It is possible to control the collective behavior of a group of agents by a shill.


Simulation

(t),u

(t),uu

r

βt

,2,1, kkht

where

)),(())(),((: )(00 txttxu trr

)}({maxarg)( 1

)(:txtr i

ti i

An Alternative Control Law

Result: The control law ut will also lead to asymptotic synchronization of the group.

otherwise

Simulations

Control Law u

Switching between u and ur

Remarks on Soft Control It is not just for the above model Can be applied to other MAS ,e.g.,

Panic in Crowd Evolution of Language Multi-player Game ……

“Add the special agent(s)” is just one way Should be other ways for different systems: Remove agents Put obstacle … …

We need a theory for Soft Control !We need a theory for Soft Control !


Intervention ViaLeader-Follower Model (LFM)

Example 1: Leadership by Numbers Couzin, et al., Nature, Vol. 433, 2005

The larger the group is, the smaller the leaders are needed.

Leader-Follower Model

Problem statement:

System: A group of n agents;

Goal: All agents move with the expected direction eventually.

Intervention by leaders: Add some information agents-called “leaders”, which move

with the expected direction.

Leader-Follower ModelOrdinary agents

Information agents

Key points: Not to change the local rule of the existing agents. Add some (usually not very few) “information” agents –

called “leaders”, to control or intervene the MAS; But the existing agents treated them as ordinary agents.

The proportion of the leaders is controlled by us (If the number of leaders is small, then connectivity may not be guaranteed).

Open-loop intervention by leaders.

Mathematical Model

})()(:{)( ''' rtxtxjtN

jii


Heading:

)(

)(

)(cos

)(sinarctan)1(

tNj

tNj

i

i

i

tj

tj

t

Position: ))1(sin),1((cos)()1( ttvtxtx iiii

Ordinary agents (labeled by 1,2,…,n):

0)1(' tiHeading:

Position: ))1(sin),1((cos)()1( '''' ttvtxtxiiii

Leader agents (labeled by ):)(,2,1 '''nn nM

)(#)(),(#)( '' tNtntNtn iiii

)()(

)()(

'

'

)(cos

)(sinarctan)1(

tNtNj

tNtNj

i

ii

ii

tj

tj

t

Simulation Example

N=1000

Q: How many leaders are required for consensus/synchronization?

Assumption on the initial states

1) The initial positions of all agents are independently and uniformly distributed in the unit square.

2) The initial headings of the agents are uniformly and independently distributed in [-π, π), and the initial headings of the leaders are . The headings and the positions are mutually independent.

0

Random Framework

))(,),(()( 1 tdtddiagtD n

)(min),(max)(,,,1),( minmax tddtdtdnitd ii

ii

i Degree:

Degree matrix:

Adjacency matrix:

0

,1)(taij

If i ~ j

Otherwise)},({)( tatA ij

Some Notations

otherwise

jiifttatatA

j

ijij

0

~)(cos)(~)],(~[)(

)()(

)(cos),(cos)(1 tNj

jtNj

j

n

ttdiagtG

Weighted adjacency matrix:

Weighted degree matrix:

));(,),(()( ''1 tntndiagtN nLeader degree matrix:

)())()(()( 1 tAtNtDtP

)(~

))()(()(~ 1 tAtNtGtP

Average matrix:

Weighted average matrix:

“Normalized Laplacian” :2/12/1 ))0()0())(0()0(())0()0(()0( NDNLNDM

)0()0(0 1 n Spectrum :

Vj jjj

Vj jjji ji

fn nnf

fnff

))0()0((

)0()(sup)0(

'2

2'

~

2

0

)0(1,)0(1max)0( 1 n “Spectral gap”:

where

Laplacian : L(0)=D(0) – A(0)

Some Notations (cont.)

Vj jjj

Vj jjji ji

f nnf

fnff

))0()0((

)0()(inf)0(

'2

2'

~

2

01

Key Steps in the Analysis of the LFM

Analysis of the system dynamics

Estimation of the rate of consensus

Dealing with the matrices with increasing dimension

Dealing with the inherent nonlinearity

Analysis of the System Dynamics

Evolution of the distance

Lemma 1: For any two agents i and j, their distance satisfy the following inequality:

where

is important for the evolution of the distance!


Step 1: Projection

)(tan)(cos)(

)(cos)1(tan

)()(

't

ttn

tt j

tNjtNj

ji

ji

i

i

0)()( tt ii

Evolution of the headings

)(tan)(~

)(tan)(~

))()(()1(tan 1 ttPttAtNtDt


Step 2: Analyze the stability of

Step 3: Dealing with the changing neighbor graphs

)0()(sup)()(~

sup11

PkPkPkPtktk

)0()(~

sup1

PkPtk

)0()0(min

)0()0(max'

1

'

1

iini

iini

nn

nnk

where

'

n n

' 'n n

{ : (1 ) (1 ) }

{ : (1 ) (1 ) }

i j i

i ij

R j r x x r

R j r x x r

n(1 )r

r

(1 )n r

Estimation of Consensus Rate

1) A key lemma: For any vector f=[f1,f2,…,fn]τ, we have

)0(The consensus rate depends on

Vj jjj

Vj jjji ji

fn nnf

fnff

))0()0((

)0()(sup)0(

'2

2'

~

2

0

Vj jjj

Vj jjji ji

f nnf

fnff

))0()0((

)0()(inf)0(

'2

2'

~

2

01

..)),1(1(1)0( saoM

n

n

2)

Dealing with the Matrices with Increasing Dimension

Estimation of multi-array martingales

..,log34

3),(maxmax

11

11sanS

Cnkwf n

wm

jjj

nknm

.),(),(sup, 21

,11

2 nkFnkwECfS jjnjk

w

n

jjn

where

..log3),(maxmax1

111

sanSCnkwf nw

m

jjj

nknm

,log4 1 nCS wnMoreover, if then we have

Dealing with the Matrices with Increasing Dimension

.1.,.,log1

1)(cosmax)4

..,log1

)1(tanmax)3

..,log)0(cosmax)2

..,log)0(sinmax)1

1

1

)0(1

)0(1

tsan

nOt

san

nO

sannO

sannO

nj

ni

nj

ni

Njj

ni

Njj

ni

i

i

Using the above corollary, we have for large n

where .1log6

nn

n

The Degree of The Initial Graph

])log[()0(#max)4

..],)log[()0(#max)3

..],)log[()0(max)2

..],)log[()0(max)1

2/1

1))0((

1

2/1

1))0((

'

1

2/1

1))0((

1

2/1

1))0((

'

1

''

''

nnOEIR

sannOEIR

sannOEIn

sannOEIn

n

jRji

ni

n

M

jRji

ni

n

jNji

ni

n

M

jNji

ni

i

n

i

i

n

i

Lemma: For initial graph G0, we have for large n

saornR

saornR

sanCnnn

sanCnnn

saon

n

nnini

nini

nini

nini

ini

ini

ni

i

.)),1(1(4)0(#max)5

.)),1(1(4)0(#max)4

.,)0(min,)0(max)3

..,)0(min,)0(max)2

..)),1(1()0(

)0()1

2'

1

2

1

2'

1

'

1

111

'

Corollary:

The Degree of The Initial Graph

Dealing with the Inherent Nonlinearity

Proposition 1For any positive v and r, we have for large n

..,1)),1(1()0()( satordtd nijij

..,1)),1(1(3

11))(()0( 21 satotk n

where

.3

1,)()(

~sup)(

,24

161,4min

21

1

2

22

nts

nn

sPsPt

r

r

Theorem 5 Let the velocity v > 0 and radius r > 0 be positive

constants. If the proportion of the leaders satisfies

where C is a constant depending on v and r, then the headings of all agents will converge to almost surely when the population size n is large enough.

6

log

n

nC

n

M n

0

Main Result

Concluding Remarks

In this talk, we talked about intervention to the multi-agent systems:

Soft control

Design the control law of the “shill”

Leader-follower model

Control the number of the leaders

Concluding RemarksThese two lectures mainly focus on the

collective behavior of the MAS.

In the next lecture, we will talk about game theory.

Thank you!

lecture vi: collective behavior of multi- agent systems ii: intervention zhixin liu complex systems...

Documents

collective behavior

analysis slide

plane slide

cas slide

agents collective behavior

vicsek model r

vicsek r

vicsek neighbors