Interactions in Multi Agent Systems

Dr. Sara Manzoni

Complex Systems and Artificial Intelligence research centerDepartment of Computer Science, Systems and

CommunicationUniversity of Milano-Bicocca

4th Summer School AACIMP-2009Achievements and Applications of

Contemporary Informatics, Mathematics and Physics

Lecture 2 – 12.08.2009

Multi Agent System (MAS)

“A modeling and computational approach considering that simple or complex

activities can be the fruits of interaction between autonomous and independent

entities (i.e. agents) which operate within communities (i.e. organized structures) in accordance with modes of cooperation (= collaboration + coordination + conflict

resolution) in order to fulfill given goals”

How to describe a phenomenon (solve a problem) as the result of collective

behavior• Modeling the problem as a structured set of

entities (i.e. organization) able to– Act in an environment– Interact: communicate and cooperate in order to fulfill

(common) tasks– Perceive (locally) the environment and adapt their

behavior according to perceptions– Possess their own resources, skills, tendencies and

objectives (explicit or implicit)– Behave (e.g. plan actions) tending towards the

satisfaction of objectives, taking into account available resources, according to their skills, and depending on their perceptions

Design of a MAS What should be modeled?

• Agents• Organization• Interactions• Environment

Design of a MAS (1) Agents

• Agent architecture (Internal structure) and agent behavior (Agent model) – actions that can be undertaken– environment perception– adaptation mechanism– goal fulfillment mechanism

• Tools: operative modeling, formalization and specification languages, knowledge representation languages– E.g. production rules, Petri nets

Design of a MAS(2) Organization

Leaving aside the dynamic dimension, an organization can be defined and analyzed– Functionally (roles, tasks, capacities)– Structurally (divisions, interconnections,

relationships)

Fixed, predefined structure (e.g. Hierarchy)

Variable according to predefined mechanisms (e.g. auction protocols)

Variable, structure emerging from system behaviour

Design of a MAS (3) Interactions (1/3)

• An interaction occur when two or more agents are brought into a dynamic relationship through a set of reciprocal actions

• Interactions develop out of a series of actions whose consequences in turn have an influence on the future behavior of agents

• During interactions, agents are in contact with each other– Directly– Through another agent– Through the environment

Interactions assume ...

• Presence of agents capable of interacting and/or communicating

• Situations which can serve as meeting point of agents

• Dynamic elements allowing local and temporary relationships between agents

• “slack” in relationships between agents enabling them to detach themselves from it (agent autonomy)

Interactions and organizations

• Interactions are an element necessary for the setting up of social organization

• Groups are – the result of interactions– the preferred locations in which interactions

occur• Interaction is the crucial element in

organizations Source and Product of the permanence of the organization

Interaction situation

• A concept introduced to describe activities of agents in order to identify different types of interactions by linking interactions to the elements of which they are composed

• Defines abstract interaction categories independent of their concrete realizations, by distinguishing them according to– Main invariables that we find everywhere – Differences between situations

An assembly of behaviors resulting from the grouping of agents which have to act in order to attain their

objectives, with attention being paid to the more or less resources which are available to them and their

individual skills

Example – Building of a house

• Type of interaction Cooperation situation requiring coordination of actions

• Interaction situation in which the assembly of behaviors of the agents (i.e. workforce, architect, owner, project manager, ...) is characterized by their own objectives (the same house looked at from the viewpoints of different agents) and their skills (know-how of the architect and of different skilled workers) with attention being paid to the available resources (raw materials, financing, tooling, building site)

Collective roboticsBio-inspired opt algorithms

A classification of Interaction situations

• According to compatibility of goals– Agents cooperate when their goals are compatible

positive interaction situations– Agents compete when their goals are incompatible

negative interaction situations• According to agent ability to available resources

– Conflict arises when resources are insufficient negative interaction situations

• According to agent ability to fulfill tasks– Collaboration arises when agents have insufficient

ability to solve complex problems positive interaction situations

Compatibility of goals in reactive agents

• Negative interaction: the survival behavior of the one entail the death of the other

• Positive interaction: the behavior of the one is not negatively affected by that of the other– Cooperation: the behavior of the one is

reinforced by the behavior of the other

• Indifference: the behavior of the one is not affected at all (neither positively nor negatively) by the behavior of the other

Symbiosis and prey-predator

• Symbiosis between organisms A and B (e.g. A nourishes B and B defends A from predators): reactive cooperation– Heterogeneous organisms cooperate since each

organism is reinforced by the presence and behavior of other one

• Prey-predator model: antagonistic cooperation – Predators cooperate (e.g. group formation) to hunt the

prey– Antagonistic relationship between predators and their

preys – the survival of preys entails the failure of predators

Resources

• All the environmental and material elements that can be used by agents to carry out their actions

• Conflicts arise when two or more agents need the same resources at the same time and in the same place

Resources wanted by A

Resources wanted by B

Conflict zone for

accessing resources

When?

Where?

Solving conflict situations with coordination

• Synchronization (from distributed systems research)– of movements– of access to resources

• Coordination by planning (from AI): Multi-agent planning– Centralized planning for multiple

agents – one planner– Centralized coordination for

partial planning – one coordinator– Distributed planning

• Reactive coordination– Coordination by situated actions

(potential fields or marking the environment)

• Coordination by regulation: rules– to anticipate and eliminate a-

priori conflict situations– to manage conflict resolution

Coordination in forest ecosystem

Competition on available resources, needed for survival and reproduction

Each portion of the territory - can be inhabited by a tree- contains a given amount of resources needed by plants to sprout, grow, survive, and reproduce themselves

C = {R, P, M, T, S}, where:

R = {R1,…,Rm} – amount of resources M = {M1,…,Mm} – maximum amount of each resourceP = {P1,…, Pm} – amount of each resource produced by the cell at each update stepT – plant state (if any)S = {s1,...,sn} – number of seeds of each species present in the cell

Different plant species can inhabit the same area and compete for the same resources

Interaction through resources

• The presence of a plant limits the sunlight diffusion to neighbours and seeds’ growth

• Different species have different needs in terms of resources

• Resources are produced and consumed by plants

• Resource distribution on the territory

Agents skills and tasks

• Tasks – can be carried out by a single alone (no interaction

required)– can be carried out alone but the accomplishment

is facilitated by the support of other agents– need several agents to be accomplished

• In cases of interaction, the resulting system posses new properties that can be described as new emerging functionalities– the produced object is more than the simple sum

of the skills of each of the agents– interactions between agents enhance the result

Types of interaction (1)

Goals Resources Skills Type

Compatible Sufficient Sufficient Independence Compatible Insufficient Sufficient ObtrusionCompatible Insufficient Insufficient Coordinate

CollaborationIncompatible Sufficient Sufficient Individual Competition

Incompatible Sufficient Insufficient Collective Competition

Incompatible Insufficient Sufficient Individual Conflict on resources

Incompatible Insufficient Insufficient Collective Conflict on resources

J. Ferber, “Multi-Agent Systems: an introduction to distributed artificial intelligence”, 1999

Types of interaction (2)• Independence (G, R, S): simple juxtaposition of actions

carried out by agent independently without effective interaction

• Simple collaboration (G, R, s): simple addition of skills, without requiring coordination of actions (e.g. When knowledge is shared among agents)

• Obstruction (G, r, S): agents get in touch in accomplishing their tasks, but they do not need one another

• Coordinated collaboration (G, r, s): agents have to coordinate their actions to have synergic advantages of pooled skills (e.g. industrial activities, network control, design and manufacturing of product) – most complex coordination

Types of interaction (3)• Pure individual competition (g, R, S): resources are not

limited and the competition is not related to them (e.g. running racing)

• Pure collective competition (g, R, s): agents have to group into coalitions or associations to be able to achieve their goals. Two phase process: individuals ally into groups + groups are set one against another (e.g. sailing competition)

• Individual conflict over resources (g, r, S): the object of conflict is the insufficient resource (e.g. Territory, financial position, animals defending their territory, humans willing to obtain a better job)

• Collective conflicts over resources (g, r, s): all forms of collective conflicts in which the objective is to obtain possession of territory or a resource (e.g. Wars, monopoly of a good) – collective competition + individual conflict on resources

INTERACTION MODELS IN MULTI-AGENT SYSTEMS

• Agent internal architecture can be separated by the (interaction) model that defines the way agents communicate

• This approach allows the modelling, design and implementation of heterogeneous entities, sharing an environment in which they can interact

• Many different interaction models have been defined and implemented

• Often inspired by other disciplines (e.g., social science, linguistics, biology)

INTERACTION MODELS IN MAS: A TAXONOMY

Agentinteraction

Directinteraction

Indirectinteraction

With a-prioriacquaintance

Agent discoverythrough middle agents

Middle agents &acquaintance models

Guided/mediatedby artifacts

Spatially foundedinteraction

Direct interaction models

• Agents are able to directly exchange information

• Information exchange– Communication/conversation rules (“protocol”)

Agent Communication Language (ACL)– Message structure (shared ontology) Content

Language• Information exchange is indiscriminate

– Once an agent knows another one, it will be able to communicate with it

– No external, contextual factors are considered

Direct interaction model example: KQML• Knowledge Query and Manipulation Language (KQML) and

Knowledge Interchange Format (KIF) are results of the ARPA Knowledge Sharing Effort– KQML is an ACL, a high level interaction language– KIF is a content language, defining syntax of contents

• KQML defines performatives (basic messages to compose conversations among agents)

• KIF allows to represent information and knowledge about agents, beliefs, desires, intentions, perceptions plans and thus their environment

• Agents must share an ontology, in terms a common vocabulary and agreed upon meanings to describe a domain subject

KQML Message (speech act)

A KQML speech act is described by a list of attribute/value pairs e.g. :content, :language, :from, :in-reply-to.

(tell :sender bookShopAgent123 :receiver ksAgent :in-reply-to id7.34.96.45391 :ontology books :language Prolog :content “price(ISBN3429459,24.95)”)

performative

parametervalue

A KQML DialogueAgents A and B “talking” about the prices of books bk1 and bk2:

A to B: (ask-if (> (price bk1) (price bk2)))B to A: (reply true)B to A: (inform (= (price bk1) 25.50))B to A: (inform (= (price bk2) 19.99))

For convenience message format above is simplified and attribute/value pairs for :ontology etc. are omitted.

KQML performatives

Some requirements• Agents need to know their communication partners

– Common approach is to have specific facilitators that are known by every agent and allow them to get acquainted

– Problems: how many of those ‘middle agents’ (robustness) ? How to keep the aligned ?

• A semantic must be defined to obtain/enforce meaningful conversations– Agent considered as a logical reasoner with beliefs, desires

and intentions– Pre and post conditions defined in terms of a of logic

formalization– Actualization of postconditions triggers preconditions of

other performatives– What about autonomy ?

Other tools for communication semantics

• The specification of conversations can be done through several formal models– Finite State Machines based– Petri nets based

• The former approach has been widely used to model, analyze and demonstrate properties of network protocols

• These appraches also limit agents’ autonomy

Direct interaction models: pros

• Similarity to existing protocols for distributed systems– Point-to-point message passing– Easy implementation on top of existing middleware

platforms• Simple integration with deliberative agents approach

– Agents exchange facts conforming to some kind of formally defined ontology

• Formal semantics of ACLs can be easily specified– Communication semantics is related to agents’ beliefs,

decisions, intentions

Direct interaction models: cons• Information exchange occurs according to specific rules

– Network protocol like issues (conversation rules, message formats)

Semantical issues• communication semantics related to agent internals (beliefs,

decisions, intentions)• normative semantics limits agents’ autonomy

• Exchanged information must conform to an ontology that is somehow shared by the agents Ontology issue

• Agents need to be aware of the presence of a communication partner Discovery issue

• Direct interaction models do not provide abstractions to represent elements of agents context

Direct interaction models:some enhancements

• Discovery issue and agent context– Middle agents as specific agents collecting

and providing acquaintance information to entities of the system

– Not a single middle agent, but a network of them, organized in order to provide robustness and structure

– Not just mere agent name service, but information on provided services

Agentinteraction

Directinteraction

Indirectinteraction

With a-prioriacquaintance

Agent discoverythrough middle agents

Middle agents &acquaintance models

Guided/mediatedby artifacts

Spatially foundedinteraction

INTERACTION MODELS IN MAS: A TAXONOMY

Indirect interaction models

• Agents interact through an intermediate entity

• This medium supplies specific interaction mechanisms and access rules

• These rules and mechanisms define agent local context and perception

• Time and space uncoupling

• Name uncoupling

Agentinteraction

Directinteraction

Indirectinteraction

With a-prioriacquaintance

Agent discoverythrough middle agents

Middle agents &acquaintance models

Guided/mediatedby artifacts

Spatially foundedinteraction

INTERACTION MODELS IN MAS: A TAXONOMY

Artifact-mediated interaction

• Agents access a shared artifact that– they can observe – they can modify

• Such artifact is a communication channel characterized by an intrinsically broadcast transmission

• Specific laws regulating access to this medium

• It represents a part of agents’ environment

Blackboard systems“Metaphorically we can think of a set of workers,

all looking at the same blackboard: each is able to read everything that is on it, and to judge when he

has something worthwhile to add to it.”(A. Newell, 1962)

Blackboard

W1 W2 Wn

Concurrent access control

Linda: a specific blackboard based system

• Tuple space: a sort of blackboard in which tuples (record-like data structures) can be inserted, inspected and extracted by agents

• Operations– out(t) puts a new tuple in the Tuple Space, after

evaluating all fields; the caller agent continues immediately

– in(t) looks for a tuple in the Tuple Space; if not found the agent suspends; when found, reads and deletes it

– rd(t) looks for a tuple in the Tuple Space; if not found the agent suspends; when found, reads it

– inp(t) looks for a tuple in the Tuple Space; if found, deletes it and returns TRUE; if not found, returns FALSE

– rdp(t) looks for a tuple in the Tuple Space; if found, copies it and returns TRUE; if not found, returns FALSE

Matching rules in Linda

• Example:out("string", 10.1, 24, "another string")real f; int i;rd("string", ?f, ?i, "another string") succeedsin("string", ?f, ?i, "another string") succeedsrd("string", ?f, ?i, "another string") does NOT

succeed• Example:

out(1,2)rd(?i,?i) does not succeed (whatever is the type of i)

From Linda, to mobility and beyond

• Distributed tuple spaces: these systems allow to have a conceptually shared tuple space that is spread in a distributed environment

• More than just distribution– Programmable, reactive tuple spaces: adding a

behaviour to tuple spaces– Including organizational abstractions (roles,

policies) to enhance access rules

• References: M. Mamei, F. Zambonelli

Artifact-mediated interaction models: pros and cons

• Advantages– The artifact represents an abstraction of agents’

environment, and the burden of interaction is moved from the agents to their environment

– Interaction is mediated, and can thus be controlled (enforcement/enactment of organizational rules)

• Issues– Complex implementation (in distributed

environments)– How to integrate different artifacts and contexts ?

Agentinteraction

Directinteraction

Indirectinteraction

With a-prioriacquaintance

Agent discoverythrough middle agents

Middle agents &acquaintance models

Guided/mediatedby artifacts

Spatially foundedinteraction

INTERACTION MODELS IN MAS: A TAXONOMY

Spatially founded interaction

• Artifact mediated interaction are a first step in agents’ environment modelling

• Such artifacts represent very focused parts of the environment, and cannot consider the parts of agents’ context that does not pertain the specific artifact– They represent a single specific context of interaction

• Other approaches bring the environment metaphor to a deeper level, providing spatially founded interaction mechanisms

• Spatial features of the environment are explicitily considered by interaction mechanisms

Ancestors of Spatial Interaction: CAs

• A Cellular Automata (CA) is a set of homogeneous cells, evolving in discrete time steps

• Cells form a regular n-dimensional lattice– Homogeneous neighborhood (e.g. Von Neumann, Moore)

• Cells characterized by– A state, belonging to a finite set representing possible cell states– A transition rule, describing cell state dynamics

• Cell sort of reactive agent– Which cannot move in the environment – Can only interact with neighbouring cells according to precisely

defined rules

von Neumann Neighbourhood

Moore Neighbourhood

Extended Moore Neighbourhood

Swarm (and the likes) agent environment

• Swarm and many derived projects provide specific environments in which agents may be placed and interact

• Regular lattices supporting diffusion of signals that are– Emitted by entities – Spread in the spatial structure– Affecting other entities– Evaporating over time

• Diffusion strictly related to specific environmental structures

Spatialstructure

Agents andbehaviours

At-a-distanceinteraction

A coordination model for self-organizing agents

[S. Bandini, S. Manzoni, C. Simone, Dealing with Space in Multi-Agent System: a model for Situated MAS, in Proc. of AAMAS 2002, ACM Press, New York, 2002]

SCA (MMASS) –Formal and computational framework where to describe, represent and simulate complex systems according to a situated MAS approach

Coordination as result of interactions

Field-based interaction model

- Indirect interaction model between agents

- Intrinsically multicast

- Agent interactions occur when agent states are “compatible”

Interaction through Fields

• Fields are generated by agents to interact at-a-distance and asynchronously

• f = <Wf, Diffusionf, Comparef, Composef>– Wf: set of field values– Diffusionf: P X Wf X P Wf X…XWf

field distribution function– Composef: Wf …XWf Wf

field composition function– Comparef: Wf X Wf {True, False}

field comparison function

Agents Perception

T < ∑T, PerceptionT, ActionT>

Set of states that agents of type T can assume

Set of allowed actions for agents of type T

PerceptionT: ∑T [N X Wf1] …[N X Wf|F|]•PerceptionT(s) = (cT(s), tT(s))•cT(s): coefficient applied to field values•tT(s): sensibility threshold to fields•An agent perceives a field fi when

CompareT(ciT(s)…wfi,ti

T(s)) is True

Field based interaction: emission & perception

• Fields are signals emitted by agents and diffused in the environment

• Their intensity is possibly modulated in their diffusion

• Other agents may perceive these signals according to their perceptive capability, state and the signal value they receive

• Effect of perception defined by agent behavioural specificationCompareT(f×c,t) = false

CompareT(f×c,t) = true

CompareT(f×c,t) = falseemit(f)

Agent Coordination Language: primitives

action: emit(s,f,p)condit: state(s)effect: present(f, p)

action: trigger(s,fi,s’)condit: state(s), perceive(fi)effect: state(s’)

Subway station scenario

• Various crowd behaviors can take place

• Passengers' behaviors difficult to predict

• Crowding dynamics emerges– Social interactions between

passengers social rules– Interactions between single

passengers and the environment (signs, doors, constraints)

action: transport(p,fi,q)condit: position(p), empty(q), near(p,q), perceive(fi)effect: position(q), empty(p)

Coordinated movement in space