A Personal Odyssey in the World of Multi-Agent Research

Victor R. Lesser

Computer Science Department

University of Massachusetts, Amherst

July 26, 1999

2

Theme

To provide a personal perspective on how a research career develops

How ideas get created and evolve over time

My personal research agenda

Thoughts on where the field should be going

3

Creation and Evolution of MAS/DAI Approaches

Functionally/Accurate Cooperative (FA/C) paradigm (Corkill and Carver) Tolerance of inconsistency

Range of acceptable answers

Error resolution through exchange of partial and tentative subproblem

solutions

Layered Agent Control (Corkill) Organizational structuring

Satisficing agent coordination

Interplay between local and non-local control

4

MAS/DAI ideas (continued)

Partial Global Planning (PGP) and its successor GPGP as a framework for real-time agent coordination (Durfee & Decker) Quantitative view of coordination based on a distributed search model

Taxonomy of Subproblem interactions

Agent Task Language —TAEMS

Fault-detection, diagnosis and adaptation as mechanism for

coordination adaptation (Hudlicka & Sugawara)

Negotiation as a distributed search

Multi-stage negotiation (Conry, Kuwabara & Meyer)

Level-commitment as protocol for search by self-interested agents (Sandholm)

5

Precursor Research

Multi-computer operating system (1963-65) Excitement and complexity

Reconfigurable Multi-processor Architecture (1966-1972) Dynamic Mapping of Process network on to Processor network

Hardware kernel micro-operating system

Mapping had to take into account of interrelationship among processes Issues of scale

Control working set

Importance of careful attention to empirical dataDetail simulation

6

Hearsay-II: Parallel, Cooperating Knowledge-Source Model (1973-1975)

Blackboard Architecture

Title: hearsayII.EPSCreator: Adobe Photoshop Version 4.0Preview: This EPS picture was not saved with a preview (TIFF or PICT) included in itComment: This EPS picture will print to a postscript printer but not to other types of printers

7

Functional Descriptionof the Speech-Understanding KSs

8

Key Ideas in Blackboard Architecture

Distributed, multi-level asynchronous search Integrated search at different abstraction levels

Error-resolution through search and combining of approximate knowledge

Sophisticated control reasoning Use of approximate knowledge for control Probabilistic view of search control

Ideas and lesson learned in HS-II will be an important influence on future work

9

Transition to Distributed AI research — the importance of serendipity

Parallel Processing Experiments (Fennel)

The Wisdom of Allen Newell in forbidding me to talk

Conversation with Bob Kahn (one of the founders of the Internet)

10

Distributed Hearsay-II Experiments (Erman, 1977-1979)

Three node System with Overlapping Acoustic Data

Communication of only High-Level Results

No changes to basic architecture except for transmit and receive KSs

11

Network of Hearsay-II

Systems

sensor1

sensor2

sensor3

12

Results of Experiments

Reproduced Results of Centralized System

Slight Speed up and Reduced Communication

Robustness in face of errorful communication channel

Handled 35% error rate

13

Lessons — It worked but...!

Lack of coherent behavior Distraction Inappropriate Communication/Computation

RedundantLack of timelinessLack of Focus

Simplistic Local and Network Control Inadequate Local agent control needs to be more sophisticated when

taking into account interactions with other agents

14

Key Question that Focused Research in the 1980’s

Can computationally tractable cooperative strategies be developed that maintain both coherent agent activity and system robustness?

Implicit recognition of tension between reactive and reflective control

15

DVMT: an Environment for Research in Cooperative Distributed Problem Solving

Build and evaluate more complex local agent control, coordination strategies and organization strategies

Title: carver-DVM.EPSCreator: Adobe Photoshop Version 4.0Preview: This EPS picture was not saved with a preview (TIFF or PICT) included in itComment: This EPS picture will print to a postscript printer but not to other types of printers

16

DVMT: Agent Architecture(Corkill, 1983)

Static Meta-level Control Organizational structuring

Goal-Directed requests for information Integrating external and internal requests for processing

17

Integrating Data and Goal-Directed Control and Organizational Structuring

18

Limitations of Static Meta-Level control (1987)

Transmission of meta-level state information — only

partially successful

Missing information about future activities

Transmission of activity plans

Partial global planning (Durfee)

19

PartialGlobal

PlanningArchitecture

20

Partial Global Planning(Durfee)

Representation of near-term agent activities

Intermediate goals of activities

Region and likely vehicles

Behavioral characteristics

Timing and likelihood of success

Relationship of interagent activities Spatial overlapping and adjacent interpretation regions

Basis for reorganizing local activities Exploiting predictive information

Avoiding redundant activities

Allow for load sharing

21

Some Important Digressions in Local Agent Control

Meta-level control through Diagnosing of Problem-Solving Behavior (Hudlicka, 1984) Led to work on learning new situation specific coordination

rules via detection and diagnosis (Sugawara, 1993) RESUN — a framework for problem solving control based on

symbolic reasoning about source of uncertainty (Carver, 1989) Led to work on DRESUN which provide a distributed

framework for focused communication to resolve inconsistent agent beliefs (Carver, 1992-1997)

22

Further Work on Local Control

Design-to-Time Scheduling (Garvey, Wagner) Approach to real-time agent control by dynamically

constructing a schedule of activities to meet real-time deadlines

Exploit the existence of alternative algorithms that trade off quality of solution for resource usage

Led to deeper understanding of the issues of uncertainty

Techniques for Sophisticated Local Control have strong implications for Non-Local Control

23

Digressions into the World of Negotiation

Trying to understand cooperative and self-interested contracting

Multi-stage negotiation (Conry et al., Lander, Laasris, Moehlman, Neiman) Cooperative dialogue among agents (1987-1997)

Self-Interested Agent Interaction (Sandholm) Level commitment negotiation protocol

Digressions are sometimes important for validating old intuitions and gaining new ones

24

Key Questions that Focused Research in the 1990’s

Is there some deeper theory of Agent Coordination implicit in this work

Can we create infrastructure/ frameworks that eliminate a lot of the work in constructing MAS systems

The search for the Holy Grail!

25

Generalizing PGP (Decker)

DATA/Resources

A distributed goal search tree involving Agent1 and Agent2. The dotted arrows indicate interdependencies

between goals and data in different agents, solid arrows dependencies within an agent. The superscripts

associated with goals and data indicate the agent which contains them (Jennings, 1993).

Agent1 Agent2

G10 G2

0

G11 G1

2 . . . . . . G1k G1,2

m G2p . . . . . . . . . . . G2

t

G11,1 G1

1,2 G1m,1 G2

m,2 G2p,1 G2

p,2

G1m,1,1 G1

m,1,2 G2p,1,3 (G2

p,1,4) G2p,2,2

AND

AND AND

ANDAND

OR

OR

OR

d11 ………………………………………. d1

j d2j+1 ……………………………………………. d2

z

26

TAEMS: A Domain Independent Framework for Modeling User Activities

The top-level goals/objectives/abstract-tasks that an

agent intends to achieve

One or more of the possible ways that they could be

achieved, expressed as an abstraction hierarchy whose

leaves are basic action instantiations, called methods

A precise, quantitative definition of the degree of

achievement in terms of measurable characteristics such

as solution quality and time

27

TAEMS (continued)

Task relationships that indicate how basic actions or abstract task achievement affect task characteristics (e.g., quality and time) elsewhere in the task structure Hard relationships (e.g., enables) denote when the result from one

problem-solving activity is required to perform another, or when performing one activity precludes the performance of another

Soft relationships (e.g. facilitates) express the notion that the results of one activity may be beneficial (or harmful) to another activity, but that the results are not required in order to perform the activity.

The resource consumption characteristics of tasks and how a lack of resources affects them.

28

Information Gathering Example

Recommend a High-End PC System

Make Decision

MoneyResource

Build ProductObjectsOutcomesNum Prod 1-4Num Prod 5-8Num Prod 9-12Num Prod ...

Get Basic ProductInformationQuery & ExtractVender mQuery & ExtractPossible Maker n

Gather ReviewsSearch & ProcessZDnet Reviews Search & ProcessPC World

Query & ProcessConsumers Reports

q_sum_all()q_sum() q_sum()

q_seq_last()

q (10% 0)(90% 10)c (100% 0)d (10% 2min)(10% 2.5min)(80% 3min)q (20% 0)(80% 8)c (100% 0)d (50% 1min)(50% 2min)Query & ExtractPC Connection Query& ExtractNECX q (25% 0)(75% 20)c (100% $2)d (90% 3)(10% 5)

q (15% 0)(75% 10)c (100% 0)d (30% 3min) (30% 4min) (40% 5min)q(..), c(..), d(..)q(..), d(..), c(..)

consumes $2limitsq multiplier (100% 0)c multiplier (x)d multiplier (x)

q(..), c(..), d(..)

facilitates & hindersfacilitates & hindersq multiplier (100% +20%)d multiplier (100% +20%)

TaskMethodResource nleTask nleSubtask RelationKey

enables.........q_sum()

29

30

Important Aspects of TAEMS

Abstract View of Agent Activities level of detail necessary for understanding interactions and scheduling

decisions

Relationships among activities based on data flow: enables, facilitates, favor, disable, etc.

Relationships among activities how they contribute to the overall goal — quality accumulation functions: min, max, sum, etc.

Schedule represents policy — guidance for resource consumption and goals

Worth Oriented Coordination as an Optimization Problem

Real-time deadlines

31

WARREN Style Model of Multi-Agent Information Gathering

MoneyResource

enables

enablesRecommend a High-End PC System

Make Decisionq_sum_all()q_seq_last()Build ProductObjects

Get Basic ProductInformationNon-local Task Gather ReviewsNon-local Task

Get Basic ProductInformationQuery & ExtractVender mQuery & ExtractPossible Maker n

q_sum()Query & ExtractPC Connection BuyInformation

Gather ReviewsSearch & ProcessPC Worldq_sum()Search & ProcessZDnet ReviewsQuery & ProcessConsumers Reports

task contractingtask contracting

consumes / limitsconsumes/ limits

Task Agent

InformationAgentInformation Agent

32

GPGP Agent Architecture

Title: agent-arch-fixed-subset.fig

Creator: fig2dev Version 3.1 Patchlevel 1

Preview: This EPS picture was not saved with a preview (TIFF or PICT) included in it

Comment: This EPS picture will print to a postscript printer but not to other types of

printers

33

Clear Separation of Local Control from Coordination

Coordination is generation of commitments Importance, utility, negotiability, decommitment

Commitments lead to constraints on local scheduling Earliest start time of a task

Deadline for completion of a task

Interval when can’t be executed

34

Integrating GPGP with other Approaches

Title: evolution_gpgp3Creator: FreeHand 7.0Preview: This EPS picture was not saved with a preview (TIFF or PICT) included in itComment: This EPS picture will print to a postscript printer but not to other types of printers

No person is an island unto themselves!

35

Putting it all together — An architecture for Large Agent Societies

This EPS image does not contain a screen preview.

It will print correctly to a PostScript printer.

File Name : agent-arch.eps

Title : agent-arch.fig

Creator : fig2dev Version 3.1 Patchlevel 1

CreationDate : Mon Jan 6 14:19:42 1997

Pages : 0

36

Personal Perspective on MAS based on this research path

Agent Flexibility in Open Environments Agents need to be able to adapt their local problem solving to the available resources

and goals of the system. Long-term learning needs to be an integral part of an agent architecture

Agents not restricted to solving one goal at a time but may flexibly interleave their activities to solve multiple goals concurrently

Error resolution/management needs to be integral part of agent problem solving

Satisficing control Less than optimal but still acceptable levels of coordination among agents is traded off

for a significant reduction in computational costs to implement cooperative control. Emphasis on satisficing behavior subtly moves the focus from the performance of

individual agents to the properties and character of the aggregate behavior of agents.

37

Personal Perspective (continued)

Predicting Performance of MAS systems is possible via probabilistic analysis Requires detail model of the environment

Interaction between local and non-local agent control For effective agent coordination local agent control must have a certain

level of sophistication in order to be able to understand what it has done, what is currently doing and what it intends to do

Agent Roles and Responsibilities for large agent societies organizing agents in terms of roles and responsibilities can significantly

decrease the computational burden of coordinating their activities.

38

Personal Perspective (continued)

Centrality of Commitment to coordinated behavior Both long- and short-term coordination can be viewed in terms of

commitments that have varying duration and specificity.

Layered Control Modulation—higher layers providing constraints (policies) to lower

levels that modulate (circumscribe) their control decisions

Bi-directional Interaction(negotiation) among Layers — Though constraints flow down the layers, information that flows in the other direction allows these constraints to be modified in case they can’t be met or they lead to inappropriate behavior

39

Personal Perspective (continued)

Situation-specificity There is no one best approach to organizing and controlling

computational activities for all situations when the computational and resource costs of this control reasoning is taken into account.

Quantitative View of Coordination Efficient and effective coordination must account for the benefits and

the costs of coordination in the current situation.

Coordination can be seen as a distributed mechanism for approximating a global optimization problem of task assignment

40

Personal Perspective (continued)

Domain-independence —The aspects of a domain that affect coordination can be abstracted and represented in a domain-independent language. An agent’s goals and criteria for their successful performance

The performance characteristics and resource requirements of the alternative methods it possesses for accomplishing its goals,

Qualitative and quantitative interdependencies among its methods and those of other agents

41

Personal Perspective (continued)

Representing and Reasoning about Assumptions To the degree that the system can either re-derive or explicitly

represent the assumptions behind these control decisions The more the system can effectively detect and diagnose the

causes for inappropriate or unexpected agent behavior.

Importance of Experimentation — we are still an experimental science Don’t yet have good ways to predict performance Statistical analysis is important but don’t forget to look at the

details

42

Current Research Projects

Organizational Structuring, Design and Adaptation for Large Agent Societies (Horling, Vincent, Wagner) Real-time negotiation

Layered, Domain-Independent Coordination GPGP-II (Wagner and Xuan) JIL to GPGP (Raja and Zhang)

Distributed Situation Assessment/DRESUN Satisficing termination (Carver) Distributed Dynamic Bayesean Network and Influence Diagrams

(Carver, Xiang, Zhang, Zilberstein)

43

Current Research Projects, Cont’d

Survivable MAS Systems (Xuan) Coordinating for fault-tolerance Distributed Markov Decision Processes

Resource Bounded Reasoning/Design-to-Criteria Scheduling (Raja and Wagner)

Application Focus: Cooperative Information Gathering Intelligent Home Supply Chain Manufacturing

44

Questions on my mind

Is there a unified perspective with in which self-interest and cooperative agents can be understood? Is that perspective distributed search?

How viable is it to think about emotions and power relationships as computational mechanisms making it possible to approximate the global optimal solution in a distributed way through local optimizations? Early work on skeptical nodes

What type of meta-level framework (with limited and bounded computational overhead) will allow us reason about coordination costs as first-class objects so that it possible to dynamically balance problem-solving activities with coordination activities?

45

Questions on my mind (continued)

Are Markov Decision Processes a computationally viable approach for dynamic multi-agent coordination?

What knowledge and reasoning is necessary for designing top-down an agent organization? In what situation can bottom-up evolutionary organizational

structuring produce good organizations

Will the world of MAS/DAI be dominated in the future by game theoretic ideas and market mechanisms? what is the role of cooperative agents?

46

MAS in the 21st Century—A Dominant Model

Cooperating, Intelligent Agent Societies (seamless integration among people/machines)

Constructionist perspective built out of heterogeneous, semi-autonomous agents

having varying motivations from totally self-interested to benevolent

High-level artificial language for cooperation

Problem solving for effective cooperation will be as or more sophisticated than the actual domain problem solving reasoning about goals, plans, intentions, and knowledge of other agents

47

MAS in the 21st Century, Cont’d

Operate in a “satisficing” mode Do the best they can within available resource constraints Deal with uncertainty as an integral part of network problem solving Complex organizational relationships among agents

Highly adaptive/highly reliable Learning will be an important part of their structure

(short-term/long-term) Able to adapt their problem-solving structure to respond to

changing task/environmental conditions

Profound implications for AI & Computer Science!

48

Important Directions for the Field to Realize this Vision

Development of software infrastructure to help build sophisticated, interacting agents What will it be wrappers, languages or frameworks or

some combination?

Techniques for Sharing of knowledge/data among heterogeneous agents Are ontologies the answer or will there be the need for

more sophisticated knowledge translation approaches or specialized languages?

49

Directions (continued)

Mechanisms for dynamically establishing interaction protocols among heterogeneous agents Are recent ideas such as “civil agent societies” by

Dellacros and Klein a viable approach?

Analysis tool for understanding the performance of such systems before they are implemented

Design rules and mechanisms for agent societies so that they will not evolve in ways that lead to inappropriate behavior or poor performance

50

Parting Thoughts

This is a very exciting time for researchers in MAS The practical application of this technology is here!

The set of ideas that the field has developed only scratch the surface There is a tremendous amount of work to be done

There are a lot of hard problems to work on

Let your intuitions drive you — not what is necessarily currently in fashion