Sept 14, 2015

Introduction to Intelligent Agent

Dr. Hao WANG, R&L Group, CS, NJU

2

What is an intelligent agent?

Fields that motivate agent technologies Artificial intelligence

AI is the intelligence by machines or software Major AI researchers and textbooks define this field

as "the study and design of intelligent agents"

[wikipedia: artificial intelligence]

Artificial Intelligence: Foundations of Computational Agents [link], Cambridge University Press, 2010

"AI is the filed that studies the synthesis and analysis of computational agents that act intelligently. "

Agent is the carrier of artificial intelligence.

Topics in the book: reasoning, learning, planning, etc.

3

What is an intelligent agent?

Fields that motivate agent technologies Software engineering

SE is the study and an application of engineering to the design, development, and maintenance of software.

[wikipedia: software engineering]

M. Wooldridge [Wooldridge 1998]

System

Environment

input output

[Wooldridge 1998] M. Wooldridge, Agent and software engineering, in: AIIA, 1998.

OS, physical world, ...

functional,

reactive,

...

* Image from http://www.ted.com/talks/raffaello_d_andrea_the_astounding_athletic_power_of_quadcopters, Aug 4, 2015.

*What is a good software / system?

Complexities come from: environment, interaction, system specification, …

4

What is an intelligent agent?

Fields that motivate agent technologies Distributed system

A software system in which components located on networked computers communicate and coordinate their actions by passing messages

[wikipedia: distributed computing]

战区高空防御导弹战区防空与导弹防御

军事战略战术中继卫星

联合监测目标攻击雷达系统

天基红外系统

5

What is an intelligent agent?

Fields that motivate agent technologies Economics

In economics, an agent is an actor and more specifically a decision maker in a model of some aspect of the economy. [wikipedia: agent (economics)]

[wikipedia: program trading]

[wikipedia: algorithmic trading]

[wikipedia: high-frequency trading]

6

What is an intelligent agent?

Fields that motivate agent technologies Economics

In economics, an agent is an actor and more specifically a decision maker in a model of some aspect of the economy. [wikipedia: agent (economics)]

paid listing

gaming / bidding

7

Gradual changes lead to crisis

Big data Volume Velocity Variety Veracity

Example: find the top-k frequent integers in… an array, a file, a database, a distributed archive, a

stream

* Image from http://www.datasciencecentral.com/profiles/blogs/data-veracity, Aug 6, 2015.

*

With the growth of complexity, there will be a point where we need brand new tools and even brand new ways of thinking.

8

Agent: general definitions

Merriam-Webster A person who does business for another

person: a person who acts on behalf of another

A person or thing that causes something to happen

[Russell&Norvig 2009] S. Russell & P. Norvig, Artificial Intelligence: A Modern Approach (3rd edition), Prentice Hall, 2009.

S. Russell & P. Norvig An agent is anything that can be viewed as perceiving

its environment through sensors and acting upon that environment through effectors.

[Russell&Norvig 2009]

9

[Maes 1995] P. Maes, Artificial life meets entertainment: lifelike autonomous agents, Communications of the ACM, 38(11): 108-114, 1995.

P. Maes Autonomous agents are computational systems that inhabit

some complex, dynamic environment, sense and act autonomously in this environment, and by doing so realize a set of goals or tasks for which they are designed.

[Maes 1995]

Agent: general definitions

D. Smith, A. Cypher & J. Spohrer Let us define an agent as a persistent software entity

dedicated to a specific purpose. “Persistent” distinguishes agents from subroutines; agents

have their own idea about how to accomplish tasks, their own agendas. “Specific purpose” distinguishes them from entire multifunction applications; agents are typically much smaller.

[Smith et al. 1994]

[Smith et al. 1994] D. Smith, A. Cypher & J. Spohrer, KidSim: programming agents without a programming language, Communications of the ACM, 37(7): 54-67, 1994.

10

[Hayes-Roth 1995] B. Hayes-Roth, An architecture for adaptive intelligent systems, Artificial Intelligence, 72(1): 329-365, 1995.

B. Hayes-Roth “Intelligent agents” continuously perform three functions:

perception of dynamic conditions in the environment; action to affect conditions in the environment; and reasoning to interpret perceptions, solve problems, draw inferences, and determine actions.

Conceptually, perception informs reasoning and reasoning guides action, although in some cases perception may drive action directly.

[Hayes-Roth 1995]

Agent: general definitions

11[Wooldridge&Jennings 1995] M. Wooldridge & N. R. Jennings, Intelligent agents: theory and practice, Knowledge Engineering Review, 10: 115-152, 1995.

… the term agent is used to denote a hardware or (more usually) software-based computer system that enjoys the following properties:

Autonomy: agents operate without the direct intervention of humans or others, and have some kind of control over their actions and internal state;

Reactivity: agents perceive their environment, and respond in a timely fashion to changes that occur in it;

[Wooldridge&Jennings 1995]Agent: a weak notion

12[Wooldridge&Jennings 1995] M. Wooldridge & N. R. Jennings, Intelligent agents: theory and practice, Knowledge Engineering Review, 10: 115-152, 1995.

… the term agent is used to denote a hardware or (more usually) software-based computer system that enjoys the following properties:

Proactiveness: agents do not simply act in response to their environment; they are able to exhibit goal-directed behavior by taking the initiative;

Social ability: agents interact with other agents (and possibly humans) via some kind of agent-communication language.

[Wooldridge&Jennings 1995]Agent: a weak notion

13

[Wooldridge&Jennings 1995] M. Wooldridge & N. R. Jennings, Intelligent agents: theory and practice, Knowledge Engineering Review, 10: 115-152, 1995.

Stronger notions = weak notions + … Mentalistic notions: conceptualized or implemented using

notions such as Knowledge Belief Intention Obligation

Emotional notions human-like

[Wooldridge&Jennings 1995]Agent: stronger notions

* Image from http://edition.cnn.com/2014/07/11/opinion/bare-underwater-robot-lessons/, Aug 10, 2015.

*

** Image from http://aramatheydidnt.livejournal.com/1475842.html, Aug 10, 2015.

**

14[Wooldridge&Jennings 1995] M. Wooldridge & N. R. Jennings, Intelligent agents: theory and practice, Knowledge Engineering Review, 10: 115-152, 1995.

[Wooldridge&Jennings 1995]Agent: other properties

Other properties Mobility: agent is able to move around an electronic

network (to perform local processing of information);

Veracity: agents will not knowingly communicate false information;

Benevolence: agents do not have conflicting goals, and that every agent will therefore always try to do what is asked of it;

Rationality: an agent will act in order to achieve its goals, and will not act in such a way as to prevent its goals being achieved.

15

An agent inhabits in its environment, which is typically dynamic & complex.

An agent perceives its environment, and affects its environment by its actions, and from its perception to its action there is typically a decision-making process (reasoning, planning, …).

An agent usually has the so-called weak characteristics: autonomy, reactivity, proactiveness, and social abilities.

An agent may have stronger characteristics, being mentalistic or even emotional.

In some applications, an agent may be designed with emphases on other characteristics, such as mobility, veracity, benevolence, and rationality.

Agent: summary

16

Synonyms of the term “intelligent agent” Robot, software agent, softbot, knowbot, taskbot, userbot, …

Chinese translations of the term “intelligent agent” “ 智能代理”

A person who does business for another person: a person who acts on behalf of another

“ 智能体”、 “智能主体” A thing that causes something to happen; the carrier of AI

“ 智能艾真体” “ 以音译为基础，又兼顾了一定的物理含义，而且‘艾’含有漂

亮的意思。”

Synonyms and Chinese Translations

[Cai&He 2002, Cai 2003?]

[Cai&He 2002] 蔡自兴 , 贺汉根 , 智能科学发展的若干问题 , 自动化学报 , 28(suppl): 142-150, 2002.

[Shi 2000]

[Shi 2000] 史忠植 , 智能主体及其应用 , 科学出版社 , 2000.

[Cai 2003?] 蔡 自 兴 , 对 Agent 中 文 译 法 的 讨 论 , Available online: http://downlunwen.zdnet.com.cn/H047542/pdf/1.pdf, Accessed: Aug 6, 2015.

17

Examples of agents

Air-conditioner / any control system Thermostat

Software daemon Print server Http server

18

Think…

Find some other examples of agents (not necessarily intelligent); in particular, specify:

1. The environment that the agent inhabits;

2. Possible actions of the agent, and how each of them will change the environment;

3. The goal or design purpose of the agent.

19

Think…

If a traffic light together with its control system is designed as an intelligent agent, what are the properties that this agent should exhibit?

1. Weak properties? (autonomous, reactive, proactive, social)

2. Strong properties? (mental, emotional)

3. Other properties? (mobile, veracious, benevolent, rational)

20

The environment

Why is the ENVIRONMENT important when we are interested in AGENTS? From the perspective of problem solving, the environment,

together with the goal of the agent, defines the PROBLEM; Understanding the environment can largely guide the design

of the intelligent agent.

21

The environment [Sutton&Barto 1998]

The thing the agent interacts with, comprising everything outside the agent is called the environment.

Agent-environment boundary is not often the same as the physical boundary of a robot’s body. Motors, mechanical linkages, sensing hardware

Anything that cannot be changed arbitrarily by the agent should be considered part of its environment.

22

Properties of environments

Accessibility: accessible vs. inaccessible

Determinism: deterministic vs. nondeterministic

Dynamism: static vs. dynamic

Continuity: discrete vs. continuous

[Russell&Norvig 2009] S. Russell & P. Norvig, Artificial Intelligence: A Modern Approach (3rd edition), Prentice Hall, 2009.

[Russell&Norvig 2009]

Inaccessible (partially observable) The agent needs to keep track of the environment through its

internal state. Example: observe the current weather

in a windowless room at the South Pole of Pluto

Accessible + noise = inaccessible

23

Accessibility

Accessible (fully observable) An accessible environment is one in which can obtain

complete, accurate, up-to-date information about the environment’s state.

Example: observe the weather, is it sunny / rainy / cloudy / …?

Most real-world environments are inaccessible.

The more accessible an environment is, the simpler it is to build agents.

Nondeterministic (stochastic) Nondeterminism captures the facts that…

Agents have at best partial control over their environment Actions may fail to have the desired result

24

Determinism

Deterministic A deterministic environment is one in which any action has

single guaranteed effect. There is no uncertainty about the state that will result from

performing an action.

A deterministic environment may exhibit nondeterminism if it’s sufficiently complex

Lack of a good environment model Lack of complete sensor coverage

Dynamic A dynamic environment is one that has other processes

operating on it, and which hence changes in ways beyond the agent’s control.

Two important properties: If the agent performs no external action between times and ,

then it can NOT assume that the environment at will be the same as it was at time .

Other processes can interfere with the actions the agent attempts to perform.

25

Dynamism

Static A static environment is one that can be assumed to remain

unchanged except by the performance of actions by the agent.

Continuous A continuous environment is one that may be in uncountably

many states. The information a discrete-state agent uses in order to select

an action in a continuous environment will be made on the basis of information that is inherently approximate.

26

Continuity

Discrete A discrete environment is one that can be guaranteed to only

ever be in a finite number of discrete states.

Most complex general class of environments Inaccessible, nondeterministic, dynamic, and continuous a.k.a. open environments

Simplest case The agent knows (1) the maze and (2) the goal position. Actions of the agent have deterministic effects. The maze is fixed. accessible, deterministic, static, and discrete

27

Examples of environments: maze games

Maze games are common testbeds for intelligent agents

N

S

EW

agentgoal position

obstacles

If actions are performed via motors… There are chances of mechanical failure accessible, nondeterministic, static, and discrete

28

Examples of environments: maze games

N

S

EW

agentgoal position

obstacles

If, in addition, the sight is limited… inaccessible, nondeterministic, static, and discrete

29

Examples of environments: maze games

If there is another agent, or the maze is random… inaccessible, nondeterministic, dynamic, and discrete

If the maze becomes more practical…

inaccessible, nondeterministic, dynamic, and continuous

30

Think…

Determine the types of the following environments

Mine-sweeper

E-shopping

PokerChess

Deterministic?

Accessible?

Discrete?

Static?

31

Agents vs. functional systems

[Recall] An intelligent agent is: Proactive: goal-directed Reactive: perceive and respond to its environment Sociable: interact with other agents / human

Functional system Precondition + postcondition Example: design by contract (DbC)

*

* Image from https://en.wikipedia.org/wiki/Design_by_contract (license: CC0 1.0), Aug 11, 2015.

accessible, deterministic, static, discrete

inaccessible, nondeterministic, dynamic, continuous

32

Agents vs. objects

Object Computational entities that encapsulate some state, are able to

perform actions, or methods on this state, and communicate by message passing

Combination of variables, functions, and data structures

[Wooldridge&Ciancarini 2001] M. Wooldridge & P. Ciancarini, Agent-oriented software engineering: the state of the art, in: Proceedings of the 1st International Workshop on Agent-oriented Software Engineering, 2001.

[Wooldridge&Ciancarini 2001]

[Wikipedia: Object (computer science)]

a set of assignments to variables

WALL·E

move_north( )

move_south( )

move_east( )

move_west( )

(x, y)

33

Agents vs. objects

Differences between agents and objects The level of autonomy

Objects have control over its internal state (encapsulation: private variables)

WALL·E

move_north( )

move_south( )

move_east( )

move_west( )

(x, y)

Objects invoke other objects’ methods, whereas agents can only request other agents to perform actions.

Objects do it for free; agents do it for money.

Objects have no control over its behavior: once a method is made public, objects cannot decide whether or not that method is executed.

34

Agents & OOP

OOP: object-oriented programming

Standard OOP does not require agent-like properties.

In standard OOP, there is a single thread of control, whereas in principle each agent should have its own thread of control.

Agent is similar to active object.WALL·E

move_north( )

move_south( )

move_east( )

move_west( )

(x, y)

run( )

35

Agents vs. expert systems

Expert system Computer system that emulates the decision-making ability of

human expert

Designed to solve complex problems by reasoning about knowledge, represented primarily as if-then rules rather than through conventional procedural code

Key difference Expert systems are not coupled to their environment;

Expert systems are not designed for reactive, proactive behavior;

Expert systems do not consider social ability.

[Wooldridge 2009] M. Wooldridge, An Introduction to Multiagent Systems (2nd edition), John Wiley & Sons, 2009.

[Woooldridge 2009]

[Wikipedia: expert system]

36

Distributed Artificial Intelligence (DAI)

DAI is a subfield of AI Dedicated to the development of distributed

solutions for complex problems regarded as requiring intelligence

Main areas of DAI

– Distributed problem solving (DPS)

Centralized control and distributed data

– Multi-agent system (MAS)

Distributed control and distributed data

[Wikipedia: distributed artificial intelligence]

37

DAI considers…

Agent granularity (agent size) Heterogeneity of agents (agent type) Methods of distributed control (among agents) Communication

MAS– Coarse agent granularity– High-level communication

distributed computing

distributed AI

AI

MASDPS

38

Motivation behind MAS

To solve problems too large for centralized agents– e.g., economic system

To allow interconnection and interoperation of multiple legacy systems– e.g., web crawling

To provide solutions to inherently distributed systems

To provide solutions when expertise is distributed

To provide conceptual clarity and simplicity of design

39

Benefits of MAS

Faster problem solving

Decreasing communication

– Higher semantics of communication (speech-

act level)

Flexibility

Increasing reliability

40

Heterogeneity degrees in MAS

Low

– Identical agents, different resources

Medium

– Different agent expertise

High

– Share only interaction protocol (e.g. FIPA or

KQML)

41

Cooperative and self-interested MAS

Cooperative– Agents are designed by interdependent designers

– Agents act for the common good of the system

– Focus on the performance of the entire system, not

the interests of individual agents

Self-interested– Agents are designed by independent designers

– Agents have their own agenda and motivation

– Concerned with the benefit of each agent

(‘individualistic’)

Which is more realistic in an Internet-setting?

42

Our categorization of MAS

Cooperative MAS (teamwork)– Agents share a common objective

Competitive MAS– Each agent has its own objective, and these

objectives are contradictory

Coopetitive ( 竞合 ) MAS– Each agent has its own objectives; these objectives

are typically competitive, but there is still possibility

that agents may cooperate to achieve their common

good

43

Summary

Agent has general definitions, weak definitions,

and strong definitions

Classification of the environment

Differences between agents and functional

systems, objects, expert systems, etc.

A brief introduction to multi-agent systems