1

©CS540 Spring 2013

Intelligent Agents: Software and Robotic

“An agent is just something that perceives and acts.”(R&N)

Properties: n  Provides some service to humans n  Has some autonomy n  Can adapt to its environment n  Can interact with other agents n  has knowledge about its tasks and environment.

©CS540 Spring 2013

Defining Agency as Relationship

n  One entity depends on another to undertake some task on its behalf.

n  Exploits specialization (computers do not object to tedium, agents have access to and knowledge of different information).

n  Distinguish agents by the information they access and process

©CS540 Spring 2013

Sycara’s Requirements for Agents Situatedness in either virtual or physical

environment Autonomy, acting without intervention from

humans and exerting control over own actions and internal state

Adaptivity, reacting flexibly to changes in environment to pursue goals, learns

Sociability, can act as peer with other agents and humans

©CS540 Spring 2013

Commonalities in Agent Definitions

Capability % in 25 Reactive 36 Autonomous 36 Software 32 Deliberative 28 Intentional 28 Cooperative or Communicative

24

Persistence 20 Adaptable 12

2

©CS540 Spring 2013

Key Issues

n  How do agents decide on goals? n  How do agents decompose tasks? n  How do agents communicate with each

other? With humans? n  What capabilities are needed? n  What kinds of architectures support

agent design?

Environment

Agent and Environment

©CS540 Spring 2013

Sense •  sensors •  perception

Plan •  Decide on

action •  Coordinate

Act •  Control •  Effectors

Properties of the Environment n  Accessible vs. Inaccessible: complete, accurate, up-to-

date information about environment’s state vs. not n  Deterministic vs. non-deterministic: actions have single

guaranteed effect/ predictable vs. not n  Episodic vs. non-episodic: performance depends on

discrete episodes with no link between different scenarios n  Static vs. dynamic: unchanged except by actions of the

agent vs. other forces operating in the environment n  Discrete vs. continuous: fixed, finite number of actions and

states vs. not

©CS540 Spring 2013

Abstract Architectures

n  Environment is in a set E of discrete states:

n  Agents take actions which transform the state of the environment:

n  A run is a sequence of interleaved states and actions:

©CS540 Spring 2013

E = {e, !e ,...}

Ac = {α, !α ,...}

r : e0ao! →! e1

a1! →! ... au−1! →! eu

Adapted from M. Wooldridge http://www.csc.liv.ac.uk/~mjw/pubs/imas

3

Environments (formally)

n  A state transformer function represents the environment:

n  An environment is a triple

where E is a set of states, initial state is and τ is a state transformer function.

©CS540 Spring 2013

Adapted from M. Wooldridge http://www.csc.liv.ac.uk/~mjw/pubs/imas

Env = E,e0,τ

τ :RAc →℘ E( )

e0 ∈ E

Agent (formally)

n  Agent maps runs to actions:

by deciding what action to perform based on history of the environment.

©CS540 Spring 2013

Adapted from M. Wooldridge http://www.csc.liv.ac.uk/~mjw/pubs/imas

Ag :RE → Ac

©CS540 Spring 2013

Agent Architectures

n  Reactive n  Direct mapping between environment and

action n  Reinforcement learning, reactive planning

n  Deliberative n  Computation or reasoning between

sensing and acting

©CS540 Spring 2013

Theses Underlying Reactivity

n  The world is its own best model. n  No centralized control n  No internal reasoning/complicated computation n  Complex behavior emerges from interaction of

simple behaviors n  Hard-wired or pre-computed responses

n  Purely reactive if: action(e) :E→ Ac

4

©CS540 Spring 2013

Theses Underlying Deliberation

n  Internal representation of world n  Sense-Plan-Act cycle n  Probably some off-line, non-real-time

processing

Expected Utility

n  Probability that run r occurs when agent Ag is placed in environment Env

n  Expected utility of agent Ag in environment Env (given P, u) is:

©CS540 Spring 2013

Adapted from M. Wooldridge http://www.csc.liv.ac.uk/~mjw/pubs/imas

EU(Ag,Env) = u(r)P(r | Ag,Env)r∈R(Ag,Env)∑

P(r | Ag,Env) =1r∈R(Ag,Env)∑

Optimal Agents

n  Optimal agent maximizes expected utility, does best on average.

n  Bounded Optimal Agent is an optimal agent that can actually be implemented on a computer.

©CS540 Spring 2013

Adapted from M. Wooldridge http://www.csc.liv.ac.uk/~mjw/pubs/imas ©CS540 Spring 2013

Beliefs, Desires and Intentions (BDI)

n  A BDI agent is: n  Deliberative n  Describable by states of knowledge

(beliefs), goals (desires) and commitments to act (intentions)

n  Agent Programming Languages: Agent0

n  Agents Communication Languages

5

©CS540 Spring 2013

Roles for Intentions (Bratman)

§  Intentions direct future processing – select task for attention

§  Once commit to intention, then other intentions must be consistent with it.

§  Monitor progress by success of achieving, maintaining or abandoning intentions

©CS540 Spring 2013

Plans and Intentions

n  Plans provide means for satisfying intentions.

n  Agents coordinate activities via plans. n  Plans may need to be least commitment

to support retracting intentions.

©CS540 Spring 2013

Cohen & Levesque: Formal Theory of Rational Agency n  Modal Operators

(BEL x p) p follows from x’s beliefs (GOAL x p) p follows from x’s goals (HAPPENS a) a will happen next (DONE a) a just happened (AGT x a) x is the agent for actions a

n  Action Notation a.b action b follows a a|b non-deterministic choice a||b a and b occur concurrently p? p is true? a.p? after a occurs then p holds

©CS540 Spring 2013

C&L Example of Persistent Goal (P-GOAL x p q) =

(BEL x ~p) & (GOAL x (LATER p)) & (KNOW x (PRIOR [(BEL x p) or (BEL x ~p) or (BEL x ~q)] ~[GOAL x (LATER p)])]

Means agent x believes that p is currently false, makes its becoming true later into a goal, and x knows that before abandoning this goal. It must either believe it is true, believe it can never become true or believe some other condition holds that makes abandoning the goal worthwhile.

6

©CS540 Spring 2005

Procedural Reasoning System (PRS): A BDI Architecture

From SRI: http://www.ai.sri.com/~prs/prs-arch.html

©CS540 Spring 2005

Acts in PRS From SRI: http://www.ai.sri.com/~act/act-formalism.html

©CS540 Spring 2005

PRS Application: Space Shuttle Reaction Control

From SRI: http://www.ai.sri.com/~prs/rcs.html

©CS540 Spring 2013

Agent Programming Languages

n  Agent Oriented Programming, specialization of OOP n  Fixes form of agent’s state n  Fixes form of messages (e.g., KQML) n  Constrains methods (e.g., cannot commit

to incompatible actions)

7

©CS540 Spring 2013

Agent0 (Shoham) Agent mental state has three components:

n  Beliefs n  Commitments n  Capabilities (what actions can be committed,

action descriptions) Speech act messages

n  Inform about belief n  Request commitment n  Unrequest previous commitment

n  Agent trusts everything it is told.

©CS540 Spring 2013

Agent0 (cont.) Actions:

n  DO(time, privateaction) execute one of the agent’s capabilities

n  REFRAIN(action) do not commit to action n  IF mentalcond THEN action n  Speech act messages

Commitment Rules n  COMMIT(messagepattern, mentalcond, agent, action) if the agent receives a message of messagepattern that comes

from or mentions agent AND mentalcond is true AND the receiving agent is capable of action AND is not committed to REFRAINing OR if action is REFRAIN then the agent is not already committed.

©CS540 Spring 2013

Software Agents

Operate in virtual worlds,e.g., information systems, WWW, networks…

Examples: Search engines (Google) Comparison shopping agents (MySimon) Recommenders (Amazon) “Bob” the Microsoft helper

©CS540 Spring 2013

Software Agent Example: Personalized Shopping

Interface Agent

Recommender Agent

Profiling Agent

DB Agent

User request

Information display

Profile information

feedback User Request Product

info

Customer info Product & Customer info

Purchasing and product DBs

8

©CS540 Spring 2013

Agents as Personal Assistants (Maes)

n  Capabilities: n  Communicates with human in collaborative

manner n  Performs tasks n  Monitors events

n  Applications: n  Information gathering, filtering n  Meeting scheduling

©CS540 Spring 2013

Agent Types (Nwana)

Collaborative agents: emphasizes coordination

Interface agents: personal assistants Static or Mobile agents: software that

roams networks Information/Internet agents: manage,

manipulate and distribute information Deliberative or Reactive agents

©CS540 Spring 2013

Collaborative Agent: Pleiades (Sycara)

n  Distributed collaborative agent-based architecture with two layers: n  Task-specific agents n  Information specific agents

n  Application: visitor hosting system n  Task agents resolve scheduling conflicts. n  Information agents access databases of

inidvidual’s schedules, provide information to task agents.

n  All agents include planning module and local beliefs/facts database. One task agent per user.

©CS540 Spring 2013

Interface Agent n  Assists users with software n  Application should have

n  Substantial repetitive behavior in the application program

n  Repetitive behavior should differ for each user n  Example: Letizia, an agent that assists in web

browsing. n  Conducts a breadth-first search from web pages

while user is reading one n  Recommends new pages based on user’s

browsing preferences

9

Mobile Agents n  Roam wide area networks to

perform tasks on behalf of owners n  Example: Sony’s Magic Link PDA

which assists in managing a user’s email, fax, phone and pager

n  Key Technology: TeleScript is an OO programming language for developing distributed applications.

n  Sherpa & Google Now look at email, calendar and location to present useful information

©CS540 Spring 2013 ©CS540 Spring 2013

Information/Internet Agents

Web search Google, citeulike, mendeley

Opinions/reviews Monitoring twitter feeds Travel: farecast IBM’s Watson Watson game

©CS540 Spring 2013

Reactive Software Agents

n  Few examples of such systems: n  A-life ant societies and eco-systems n  Simulated robots

Robotic Agents Operate in physical environments n  Vacuuming (IRobot’s Roomba) n  Delivery in hospitals (Matsushita’s

HOSPI) n  Unmanned aerial vehicle (Predator) n  Mars rover n  Search and rescue

©CS540 Spring 2013

10

Robotics – Why So Hard?

©CS540 Spring 2013

Robotics Chess

Inaccessible Accessible

Nondeterministic Deterministic

Non episodic Episodic

Dynamic Static

Continuous Discrete

What Are Robots Good For?

n  Manufacturing and materials handling

n  Gofer robots (Fetch Robby, Fetch!)

n  Hazardous Environments

n  Telepresence and Teleoperation

©CS540 Spring 2013

What Are Robots Made of? n  Structure: rigid parts of metal or plastic n  Joints: pivot, ball joint, bearing & axle,… n  Actuation: motors, pistons, … n  Extending actuation: wheels, cables,

hydraulics… n  Power: batteries, fuel tanks, … n  Self sensing: encoders, accelerometers,

gyros, strain gauges, volt meters, thermometers, inclinometers…

n  External sensing: command links, sonar, microphones, photo-receptors, cameras…

©CS540 Spring 2013

Effectors – Locomotion

n  Walking vs. Rolling n  Wheels are always stable, but stairs are difficult! n  Four legs needed to guarantee stability, which is easy

to control but inefficient. n  Dynamic stability requires few legs, but complicated

control. n  Control

n  Holonomic: controllable degrees of freedom match robot’s DoF

n  Nonholonomic: controllable DoF less than robot’s DoF

©CS540 Spring 2013

11

Effectors – Manipulation

n  Commonly simplified arms/hands n  Kinematics: methodology for computing

inputs needed to reach physical states n  Inputs I: joint angles, slider positions… n  Outputs O: desired hand position…

©CS540 Spring 2013

IOM →:

Sensing §  Proprioception: use encoders to measure

joint angles, etc. §  Force & Tactile sensing

n  Critical in accomplishing complaint motions. n  Material identification, change detection.

§  Sonar n  Like finding your way in a house of mirrors with a

flashlight §  Cameras

§  CCD, infrared, ladar…

©CS540 Spring 2013

Paradigms

n  Hierarchical: strict cycle of sense, plan, act. Global world model.

n  Reactive: sense-act mapping. No planning or global world model.

n  Hybrid Deliberative/Reactive: Planning decomposes task into subtasks which can be reactive.

©CS540 Spring 2013

Architectures - Hierarchical

Horizontal task decomposition

©CS540 Spring 2013

Sensors Extract Features

Combine Features Into Model

Plan Tasks

Execute Tasks

Motor Control Actuators

12

SRI: from Shakey, the first AI mobile robot to now

©CS540 Spring 2013

Shakey from http://www.ai.sri.com/timeline/

SLAM from http://www.ai.sri.com/~konolige/

Shakey – Architecture n  3 level hierarchy:

n  Low level actions: turn, roll, update position… n  Intermediate: e.g., go here to there n  High: decide what to do/plan

n  Control loop: §  Accept goal from user §  Compare world state to goal §  Execute shortest plan subsequence satisfying

goal. If no plan available, call Strips.

©CS540 Spring 2013

Architectures – Reactive

Simple model of the world: what action to take in what state (FSM, rules, policy)

©CS540 Spring 2013

sensors actuators

Build maps

explore

wander

Avoid collisions

Characteristics of Reactive Architectures

Situated agency: goals arise from robot’s ecological niche

Emergent Behaviors: overall performance arises from integration of behaviors

No Representation: only behavior specific sensing, only representation as needed to process sensory data

©CS540 Spring 2013

13

Brooks’ Behavior Based Robotics

n  Behavior is a direct mapping from sensory input to a pattern of motor actions designed to achieve some task.

n  Organized hierarchically into layers of competence.

n  Robust, coherent and real-time behavior arises from hard coded ranking of competence layers.

©CS540 Spring 2013

Subsumption Architecture

Each layer n  includes a set of behaviors operating

asynchronously. n  Is a network of Augmented FSMs (with

timers) n  Derives input from other layers of sensors. n  May be suppressed (overriding input) or

inhibited (overriding output) by high levels.

©CS540 Spring 2013

Reactivity – Potential Fields

n  Uses vectors to describe behaviors; combination through vector summation.

©CS540 Spring 2013

uniform approach avoid

Hybrid Deliberative/Reactive

n  Planning (e.g., path planning, performance monitoring) operates at a higher level than the sense-act cycle of reactivity.

n  Planning turns on/off particular behaviors to produce a sequence.

©CS540 Spring 2013

14

Components of Hybrids Sequencer: generates sequence of behaviors,

including necessary preconditions Resource Manager: allocates resources to

behaviors Cartographer: creates and maintains world

model Mission planner: translates directives into a

robot plan Performance monitor: tracks progress

©CS540 Spring 2013

Example: Task Control Architecture

©CS540 Spring 2013

Task Scheduling (Prodigy)

Path Planning

Navigation (POMDP)

Obstacle Avoidance

Sensors Effectors

“Xavier” Reid Simmons

At CMU http://www.ri.cmu.edu/projects/project_91.html

©CS540 Spring 2013

Relating Behaviors to Robot Architectures

Behavior Architecture Find Coke can Follow a person Robots flocking Avoid obstacles Go “home” Go to X Pickup Y Assemble a clock

©CS540 Spring 2013

Multi-Agent Systems

n  Several interacting intelligent agents coordinate or cooperate in their efforts to achieve a common set of goals.

n  Distinguishing features: n  Need for inter-agent communication:

coordinate and negotiate n  Synthetic characters/personality n  Orchestrating cooperation

15

Multi-Agent System Design

n  Agents: support autonomous, independent action

n  Society: support interaction between agents, even when agents cannot be assumed to all share the same goals

to carry out the tasks we delegate to them

©CS540 Spring 2013 ©CS540 Spring 2013

Multi-Agent Systems: Robotic

Swarms: homogeneous (identical) robots working together on single task

Heterogeneous: soccer, search and rescue with different views

Control: distributed vs. centralized Cooperation: active vs. non-active

©CS540 Spring 2013

Multi-Agent Systems: Software n  Requirements

n  Inter-agent communication protocol n  Set of known services/capabilities n  Knowledge of resources n  Policy for carrying out tasks

n  Platform Services n  Registration of agents’ services n  Matchmaker or yellow pages n  Dictionary or ontology of language n  Security maintenance (trust and clearance)

©CS540 Spring 2013

Agent Communication Languages n  Requirements:

n  Language with standardized syntax and semantics n  Content language for domain of discourse n  Shared ontology

n  Problem Examples: n  What was the price of FIEUX yesterday at close? n  Find out how to install a print driver for HP Deskjet

1600CM. n  Solutions:

n  Knowledge Query and Manipulation Language (KQML) n  Foundation for Intelligent Agent’s (FIPA) ACL

16

©CS540 Spring 2013

KQML: Knowledge Query and Manipulation Language

n  Standardized syntactic wrappers n  Example: “ask”performative

(ask :sender Bankagent :receiver Stockserver :reply-with IBM-stock :ontology Wordnet :language Lisp :content (PRICE IBM ?price))

Example Dialogue: A

A to B: (ask-if (> (size chip1) (size chip2)) B to A: (reply true) B to A: (tell (= (size chip1) 20)) B to A: (tell (= (size chip2) 18))

©CS540 Spring 2013

Example dialogue (B) (stream-about

:sender A :receiver B :language KIF :ontology motors :reply-with q1 :content m1)

(tell :sender B :receiver A :in-reply-to q1 :content (= (torque m1) (scalar 12 kgf)))

©CS540 Spring 2013 ©CS540 Spring 2013

KQML Layers

Content layer: application’s representation language

Communication layer: bookkeeping information between agents

Message Layer: supplied message type, language and ontology information

17

©CS540 Spring 2013

Ontology

Central issue in KR: how to organize knowledge to impart meaning n  How to divide up domain n  How to determine important concepts n  What properties to include n  How to support inference

Task Coordination/Allocation

n  Contract Net protocol 1.  Recognition 2.  Announcement 3.  Bidding 4.  Awarding 5.  Expediting

©CS540 Spring 2013