Download - Ontologies Reasoning Components Agents Simulations Architectural Patterns for Agents Jacques Robin
OntologiesReasoningComponentsAgentsSimulations
Architectural Patterns for AgentsArchitectural Patterns for Agents
Jacques Robin
Agents’ Internal ArchitecturesAgents’ Internal Architectures
Reflex agent (purely reactive) Automata agent (reactive with state) Goal-based agent Planning agent Hybrid, reflex-planning agent Utility-based agent (decision-theoretic) Layered agent Adaptive agent (learning agent)
Cognitive agent Deliberative agent
Reflex AgentReflex Agent
En
viro
nm
en
t
Sensors
Effectors
RulesPercepts Action
A(t) = h(P(t))
Remember … Remember …
Reasoning
AgentAgent
En
viro
nm
en
t
Sensors
Effectors
Goals
PerceptInterpretation:
I = f(P)
Action Choice:A = g(I,O)
A
P
So?So?
Goals
Percept Interpretation: I = f(P)
Action Choice:A = g(I,O)
En
viro
nm
en
t
Sensors
Effectors
RulesPercepts Action
A(t) = h(P(t))
A
P
Reflex AgentReflex Agent
Principle: Use rules (or functions, procedures) that associate directly percepts
to actions ex. IF speed > 60 THEN fine ex. IF front car’s stop light switches on THEN brake
Execute first rule which left hand side matches the current percepts Wumpus World example
IF visualPerception = glitter THEN action = pick see(glitter) do(pick) (logical representation)
Pros: Condition-action rules is a clear, modular, efficient representation
Cons: Lack of memory prevents use in partially observable, sequential, or
non-episodic environments ex, in the Wumpus World a reflex agent can’t remember which path
it has followed, when to go out of the cavern, where exactly are located the dangerous caverns, etc.
Automata AgentAutomata Agent
Enviro
nm
ent
Sensors
Effectors
(Past and) CurrentEnviroment Model
Percept InterpretationRules:percepts(t) model(t) model’(t)
Action ChoiceRules:model’’(t) action(t), action(t) model’’(t) model(t+1)
Model UpdateRegras: model(t-1) model(t) model’(t) model’’(t)
Goals
Automata AgentAutomata Agent
Rules associate actions to percept indirectly through the incremental construction of an environment model (internal state of the agent)
Action choice based on: current percepts + previous percepts + previous actions +
encapsulated knowledge of initial environment state Overcome reflex agent limitations with partially observable,
sequential and non-episodic environments Can integrate past and present perception to build rich
representation from partial observations Can distinguish between distinct environment states that are
indistinguishable by instantaneous sensor signals Limitations:
No explicit representation of the agents’ preferred environment states
For agents that must change goals many times to perform well, automata architecture is not scalable (combinatorial explosion of rules)
Automata Agent Rule ExamplesAutomata Agent Rule Examples
Rules percept(t) model(t) model’(t) IF visualPercept at time T is glitter
AND location of agent at time T is (X,Y)THEN location of gold at time T is (X,Y)
X,Y,T see(glitter,T) loc(agent,X,Y,T) loc(gold,X,Y,T).
Rules model’(t) model’’(t) IF agent is with gold at time T
AND location of agent at time T is (X,Y)THEN location of gold at time T is (X,Y)
X,Y,T withGold(T) loc(agent,X,Y,T) loc(gold,X,Y,T).
Automata Agent Rule ExamplesAutomata Agent Rule Examples
Rules model(t) action(t) IF location of agent at time T = (X,Y)
AND location of gold at time T = (X,Y) THEN choose action pick at time T
X,Y,T loc(agent,X,Y,T) loc(gold,X,Y,T) do(pick,T)
Rules action(t) model(t) model(t+1) IF choosen action at time T was pick
THEN agent is with gold at time T+1 T done(pick,T) withGold(T+1).
(Explicit) Goal-Based Agent(Explicit) Goal-Based Agent
Enviro
nm
ent
Sensors
Effectors
(Past and) CurrentEnvironment Model
Percept InterpretationRules: percept(t) model(t) model’(t)
Action ChoiceRules: model’’(t) goals’(t) action(t) action(t) model’’(t) model(t+1)
Model UpdateRules: model(t-1) model(t) model’(t) model’’(t)
Goal UpdateRules: model’’(t) goals(t-1) goals’(t) Goals
(Explicit) Goal-Based Agent(Explicit) Goal-Based Agent
Principle: explicit and dynamically alterable goals Pros:
More flexible and autonomous than automata agent Adapt its strategy to situation patterns summarized in its goals
Limitations: When current goal unreachable as the effect of a single action,
unable to plan sequence of actions Does not make long term plans Does not handle multiple, potentially conflicting active goals
Goal-Based Agent Rule ExamplesGoal-Based Agent Rule Examples
Rule model(t) goal(t) action(t) IF goal of agent at time T is to return to (1,1) AND agent is in (X,Y) at time T AND orientation of agent is 90o at time T AND (X,Y+1) is safe at time T AND (X,Y+1) has not being visited until time T AND (X-1,Y) is safe at time T AND (X-1,Y) was visited before time T THEN choose action turn left at time T X,Y,T, (N,M,K goal(T,loc(agent,1,1,T+N)) loc(agent,X,Y,T) orientation(agent,90,T) safe(loc(X,Y+1),T) loc(agent,X,Y+1,T-M) safe(loc(X-1,Y),T) loc(agent,X,Y+1,T-K)) do(turn(left),T)
Y+1
ok
Yv ok
A
X-1 X
Goal-Based Agent Rule ExamplesGoal-Based Agent Rule Examples
Rule model(t) goal(t) action(t) IF goal of agent at time T is to find gold AND agent is in (X,Y) at time T AND orientation of agent is 90o at time T AND (X,Y+1) is safe at time T AND (X,Y+1) has not being visited until time T AND (X-1,Y) is safe at time T AND (X-1,Y) was visited before time T THEN choose action forward at time T X,Y,T, (N,M,K goal(T,withGold(T+N)) loc(agent,X,Y,T) orientation(agent,90,T) safe(loc(X,Y+1),T) loc(agent,X,Y+1,T-M) safe(loc(X-1,Y),T) loc(agent,X,Y+1,T-K)) do(forward,T)
Y+1
ok
Yv ok
A
X-1 X
Goal-Based Agent Rule ExamplesGoal-Based Agent Rule Examples
Rule model(t) Rule model(t) goal(t) goal(t) goal’(t) goal’(t)//If the agent reached it goal to hold the gold, //then its new goal shall be to go back to (1,1) IF goal of agent at time T-1 was to find gold AND agent is with gold at time T THEN goal of agent at time T+1 is to be in location (1,1) T, (N goal(agent,T-1,withGold(T+N)) withGold(T) M goal(agent,T,loc(agent,1,1,T+M))).
Planning AgentPlanning Agent
Enviro
nm
ent
Sensors
Effectors
(Past and)Current
EnvironmentModel
Percept InterpretationRules: percept(t) model(t) model’(t)
Action ChoiceRules: model(t+n) = result([action1(t),...,actionN(t+n)] model(t+n) goal(t) do(action1(t))
Model UpdateRules: model(t-1) model(t) model’(t) model’’(t)
Goal UpdateRules: model’’(t) goals(t-1) goals’(t)
GoalsPrediction of Future EnvironmentsRules: model’’(t) model(t+n) model’’(t) action(t) model(t+1)
HypotheticalFuture
EnvironmentModels
Planning AgentPlanning Agent
Percept and actions associated very indirectly through: Past and current environment model Past and current explicit goals Prediction of future environments resulting from different possible
action sequences to execute Rule chaining needed to build action sequence from rules
capture immediate consequences of a single action Pros:
Foresight allows choosing more relevant and safer actions in sequential environments
Cons: little point in building elaborated long term plans in, Highly non-deterministic environment (too many possibilities to
consider) Largely non-observable environments (not enough knowledge
available before acting) Asynchronous concurrent environment (only cheap reasoning can
reach a conclusion under time pressure)
Planning ThreadPlanning Thread
Goals
Current,past and
futureenvironment
model
Current Model Update
Percept Interpretation
Goal Update
Future Environments Prediction
Action Choice
Hybrid Reflex-Planning AgentHybrid Reflex-Planning Agent
Enviro
nm
ent
Sensors
Effectors
Reflex ThreadReflex ThreadReflex RulesPercepts Actions
Synchronization
Hybrid Reflex-Planning AgentHybrid Reflex-Planning Agent
Pros: Take advantage of all the time and knowledge available to choose
best possible action (within the limits of its prior knowledge and percepts)
Sophisticated yet robust Cons:
Costly to develop Same knowledge encoded in different forms in each component Global behavior coherence harder to guarantee Analysis and debugging hard due to synchronization issues Not that many environments feature large variations in available
reasoning time in different perception-reasoning-action cycles
Layered AgentsLayered Agents
Many sensors/effectors are too fine-grained to reason about goals using directly the data/commands they provide
Such cases require a layered agent that decomposes its reasoning in multiple abstraction layers
Each layer represent the percepts, environment model, goals, and actions at a different level of details
Abstraction can consist in: Discretizing, approximating, clustering, classifying data from prior
layers along temporal, spatial, functional, social dimensions Detail can consist in:
Decomposing higher-level actions into lower-level ones along temporal, spatial, functional, social dimensions
Abstract
Decide Abstractly
Detail
Act in DetailPerceive in Detail
Percept InterpretationPercept Interpretation
Am
bie
nte
Sensors
Effectors
Environment ModelEnvironment ModelEnvironment Model UpdateEnvironment Model Update
Action Choice and Execution ControlAction Choice and Execution Control
Layered Automata Agent
Layer0: f(x).dxy
Layer1: y).P(y)|P(zP(s)
Layer0: f(x).dxy
Layer1: y).P(y)|P(zP(s)
Layer2: q(A)r(B)B)s(A,
Layer2: q(A)r(B)B)s(A,
q(A)r(B)B)s(A, q(A)r(B)B)s(A, Layer2: Layer2:
Exemplo de camadas de Exemplo de camadas de abstração: abstração:
XX
YY
Abstraction Layer ExamplesAbstraction Layer Examples
XX
YY
Utility-Based AgentUtility-Based Agent
Principle: Goals only express boolean agent preferences among environment states A utility function u allows expressing finer grained agent preferences
u can be defined on a variety of domains and ranges: actions, i.e., u: action R (or [0,1]), action sequences, i.e., u: [action1, ..., actionN] R (or [0,1]), environment states, i.e., u: environmentStateModel R (or [0,1]), environment state sequences, i.e., u: [state1, ..., stateN] R (or [0,1]), environment state, action pairs,
i.e., u: environmentStateModel x action R (or [0,1]), environment state, action pair sequences,
i.e., u: [(action1-state1), ..., (actionN-stateN)] R (or [0,1]), Pros:
Allows solving optimization problems aiming to find the best solution Allows trading-off among multiple conflicting goals with distinct
probabilities of being reached Cons:
Currently available methods to compute (even approximately) argmax(u) do not scale up to large or diverse environments
Utility-Based Reflex AgentUtility-Based Reflex Agent
Enviro
nm
ent
Sensors
Effectors
Percept Interpretation:Rules: percept actions
Goals
Action Choice:Utility Functionu:actions R U(a))argmaxdo(
actionsa
Utility-Based Planning Agent
Enviro
nm
ent
Sensors
Effectors
Past &Current
EnvironmentModel
Percept InterpretationRegras: percept(t) model(t) modelo’(t)
Model UpdateRegras: model’(t) model’’(t)
Future Environment PredictionRegras: model’’(t) ação(t) model(t+1) model’’(t) model(t+1)
HypothesizedFuture
EnvironmentsModel
Action Choice])))action actionU(result([argmaxdo( i
ni
i1
actioni1
...
Utility Function:u: model(t+n) R
Learning Component
Performance Analysis Component
Adaptive AgentAdaptive Agent
Enviro
nm
ent
Sensors
Effectors
ActingComponent
New Problem Generation Component
• Reflex• Automata• Goal-Based• Planning• Utility-Based• Hybrid
• Learn rules or functions: • percept(t) action(t)• percept(t) model(t) modelo’(t)• modelo(t) modelo’(t)• modelo(t-1) modelo(t)• modelo(t) action(t)• action(t) model(t+1)• model(t) goal(t) action(t)• goal(t) model(t) goal’(t)• utility(action) = value• utility(model) = value
Simulated EnvironmentsSimulated Environments
Environment simulator: Often themselves internally follow an agent architecture Should be able to simulate a large class of environments that can
be specialized by setting many configurable parameters either manually or randomly within a manually selected range ex, configure a generic Wumpus World simulator to generate world
instances with a square shaped cavern, a static wumpus and a single gold nugget where the cavern size, pit numbers and locations, wumpus and gold locations are randomly picked
Environment simulator processing cycle:1. Compute percept of each agent in current environment2. Send these percepts to the corresponding agents3. Receives the action chosen by each agent4. Update the environment to reflect the cumulative consequences
of all these actions
EnvironmentEnvironmentSimulationSimulation
ServerServer
RedeRede
Environment Simulator ArchitectureEnvironment Simulator Architecture
SimulatedEnvironmentModel
SimulationVisualization
GUIEnvironment UpdateRules: model(t-1) model(t)action(t) model(t-1) model(t)
Percept GenerationRules: model(t) percept(t)
percepts
actionsAgent
Client 1
AgentClient N
...
KBA ArchitecturesKBA Architectures
A KBA is a: Reflex Agent? Automata Agent? Goal-Based Agent? Planning Agent? Hybrid Agent? Utility-Based Agent? Adaptive Agent? Layered Agent?
Can be anyone !
Is there any constraint between the reasoning performed by the inference engine and the agent architecture?
Adaptive agent requires analogical or inductive inference engine
AskTell
Non-Adaptive KBANon-Adaptive KBA
Enviro
nm
ent
Sensors
Effectors
Persistent Knowledge Base (PKB):rules, classes, logical formulas or probabilities
representing generic laws about environment class
Inference Engine for Deduction, Abduction, Inheritance, Belief Revision, Belief Update,
Planning, Constraint Solving or Optimization
Volatile Knowledge Base (VKB):facts, objects, constraints, logical formulas or probabilities
representing environment instance in current agent execution
Retract
Ask
Non-Monotonic
Engine
AskTell
Analogical KBAAnalogical KBA
Enviro
nm
ent
Sensors
Effectors
Persistent Knowledge Base (PKB):facts, objects, constraints, logical formulas or probabilities
representing environment instances in past agent executionsstructured by similarity measure
Inference Engine for Analogy
Volatile Knowledge Base (VKB):facts, objects, constraints, logical formulas or probabilities
representing environment instance in current agent execution
Retract
Ask
Remember the Planning Agent?Remember the Planning Agent?
Enviro
nm
ent
Sensors
Effectors
(Past and)Current
EnvironmentModel
Percept InterpretationRules: percept(t) model(t) model’(t)
Action ChoiceRules: model(t+n) = result([action1(t),...,actionN(t+n)] model(t+n) goal(t) do(action(t))
Model UpdateRules: model(t-1) model(t) model’(t) model’’(t)
Goal UpdateRules: model’’(t) goals(t-1) goals’(t)
GoalsPrediction of Future EnvironmentsRules: model’’(t) model(t+n) model’’(t) action(t) model(t+1)
HypotheticalFuture
EnvironmentModels
How would be then aHow would be then aknowledge-based planning agentknowledge-based planning agent??
En
viro
nm
en
t
Sensors
Effectors
VKB: Past and Current
Environment Models
VKB: Hypothetical FutureEnvironment Models
VKB: Goals
PKB: PerceptInterpretation
PKB: EnvironmentModel Update
PKB: GoalsUpdate
PKB: Prediction ofFuture Environments
PKB: ActingStrategy
InferenceEngine
Alternative Planning KBA Alternative Planning KBA ArchitectureArchitecture
En
viro
nm
en
t
Sensors
Effectors
VKB: Past and Current
Environment Models
VKB: Hypothetical FutureEnvironment Models
VKB: Goals
PKB: PerceptInterpretation
PKB: EnvironmentModel Update
PKB: GoalsUpdate
PKB: Prediction ofFuture Environments
PKB: ActingStrategy
InferenceEngine 1
InferenceEngine 2
InferenceEngine 3
InferenceEngine 4
InferenceEngine 5
Why Using Multiple Inference Why Using Multiple Inference Engines?Engines?
En
viro
nm
en
t
Sensors
Effectors
VKB: Past and Current
Environment Models
VKB: Hypothetical FutureEnvironment Models
VKB: Goals
PKB: PerceptInterpretation
PKB: EnvironmentModel Update
PKB: GoalsUpdate
PKB: Prediction ofFuture Environments
PKB: ActingStrategy
InferenceEngine 1
InferenceEngine 2
InferenceEngine 3
InferenceEngine 4
InferenceEngine 5
Abduction
BeliefUpdate
Deduction
ConstraintSolving
Optimization
Off-Line Inductive Agent:Off-Line Inductive Agent:Training PhaseTraining Phase
Data, Examples or Case Base facts, objects, constraints or
logical formulas codifyingrepresentative sample of
environment entities
PerformanceInference Engine:
Any Reasoning TaskExcept Analogyand Induction
Intentional Knowledge Base (IKB):rules, classes or logical formulas
representing generic lawsabout environment class
InductiveInference Engine
Ask
Tell
Retract
HypothesisFormation
HypothesisVerification
Ask
Tell
Retract
Ask
Ask
Off-Line Inductive Agent: Usage Off-Line Inductive Agent: Usage PhasePhase
AskTell
Enviro
nm
ent
Sensors
Effectors
Inductively LearnedPersistent Knowledge Base (PKB):
rules, classes, logical formulas or probabilitiesrepresenting generic laws about environment class
Inference Engine for Deduction, Abduction, Inheritance, Belief Revision, Belief Update,
Planning, Constraint Solving or Optimization
Volatile Knowledge Base (VKB):facts, objects, constraints, logical formulas or probabilities
representing environment instance in current agent execution
Retract
Ask