a mechanism for learning, attention switching, and cognition
DESCRIPTION
A Mechanism for Learning, Attention Switching, and Cognition. Janusz Starzyk. School of Electrical Engineering and Computer Science, Ohio University, USA http://people.ohio.edu/starzykj. Cathedral of Applied Information Systems University of Information Technology and Management Poland. - PowerPoint PPT PresentationTRANSCRIPT
A Mechanism for Learning, Attention A Mechanism for Learning, Attention Switching, and CognitionSwitching, and Cognition
School of Electrical Engineering and Computer Science, Ohio University, USAhttp://people.ohio.edu/starzykj
Dagstuhl Seminar, March 27- April 1, 2011.
Cathedral of Applied Information SystemsUniversity of Information Technology and ManagementPoland
Janusz StarzykJanusz Starzyk
Motivated LearningMotivated Learning
Various pains, internal, and external signals compete for attention. Attention switching results from competition. Cognitive perception is aided by attention switching.
Definition: Motivated learning (ML) is pain based motivation, goal creation and learning in embodied agent. ML applies to EI working in a hostile environment. Machine creates abstract goals based on the pain
signals. It receives internal rewards for satisfying its goals
Reinforcement LearningReinforcement Learning Motivated Learning Motivated Learning External rewards Predictable Objectives set by designer Maximizes the reward
Potentially unstable
Learning effort increases with complexity
Always active
Internal rewards Unpredictable Sets its own objectives Solves minimax problem
Always stable
Learns better in complex environment than RL
Acts when needed
http://www.bradfordvts.co.uk/images/goal.jpg
Primitive Goal CreationPrimitive Goal Creation
- +
Pain
Dry soilPrimitive
level
opentank
sit on garbage
refillfaucet
w. can water
Dual pain
Reinforcing a proper action
Abstract Goal HierarchyAbstract Goal Hierarchy
Abstract goals are created to reduce abstract pains and to satisfy the primitive goals A hierarchy of abstract goals is created to satisfy the lower level goals
ActivationStimulationInhibitionReinforcementDifferenceNeedExpectation
- +
+
Dry soilPrimitive Level
Level I
Level IIfaucet
-
w. can
open
water
+
Sensory pathway(perception, sense)
Motor pathway(action, reaction)
Level IIItank
-
refill
Drought
Reservoir
Irrigate
Thirsty
Water
Drink Water
Primitive Needs Dirty
Wash in Water
Abstract Needs
Primitive needsPrimitive needs
0 50 100 150 200 250 3000
0.02
0.04
0.06
0.08
0.1
0.12Competing need signals
Iterative step
Nee
d si
gnal
leve
l
DirtyThirstyDroughtThreshold
Drought
Reservoir
Public Money
Irrigate
Spend Money to Build
Thirsty
Water
Drink Water
Spend Money to Buy
Primitive Needs
Well
Draw own Water
Dirty
Wash in Water
Abstract Needs
Abstract needsAbstract needs
Drought
Reservoir
Public Money
Tourists' Attractions
Irrigate
Spend Money to Build
Build Ecotourism
Thirsty
Water
Drink Water
Spend Money to Buy
Primitive Needs
Build Water Recreation
Wealthy Taxpayers
Rise Taxes
Well
Draw own Water
Dirty
Wash in Water
Well Building
Dig a Well
Abstract Needs
Ground Water
Water Supply
Abstract needsAbstract needs
Drought
Reservoir
Public Money
Tourists' Attractions
Irrigate
Spend Money to Build
Build Ecotourism
Thirsty
Water
Drink Water
Spend Money to Buy
Primitive Needs
Build Water Recreation
Policy
Develop Infrastructure
Wealthy Taxpayers
Rise Taxes
Well
Draw own Water
Dirty
Wash in Water
Well Building
Dig a Well
Abstract Needs
Employment Opportunities
Ground Water
Water Supply
Receive SalaryResource Management
and Planning
Management
Regulate Use
Planning
Abstract needsAbstract needs
6 levels of hierarchy Initially ML agent experiences similar
primitive pain signal Pp as RL agent. ML agent converges quickly to a stable
performance.
10 levels of hierarchy Initially RL agent experiences lower
primitive pain signal Pp than ML agent. RL agent’s pain increases when
environment is more hostile.
ML vs. RL ML vs. RL aagentgentss inin hierarchical hierarchical environmentenvironmentss
J.A. Starzyk, P. Raif, and A.-H. Tan, “Mental Development and Representation Building through Motivated Learning” , WCCI 2010 - Special Session on Mental Architecture and Representation, Barcelona, Spain, July 18-23, 2010.
Grid world problemGrid world problem
Four kinds of resources distributed over 25 x 25 grid.P. Raif, J.A. Starzyk, Motivated Learning In Autonomous Systems, submitted to IJCNN2011 - Special Session on Autonomous learning of object representation and control, San Jose, CA, July 31-Aug. 5, 2011.
Intelligence
Central executive
Attention and attention switching
Mental saccades
Cognitive perception
Cognitive action control
ConsciousnessConsciousness
Photo: http://eduspaces.net/csessums/weblog/11712.html
Computational Model of Conscious MachineComputational Model of Conscious Machine
Semantic memory
Sensory processors
Data encoders/ decoders
Sensory units
Motor skills
Motor processors
Data encoders/ decoders
Motor units
Emotions, rewards, and sub-cortical processing
Attention switching
Action monitoring
Motivation and goal processor
Planning and thinking
Episodic memory
Queuing and organization of episodes
Episodic Memory & Learning
Central Executive
Sensory-motor
Inspiration: human brainInspiration: human brainPhoto (brain): http://www.scholarpedia.org/article/Neuronal_correlates_of_consciousness
Attention switching
Action monitoring
Motivation and goal processor
Planning and thinking
Central Executive
Taskso cognitive perceptiono attentiono attention switchingo motivationo goal creation and selectiono thoughtso planningo learning, etc.
Central ExecutiveCentral Executive
Interacts with other units for o performing its tasks o gathering data o giving directions to other units
No clearly identified decision centerDecisions are influenced by
o competing signals representing motivations, pains, desires, plans, and interrupt signals
• need not be cognitive or consciously realizedo competition can be interrupted by attention switching signal
Attention switching
Action monitoring
Motivation and goal processor
Planning and thinking
Central Executive
Central ExecutiveCentral Executive
Attention Switching !Attention Switching !Dynamic process resulting from competition between
• representations related to motivations
• sensory inputs
• internal thoughts including
spurious signals (like noise).
blog.gigoo.org/.../
Input image
AB
C D
AB
C DA
B
C D
What Where
Visual SaccadesVisual Saccades
Mental SaccadesMental Saccades
This in turn activates memory traces in the global workspace area
that will be used for mental searches (mental saccades).
saccade
John
Input image
Episodic and associative memory network
his wife his house
his dogfriends
business
Spotlight on John
Frontal cortex
Mental saccade
wife house
dogfriends
business
Memory traces in frontal cortex
saccade
John
Input image
Episodic and associative memory network
his wife his house
his dogfriends
business
Spotlight on John
Frontal cortex
Mental saccade
wife house
dogfriends
business
Memory traces in frontal cortex
Selected part of the image resulting from an eye saccade.
Perceived input activates object recognition and associated areas of semantic and episodic memory.
Mental saccades in a conscious machineMental saccades in a conscious machine
Perceptual saccadesChanging perception
Changing environment
Associative memory
No
No
Action controlLoop 5
Loop 2
Perceptual saccadesChanging perception
Changing environment
Associative memory
No
No
Action control
Advancement of a goal?
Yes
Learning
Advancement of a goal?
Advancement of a goal?
Yes
Learning
Attention spotlight
Mental saccades
Continue search?
Yes
Loop 1
Attention spotlight
Mental saccades
Continue search?Continue search?
Yes
Loop 1
Plan action?
NoYes
Action?
Yes
No
Changing motivation
Loop 3
Loop 4
Plan action?Plan action?
NoYes
Action?Action?
Yes
No
Changing motivation
Loop 3
Loop 4
Loop 5
Loop 2
Action and subgoal planningAction and subgoal planning
Intended action
Induced pain
Dual pain
Perception
Pain reduction Next mental saccade
Perform action
Learning
Pain
Environment
Decide action
Attention spotlight
Desired item
Memory
Action controlAction control
Predicted changes known
Pain increase
Predicted changes
Intended action
Associative memory
Cognitive action control
Lower level action control
Lower level action control
Lower level action control
Action?
Cognitive abort
A Mechanism for Learning, Attention A Mechanism for Learning, Attention Switching, and Cognition: SummarySwitching, and Cognition: Summary
A mechanism of switching attention is fundamental for building cognitive machines.Attention switching is a dynamic process resulting from competition between goals, representations, sensory inputs, and internal thoughts.Motivated learning provides a mechanism for creation of abstract goals and continuous goal oriented motivationMental saccades of the working memory are fundamental for cognitive thinking, attention switching, planning, and action monitoring
http://www.inspirationfalls.com/the-key-to-success-quotes/key-to-all-success-concepts-1/
Motivations for actions are physically distributedo competing pain (need) signals are generated in various parts
of machine’s mind Before a winner is selected, machine does not interpret the meaning of the competing signals Cognitive processing is predominantly sequential
o winner of the internal competition is an instantaneous director of the cognitive thought process
Top down supervision of perception, planning, internal thought or motor functions
o results in conscious experience• decision of what is observed and where is it • planning how to respond
o a train of such experiences constitutes consciousness
A Mechanism for Learning, Attention A Mechanism for Learning, Attention Switching, and Cognition: SummarySwitching, and Cognition: Summary
ConclusionsConclusions1.Consciousness is computational2.Motivated intelligent machines can be conscious
Questions ??Questions ??
Photo: http://bajan.wordpress.com/2010/03/03/dont-blame-life-blame-the-way-how-you-live-it/
ReferencesReferences P.A.O. Haikonen, “The cognitive approach to conscious machines”. UK: Imprint Academic, 2003. J. Bach, “Principles of Synthetic Intelligence PSI: An Architecture of Motivated
Cognition”, Oxford Univ. Press, 2009. B. J. Baars “A cognitive theory of consciousness,” Cambridge Univ. Press, 1998. A. Sloman, "Developing concept of consciousness," Behavioral and Brain
Sciences, vol. 14 (4), pp. 694-695, Dec 1991. J. Schmidhuber, “Curious model-building control systems,” Proceedings Int. Joint
Conf. Neural Networks, Singapore, vol. 2, pp. 1458–1463, 1991. B. Bakker and J. Schmidhuber, “Hierarchical Reinforcement Learning with
Subpolicies Specializing for Learned Subgoals,” in Proc. of the 2nd Int. Conf. on Neural Networks & Computational Intelligence, Switzerland, pp.125-130, 2004.
A. Barto, S. Singh, and N. Chentanez, “Intrinsically motivated learning of hierarchical collections of skills, Proc. 3rd Int. Conf. Development Learn., San Diego, CA, pp. 112–119, 2004.
J. A. Starzyk, "Motivation in Embodied Intelligence" in Frontiers in Robotics, Automation and Control, Oct. 2008, pp. 83-110.
J.A. Starzyk, “Motivated Learning for Computational Intelligence,” in Computational Modeling and Simulation of Intellect. ed. B. Igelnik, IGI Publ, 2011.
Photo: http://s121.photobucket.com/albums/o209/TiTekty/?action=view¤t=hist_sci_image1.jpg
Embodied IntelligenceEmbodied Intelligence
– Mechanism: biological, mechanical or virtual agent
with embodied sensors and actuators– EI acts on environment and perceives its actions– Environment hostility is persistent and stimulates EI to act– Hostility: direct aggression, pain, scarce resources, etc– EI learns so it must have associative self-organizing memory– Knowledge is acquired by EI
Definition Embodied Intelligence (EI) is a
mechanism that learns how to minimize hostility of its environment
Embodiment
Actuators
Sensors
Intelligence core
channel
channel
Embodiment
Sensors
Intelligence core
Environment
channel
channelActuators
Embodiment
Actuators
Sensors
Intelligence core
channel
channel
Embodiment
Sensors
Intelligence core
Environment
channel
channelActuators
Embodiment of a MindEmbodiment of a Mind
Embodiment is a part of the environment that EI controls to interact with the rest of the environment
It contains intelligence core and sensory motor interfaces under its control
Necessary for development of intelligence
Not necessarily constant or in the form of a physical body
Boundary transforms modifying brain’s self-determination