chapter 1. cognitive systems introduction in cognitive systems, christensen et al. course: robots...
TRANSCRIPT
![Page 1: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/1.jpg)
Chapter 1. Cognitive Systems Introductionin Cognitive Systems, Christensen et al.
Course: Robots Learning from Humans
Park, Sae-Rom
Lee, Woo-Jin
Statistical Learning & Computational Finance Laboratory
Industrial Engineering
Seoul National University
http://slcf.snu.ac.kr
![Page 2: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/2.jpg)
2
Contents Introduction
Objective of Project
Motivating example Organization of the Research/Research Questions
Architecture Representations Learning Perception-Action Modelling Continuous Planning and Acting Models of Action and Communication Multi-Modal Recognition and Categrization Scenario Driven Research
Organization of the Book
![Page 3: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/3.jpg)
CoSy project
The assumption of the visionary FP6 “To construct physically instantiated … systems that can
perceive, understand … and interact with their environ-ment, and evolve in order to achieve human-like perfor-mance in activities requiring context- (situation and task) specific knowledge”
Requirements Architectures, forms of representation, perceptual mech-
anisms, learning, planning, reasoning, motivation, action, and communication
To validate science progress using test scenarios
![Page 4: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/4.jpg)
4
Contents Introduction
Objective of Project
Motivating example Organization of the Research/Research Questions
Architecture Representations Learning Perception-Action Modelling Continuous Planning and Acting Models of Action and Communication Multi-Modal Recognition and Categrization Scenario Driven Research
Organization of the Book
![Page 5: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/5.jpg)
Objective of Project
Problem Most systems able to perform complex tasks that hu-
mans and other animals can perform easily, for instance robot manipulators, or intelligent advisers, have to be carefully crafted
The way to forward Combining many different capabilities in a coherent
manner -> 4-5 year child Generic capabilities
![Page 6: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/6.jpg)
Steps to Success
Achievable sub-goals Theory deliverables Implementation deliverables
Theory deliverables The notion of an architecture combining components
Reactive Deliberative Self-reflective, meta management
Different learning processes Different varieties of communication and social interac-
tion
![Page 7: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/7.jpg)
Steps to Success
Implementation Deliverables
nature
nurture
vs
Linguistic
Visual
ReasoningPlanning
Motor skills
Integration
![Page 8: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/8.jpg)
8
Contents Introduction
Objective of Project
Motivating example Organization of the Research/Research Questions
Architecture Representations Learning Perception-Action Modelling Continuous Planning and Acting Models of Action and Communication Multi-Modal Recognition and Categrization Scenario Driven Research
Organization of the Book
![Page 10: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/10.jpg)
10
Contents Introduction
Objective of Project
Motivating example Organization of the Research/Research Questions
Architecture Representations Learning Perception-Action Modelling Continuous Planning and Acting Models of Action and Communication Multi-Modal Recognition and Categrization Scenario Driven Research
Organization of the Book
![Page 11: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/11.jpg)
Organization of the Research Research challenges
Two scenarios for study of integrated systems Two major milestones
Using intermodality and affordances for the acquisition of concepts, categories and language
Introspection of models & representations; planning for autonomy – goal seeking
architectures
Represen-tations
learningPerception-
action model-ing
Communi-cation
Planning & failure han-
dling
![Page 12: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/12.jpg)
Architecture
Putting pieces together into a complex functional system Perception, action, reasoning and communicating
![Page 13: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/13.jpg)
Representation
The representation should Enable integration of representations of objects, scenes,
actions, events, causal relations and affordances Allow incremental updating or sometimes correction Allow different types of learning (supervised, unsuper-
vised, reinforcement) Allow integration of various modalities, of very different
input signals Be suitable for recognition and categorization in the
presence of a cluttered background and variable illumi-nation
Be scalable
![Page 14: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/14.jpg)
Representation
![Page 16: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/16.jpg)
Learning
Modes of learning
Tutor Driven A user (tutor) shows to the system an object or an ac-
tion and explains to the cognitive system what he/she is showing or doing
Tutor Supervised A cognitive system detects a new object, an action,
event, affordance or a scene by itself and builds its representation in an unsupervised manner.
Exploratory Updates the representation autonomously
![Page 18: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/18.jpg)
Continuous Learning
Representations employed allow the learning to be a continuous, open-ended, life-long process Continuously updated over time, adapting to the change
in environment, new tasks, user reactions, user prefer-ences, …
Reliable continuous learning Representations have to be carefully chosen How new data is extracted and prepared
![Page 19: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/19.jpg)
Perception-Action Modelling
Abstract relation model General, non-task specific Observability of the world hand-constructed abstraction
Probability relational representation Capture uncertainty in both action and observation Tractable for localization and path planning in continuous
space Sensor-dependence
Reinforcement learning Identifying features that are relevant to predicting the outcome
on the task
![Page 20: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/20.jpg)
20
Continuous Planning
Difficulties Dynamic nature, partial observability
Conditional planning Probabilistic planning
![Page 21: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/21.jpg)
21
Continuous Planning
Active Failure Diagnosis In most approaches it is typically assumed that the sensors and ac-
tuators of the robot are reliable in the sense that their input always corresponds to the expected input and that there is no malfunction of the sensors or actuators
These approaches do not exploit the actuators of the ro-bot to identify potential faults
Once a fault has been identified, the high-level system is notified so that appropriate actions can be generated at the planning level.
![Page 22: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/22.jpg)
Continuous Planning
Collaborative Planning and Acting
Cooperation is at the heart of the Cosy project
Common language protocol Dialogue
![Page 24: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/24.jpg)
Models of Action and Communication for Em-bodied Cognitive Agents
Natural Language
Integration of communication and action Recognition of intention, attention, and grounding/under-
standing Mixed-initiative Embodiment in an unknown environment
![Page 25: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/25.jpg)
Models of Action and Communication for Em-bodied Cognitive Agents
![Page 26: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/26.jpg)
Multi-Modal Recognition and Categorization
Recognize Categorize Entry level categorization vs Recognition
Recognition of objects Categorization Multi-cue
![Page 27: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/27.jpg)
Scenario Driven Research
System level
Exploration/Mapping of Space Models of objects and concepts
![Page 28: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/28.jpg)
Exploration / Mapping of space
Where am I? How do I get to my destination? How do I detect that I have arrived at the destination?
Perception and action Localization in the World Construction of a map of the environment Plan a sequence of actions
![Page 29: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/29.jpg)
Affordances and Newer Approaches Space Object
Robustness
Wall, Door, Table
![Page 30: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/30.jpg)
The World as an Outside Memory
![Page 31: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/31.jpg)
Mapping of the Environment
Encoding of position of objects/places Encoding of environmental topology Invariant to changes to perception system Invariant to changes in action system Facilitate spatial reasoning
![Page 32: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/32.jpg)
Models for Object and Concepts
Representation Continuous Learning Robustness Categorization Architecture Communication and Language
![Page 33: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/33.jpg)
33
Contents Introduction
Objective of Project
Motivating example Organization of the Research/Research Questions
Architecture Representations Learning Perception-Action Modelling Continuous Planning and Acting Models of Action and Communication Multi-Modal Recognition and Categrization Scenario Driven Research
Organization of the Book
![Page 34: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/34.jpg)
Organization
Chapter 2 Architecture design, representation Chapter 3 perception - action Chapter 4 spatial maps Chapter 5 visual perception Chapter 6 planning recovery Chapter 7 adaptation & learning Chapter 8 Human-robot interaction Chapter 9 & 10 Demonstration
![Page 35: Chapter 1. Cognitive Systems Introduction in Cognitive Systems, Christensen et al. Course: Robots Learning from Humans Park, Sae-Rom Lee, Woo-Jin Statistical](https://reader036.vdocuments.site/reader036/viewer/2022062500/5697bfa61a28abf838c98125/html5/thumbnails/35.jpg)
Thank you for Listening