neural architectures for lifelong learning on humanoid robots · neural architectures for lifelong...
TRANSCRIPT
Neural Architectures for Lifelong Learning on Humanoid Robots
Vadym Gryshchuk
19.11.2018
Outline
• Motivation
• Background
• Approaches
• Results
• Discussion
• Conclusion
2Neural Architectures for Lifelong Learning on Humanoid Robots
• Motivation
• Background
• Approaches
• Results
• Discussion
• Conclusion
3Neural Architectures for Lifelong Learning on Humanoid Robots
What is Lifelong Learning?
• Continual acquisition of knowledge
• Fine-tuning of knowledge
• Learning from experiences
• Retaining of previously learnt experiences
Neural Architectures for Lifelong Learning on Humanoid Robots 4
Figure 1.1: NICO – Neuro-Inspired COmpanion (Source: Kerzel et al. [2]).
Catastrophic Forgetting
• Interference of learnt representations with new information
Neural Architectures for Lifelong Learning on Humanoid Robots 5
Representation 1 Representation 2
Inspiration from Biological Systems
• Neurosynaptic plasticity
• Hippocampus and cerebral cortex
• Transfer learning
• Intrinsic motivation
• Crossmodal learning
• Incremental learning
Neural Architectures for Lifelong Learning on Humanoid Robots 6
• Motivation
• Background
• Approaches
• Results
• Discussion
• Conclusion
7Neural Architectures for Lifelong Learning on Humanoid Robots
Neural Networks
Neural Architectures for Lifelong Learning on Humanoid Robots 8
Figure 2.1: Neural network representation (Source: McDonald [3]).
Convolutional Neural Networks (CNNs)
Neural Architectures for Lifelong Learning on Humanoid Robots 9
Figure 2.2: Convolutional neural network (Source: Cavaioni [1])
Self-Organizing Networks
• Self-Organizing Map (SOM)
• Grow When Required Network (GWR Network)
• Recurrent GWR
Neural Architectures for Lifelong Learning on Humanoid Robots 10
• Motivation
• Background
• Approaches
• Results
• Discussion
• Conclusion
11Neural Architectures for Lifelong Learning on Humanoid Robots
Object Recognition: CNN + Classifier
• Learning from video sequences
• Visual transformations of objects
• Changing environment
Neural Architectures for Lifelong Learning on Humanoid Robots 12
Figure 3.1: iCub (Source: Pasquale et al. [6]).
Object Recognition: CNN + Classifier
Neural Architectures for Lifelong Learning on Humanoid Robots 13
CNN
Classifier
app
le
cup
bal
l
tom
ato
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 14
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 15
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 16
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 17
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 18
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 19
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
Sensorimotor Learning: Self-Organization
• Latency in sensorimotor systems
• Predictive mechanisms for future motor states
• Online learning
Neural Architectures for Lifelong Learning on Humanoid Robots 20
Source: https://upload.wikimedia.org/wikipedia/commons/4/47/Nao_Robot_%28Robocup_2016%29.jpg
Sensorimotor Learning: Self-Organization
Neural Architectures for Lifelong Learning on Humanoid Robots 21
Figure 3.2: The imitation scenario (Source: Mici et al. [4]).
Sensorimotor Learning: Self-Organization
Neural Architectures for Lifelong Learning on Humanoid Robots 22
Figure 3.3: Visuomotor learning (Source: Mici et al. [4]).
Object Recognition: CNN + Self-Organization
Neural Architectures for Lifelong Learning on Humanoid Robots 23
RGB sequence
Pre-trained CNN
Pre-trained CNN
Depth sequence
Feat
ure
s
Self-organizing network
Label
Figure 3.4: Recognition pipeline (Adapted from Part et al. [5]).
• Motivation
• Background
• Approaches
• Results
• Discussion
• Conclusion
24Neural Architectures for Lifelong Learning on Humanoid Robots
Object Recognition: CNN + Classifier
Neural Architectures for Lifelong Learning on Humanoid Robots 25
Figure 4.1: Classification accuracy of the model, which was trained on an incremental number of objects (Source: Pasquale et al. [6]).
Object Recognition: CNN + Classifier
Neural Architectures for Lifelong Learning on Humanoid Robots 26
Figure 4.2: Classification accuracy of the model trained incrementally on different days (Source: Pasquale et al. [6]).
Sensorimotor Learning: Self-Organizing Architecture
Neural Architectures for Lifelong Learning on Humanoid Robots 27
Figure 4.3: Behaviour of the architecture (Source: Mici et al. [4]).
Object Recognition: CNN + Self-Organization
Neural Architectures for Lifelong Learning on Humanoid Robots 28
Figure 4.4: Recognition pipeline (Source: Part et al. [5]).
Discussion
• CNN + Classifier architecture for object recognition:• Features extracted from a CNN are dependent on a dataset the model was
trained on
• Old representations are overwritten by the new information
• Self-organizing architecture for sensorimotor learning:• Incremental online learning and prediction
• Unreliability of visual body tracking framework in complex body positions
• CNN + self-organization for object recognition:• Self-organizing network grows when required
• Temporal context is not considered
Neural Architectures for Lifelong Learning on Humanoid Robots 29
Conclusion
• Lifelong learning is crucial for intelligent robots
• Biological systems provide a basis for the incremental learning
• Self-organizing networks preserve the topology
• CNNs learn efficient feature descriptors
• Catastrophic forgetting increases during incremental tasks
Neural Architectures for Lifelong Learning on Humanoid Robots 30
Thank You!Questions?
Neural Architectures for Lifelong Learning on Humanoid Robots 31
References• [1] Cavaioni, M. Deep Learning series: Convolutional Neural Networks. https://medium.com/machine- learning-
bites/deeplearning-series-convolutional-neural-networks-a9c2f2ee1524 . [Online; accessed 13-November-2018].
• [2] Kerzel, M., Strahl, E., Magg, S., Navarro-Guerrero, N., Heinrich, S., Wermter, S.NICO - neuro-inspired companion: A developmental humanoid robot platform for multimodal interaction. In26th IEEE International Symposium on Robot and Human Interactive Communication, ROMAN 2017, Lisbon, Portugal, August 28 - Sept. 1, 2017, pages 113–120, 2017.
• [3] McDonald, C. Machine learning fundamentals (II): Neural networks. https://towardsdatascience.com/machine-learning-fundamentals-ii-neural-networks-f1e7b2cb3eef. [Online; accessed 13-November-2018].
• [4] Mici, L., Parisi, I. G., Wermter, S. An Incremental Self-Organizing Architecture for Sensorimotor Learning and Prediction. CoRR, abs/1712.08521, 2017.
• [5] Part l. J., Lemon, O. Incremental online learning of objects for robots operating in real environments. In Joint IEEE International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2017, Lisbon, Portugal, September 18-21, 2017, pages 304–310, 2017.
• [6] Pasquale, G., Ciliberto, C., Odone, F., Rosasco, L., Natale, L. Real-world object recognition with off-the-shelf deep conv nets: How many objects can iCub learn? CoRR, abs/1504.03154, 2015.
Neural Architectures for Lifelong Learning on Humanoid Robots 32