14 January 2011
Challenge the future
Delft University of Technology
Intelligent Control and Robotics
Jens Kober
Delft Center for Systems and Control Cognitive Robotics
Delft University of Technology The Netherlands
[email protected] www.jenskober.de
Learning and Autonomous Control
Cognitive Robotics
• Robot Dynamics (dynamic motion control, motor control)
• Intelligent Vehicles (perception and modeling, dynamics,
human factors)
• Human-Robot Interaction (physical human-robot interaction)
• Learning and Autonomous Control (intelligent control,
cognition)
Robot Dynamics
Intelligent Vehicles
Human-Robot Interaction
Robert Babuska
professor
learning and adaptive control
Jens Kober
assistant professor
robotics, learning motor skills
Javier Alonso Mora
assistant professor
robotics, motion planning
Learning and Autonomous Control
Systems and control
model-based design
Artificial intelligence, machine learning
learn a task from experience, without being explicitly programmed for that task
Intelligent control
Intelligent Control
Potential of Machine Learning
• adapt to changes in environment
• find new, better solutions
• teach by demonstration or imitation
in addition: • modeling is tedious, time consuming, expensive
Nature-Inspired Models
y(k+1) = A(q)y(k) + B(q)u(k)
Traditional control Inspiration in nature
if <condition> then <conclusion> artificial neural networks, deep learning
rule-based models
Evolutionary Optimization
xopt = argmax F(x) x
Traditional optimization Inspiration in nature
Population
011000100100101011101100101011
Fitness Function
Best individuals
CrossoverMutation
Genetic and evolutionary algorithms
Swarm Intelligence Inspiration in nature
Ant algorithms, particle swarm optimization
Growing Interest in Robotics
14 January 2011
Challenge the future
Delft University of Technology
Uncertainty due to environment
Robustness
Compliance Learning from mistakes
Safety
Autonomy
Power efficiency
Situation awareness
Challenges
Interaction with humans
Future Robots
• Need to operate under unforeseen circumstances
• Interact with humans in an intelligent way
• Learn from mistakes, improve over time (performance,
energy)
Learning and adaptation is essential!
• Impossible to preprogram all tasks or behaviors
• Economic point of view: short design time and low costs
14 January 2011
Challenge the future
Delft University of Technology
• Adaptive and learning control systems
• Reinforcement learning, optimal control
• Nonlinear adaptive control
• Deep learning for robotics
• Data-effective learning algorithms
• Sensor fusion
• Supervisory and distributed control design
• Motion planning, coordination
• Multi-agent systems
Main Research Topics
Emphasis on novel, generic, widely applicable techniques
Realistic / real-world case studies and applications
Unsupervised learning
(without ‘teacher’)
Spectrum of Learning Techniques
Reinforcement learning
(learn directly to control)
Supervised learning
(with ‘teacher’)
Construct a Robot Model from Data
actuate motors and observe the response
Two ways to model the system:
1. Physical modeling
2. System identification
Maarten Vaandrager
Unsupervised learning
(without ‘teacher’)
Spectrum of Learning Techniques
Reinforcement learning
(learn directly to control)
Supervised learning
(with ‘teacher’)
Unsupervised Learning - Clustering
• Discover automatically groups and structures in data
Applications: • Robot perception, vision
• Data-driven construction of
dynamic models
Construction of Nonlinear Models
Nonlinear and uncertain behavior
Experimental data
Unsupervised learning
(without ‘teacher’)
Spectrum of Learning Techniques
Reinforcement learning
(learn directly to control)
Supervised learning
(with ‘teacher’)
Reinforcement Learning
Goal:
Adapt the control strategy so that the sum of rewards over time is maximal.
Goals Performance evaluation
reward
state
Adaptive
controller System
action
Inspiration - animal learning (reward desired behavior)
• Approximation in high-dimensional continuous spaces
• Computationally effective methods
• Constrained learning and convergence
Reinforcement Learning for Control
Learning in Real World, in Real Time
E. Schuitema (collaboration with the Biorobotics Lab, TU Delft - M. Wisse and P. Jonker).
Examples of Recent MSc Projects
Effective Real-Time Reinforcement Learning
Learning for Disturbance Compensation
•C … Model based controller
•F … Learning disturbance compensator
RL-based Compensation
New MSc Project Vacancies
Hip Surgery Robot Design
•3D modeling of bone based on X-ray images
•Design of a specialized two-degree-of-freedom robot
•Robot – surgeon interaction
Reinforcement Learning for Walking Robot
•Movement primitives and RL
•Dimensionality reduction
Robotic Platforms
Collaboration with AE
Relevant Elective Courses
• SC42020 Modern Robotics
• ME41025 Robotics Practicals
• SC42090 Robot Motion Planning and Control
• SC42050 Knowledge Based Control Systems
• IN4320 Machine Learning
• IN4085 Pattern Recognition
• CS4180 Deep Learning
• IN4010 Artificial Intelligence Techniques
Questions?
