Cyber-physical systems architecture breakthrough:learning of visuomotor coordination

Moscow9-dec- 2015

107th meeting of INCOSE Russian chapter

2

Cyber-physical systems

• Cyber-Physical Systems or “smart” systems are co-engineered interacting networks of physical and computational components• CPS include:

- Internet of Things (IoT) - Industrial Internet - Smart Cities - Smart Grid - "Smart" Anything (e.g., Cars, Buildings, Homes, Manufacturing, Hospitals, Appliances) • NIST CPS Public Working Group -- http://

www.nist.gov/cps/index.cfm• NSF -- http://cps-vo.org/

3

Draft Framework for Cyber-Physical Systemshttp://www.nist.gov/el/nist-releases-draft-framework-cyber-physical-systems-developers.cfm

This is all about systems engineering!

4

From «smart» to «intelligent»

How CPS perform it Decision?

SensorsConsoles

ActuatorsMonitors

http://www.nist.gov/el/nist-releases-draft-framework-cyber-physical-systems-developers.cfm

5

Where is that «intelligence»?

Cyber-physical device

Software

Interfaces and communication

s

Cognitive processing

Hardware

Sensors Mechanics Actuators

6

Knowledge engineering

• Decision is carefully programmed (manually).• Example: robot-«butterfly», https://

youtu.be/kyvW5sOcZHU, https://youtu.be/V30e77x8BQA

• Every type of movement should be programmed anew• Non-adaptable to changes of environment and device• The best science available up today!

• Perfect, if CPS perform only one or two movements. Not for robots, definitely!

7

Goal: CPS capuchin-like

• Jurgen Schmithuber (July 2015): In order to pick a fruit at the top of a tree, Capuchin monkey plans a sequence of sub-goals (e.g., walk to the tree, climb the tree, grab the fruit, …) effortlessly. We will have machines with animal-level intelligence in 10 years.https://sites.google.com/site/deepernn/home/blog/briefsummaryofthepaneldiscussionatdlworkshopicml2015

•Needs planning•Needs great visuomotor coordination!• Impossible to program manually up to date.

8

New in «decisions»: learn to decide!

Machine learning and reasoning:• Symbolic (by induction)• Evolutionary (by genetic programming)• Bayes (by probability assesement)• By analogy• Connectivist (deep learning, artificial neuron nets)

The Master Algorithm: combine ‘em all!

http://www.amazon.com/dp/0465065708/

9

Evolutionary roboticshttps://en.wikipedia.org/wiki/Evolutionary_robotics

Flexible Muscle-Based Locomotion for Bipedal Creatureshttps://vimeo.com/79098420

Evolving Soft Robots with Multiple Materials (muscle, bone, etc.) https://youtu.be/z9ptOeByLA4

10

Breakthrough: deep learning

http://www.computervisionblog.com/2015/11/the-deep-learning-gold-rush-of-2015.html?m=1

11

Reinforcement learning + deep learning

Not only «visuo» but «motor» too – with a coordination!

12

Not a rocket science• Open science• ArXiv, GitXiv• Open Source libraries• GPU in all computer stores• Conferences: ICML 2014 – 2500 participants, ICML

2015 – 4000 participants• Multiple schools (summer schools, university

courses, hackathons)• Competitions (e.g. Kaggle)

http://deephack.me/10 teams

13

Visumotor policies/decisions/behavior/coordination

VisuoWorld

reconstruct

Motor Visuo Motor

mediated perception behavior reflex

And everything in between!

14

DeepDriving

• train a deep Convolutional Neural Network (CNN) using 12 hours of human driving in a video game

• show that our model can work well to drive a car in a very diverse set of virtual environments

• train another CNN for car distance estimation on the KITTI dataset, results show that the direct perception approach can generalize well to real driving images

• Open sourced

• Autopilot Driving is not a miracle now: Tesla X, Google car, AVRORA/KAMAZ, Volvo trucks, and counting

http://deepdriving.cs.princeton.edu/

15

Cortical sensory homunculus

• Body is an easy part.• Manipulation is difficult!• Non-prehensile manipulation is included.

https://en.wikipedia.org/wiki/Cortical_homunculus

16

Visuomotor examples• End-to-End Training of Deep Visuomotor Policies,

http://arxiv.org/abs/1504.00702

• Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours,http://arxiv.org/abs/1509.06825, https://youtu.be/oSqHc0nLkm8

17

Self-learning robots

• Fanuc: $7.5mln for 6% in Preferred Networks• ABB invested up to

$10mln in Vicarious

http://www.bloomberg.com/news/articles/2015-12-03/zero-to-expert-in-eight-hours-these-robots-can-learn-for-themselves

Osaro -- http://www.osaro.com/, learning in environment, including cooperation with humans, http://www.technologyreview.com/news/543956/a-supercharged-system-to-teach-robots-new-tricks-in-little-time/

18

Non-prehensile example

• Deep Spatial Autoencoders for Visuomotor Learning,http://rll.berkeley.edu/dsae/ (video)

19

Learning to plan

https://drive.google.com/file/d/0B0PX5JnpNX8yT0JoODdHaklMLWs/view

Neurocognitive Architecture for Autonomous Task Recognition, Learning and Execution (NARLE)

20

Visuomotor hardware

Quantum computer: waiting, but promising – currently 100mln times faster than classical desktop (http://googleresearch.blogspot.com/2015/12/when-can-quantum-annealing-

win.html).

21

Visuomotor sensors• Computational multi-lens optics (near future)• Solid state LIDARs for driving (below $100 in five

years, now $1000) -- http://www.quanergy.com/, http://velodynelidar.com/, includes processor and neural software• But Elon Musk tell that lidar not needed, only

optical cameras and radar

http://blog.lidarnews.com/lidar-for-self-driving-cars/

22

Voice interface for visuomotor goal settings• Deep learning is a leading technology for a voice recognition• Voice command interface is not a problem today

• General intelligence of a CPS is a problem!

Company Name of Personal assistant Google Google Apple Siri Microsoft Cortana Facebook M Amazon Alexa [Cyber-physical system vendor]

???????????

23

Thank you!Anatoly LevenchukTechInvestLab, presidentINCOSE Russian chapter, research directorhttp://ailev.ru (in Russian)ailev@asmp.msk.su

TechInvestLabINCOSE Russian chapter