Chung-Yeon LEE

Biointelligence Lab, Seoul National University

8 March 2019

BI Seminar on Neuro-Cognitive Developmental Machine Learning

Self-Taught Robots

(Diana Kwon, In Scientific American, March 2018)

2

Present-day AI system v.s. Human baby

• From early infancy onward our offspring develop by exploring their surroundings and experimenting with movement and speech.

Collect data themselves

Adapt to new situations

Transfer expertise

across domains

Learn a game with clear

rules

Play millions of times

3

Prediction Machine

Our brains are constantly trying to predict the future, and updating their expectations to match reality.

• Visual processing

The visual cortex receives inputs from the eye.

Downward connections seem to carry predictions.

A signal coding of the prediction errors returns to the higher

levels of the brain.

Other downward signals send commands to move the eye

muscles, adjusting what we see.

• Cascade of predictions

When the brain generates a prediction errors, it uses this

information to update its expectations,

and select actions that will help resolve the discrepancy

between beliefs and reality.

4

Counting with Fingers

A fundamental difference between us and many present-day AI systems is that we possess bodies that we can use to move about and act in the world.

• How crucial the body is for procuring knowledge?

“The shape of the robot’s body, and the kinds of things it can do, influences

the experiences it has and what it can learn from.”

Posture affects how robots and infants map words to objects

iCub, an android being studied at the University of Plymouth in England,

can learn new words, such as “ball” or “cup”, more easily if the experimenter

consistently places the object at the same location while naming it.

Interestingly, when the investigators repeated the experiment with 16-

month-old toddlers, they found similar results: relating objects to particular

postures helped small children learn.

Using the body can also help to gain basic numerical skills

When the robots were taught to count with their fingers, their deep neural

networks represented numbers more accurately.

5

Curiosity Engines

Novelty also helps children learn.

• Infants use “unexpectedness” as a cue for learning

Babies who saw apparently solid and weighty objects moving through a

wall or past the edge of a table without falling looked intently at them.

When given the opportunity to explore these peculiar objects,

they did so by banging them on the floor, as if to test their solidity

or dropping them, as if to test their weightiness.

Embodied information seeking& curiosity‐driven learning

Intrinsic motivation: a decrease in prediction error yields a reward

A toddler will likely choose to play with a toy car rather than with a 100-piece

jigsaw puzzle—arguably because her level of knowledge will allow her to

generate more testable hypotheses about the former.

In Oudeyer’s experiment, robots were able to acquire basic skills (e.g. grasping

objects and interacting vocally with another robot), without having to be

programmed to achieve these specific ends.

“ A side effect of the robot exploring the world,

driven by the motivation to improve its predictions. ”

6

Altruistic Androids

The drive to reduce prediction errors can induce elementary social behavior.

• Emergence of altruistic behavior through the minimization of prediction error

After iCub was taught to push a toy truck, it might observe an

experimenter failing to complete that same action.

Often it would move the object to the right place.

Help to understand developmental disorders

Certain autistic individuals may have a higher sensitivity to prediction

errors, making incoming sensory information overwhelming.

ADHD perpetually attracted to unpredictable stimuli in their surroundings.

In a predictive coding framework, both patterns can be simulated well.

Cognitive mirroring (Director: Yukie Nagai, 2016-2022)

As the robot and person communicate using body language and facial

expressions, the machine will adjust its behaviors to match its partner.

Experimenters can use robots to model human cognition.

7

Robots of the Future

Robots that can develop humanlike intelligence are far from becoming a reality

Scientists need to overcome technical hurdles

The brittle bodies and limited sensory capabilities of most robots

Advances in areas such as soft robotics and robot vision may help this happen

Far more challenging is the incredible intricacy of the brain itself

Caregivers are crucial to children’s development

The process of gradually accumulating knowledge may also be indispensable.

8

Robots of the Future

“One day humans may succeed in creating a robot thatcan explore, adapt and develop just like a child.

In the meantime, childlike robots will continue to providevaluable insights into how children learn.”

Appendix

9

BI Seminar on Neuro-Cognitive Developmental Machine Learning

10

iCub - A Humanoid Robot

• A humanoid robot as an open platform for research in embodied cognition

Built by a team at IIT for research purposes

It comes with no preprogrammed functions, allowing

scientists to implement algorithms to their experiments.

• The size of a three and half year old child (104 cm)

• Capabilities of iCub

Crawling (using visual guidance w/ optic marker on the floor)

Solving complex 3D mazes

Archery, shooting arrows with a bow(Trained to hit the center of the target)

Facial expressions(allowing the iCub to express emotions)

Force control(exploiting proximal force/torque sensors)

Grasping small objects (e.g. balls, plastic bottles)

Collision avoidance within non-static environments, as

well as, self-collision avoidance

• Project: http://robotcub.org/• API: http://www.yarp.it/• H/W: https://svn.robotology.eu/repos/iCubHardware-pub/trunk/mechanics/

11

Prediction-based Learning

• Prediction-based Learning

Jun Tani et al. developed a prediction-based neural network

for learning basic movements and tested how well these

algorithms worked in robots.

The machines, they discovered, could attain elementary

skills such as navigating simple environments, imitating

hand movements, and following basic verbal commands like

“point” and “hit.”

Jun Tani, Exploring Robotic Minds, pp.192-193

12

Curiosity Engines