jdb code biology and ai final

Code Biology and (the future of) Ar5ficial Intelligence

Joachim De Beule

Recent advances in AI Deep learning

A dark future Superintelligences more dangerous than nukes

A brighter future Collec5ve intelligence

“A revolu*on in ar*ficial intelligence is currently sweeping through computer science. The technique is called deep learning and it’s affec*ng everything from facial and voice to fashion and economics.”

“In some sense deep learning is what happened when machine learning hit big data” “Two kinds of data: raw data (pictures, music, …) and symbolic data (text)” “With deep learning, we can bridge the gap between the physical world and the world of compu5ng”

-‐-‐ Adam Berenzweig, founding CTO of Clarifai

Ref: Deep Learning: Intelligence from Big Data, Tue Sep 16, 2014, Stanford Graduate School of Business

Neural Networks of the 80’s

What’s New?

ü  Big Data •  The internet & Social Media •  Metadata: tags, transla5ons, … •  Mechanical Turk


What’s New?

ü  Big Data ü  Scale •  80’s: 1-‐10M (106) neurons/synap5c connec5ons •  Google Brain: 1B (109)

(10M video’s, 16k computers, 3 days) •  Adult: 100T (1014) •  Infant: 1Q (1015)


What’s New?

ü  Big Data ü  Scale ü  Algorithmic advances •  Successive layers of learning/representa5on •  Unsupervised pre-‐training

à Structure NN (feature detectors) •  Then supervised back-‐prop

à classify/predict labeled data


What’s New?

ü  Big Data ü  Scale ü  Algorithmic advances

We have been able to reduce the word error rate for speech by over 30% compared to previous methods. This means that rather than having one word in 4 or 5 incorrect, now the error rate is one word in 7 or 8. While s5ll far from perfect, this is the most drama5c change in accuracy since the introduc5on of hidden Markov modeling in 1979, and as we add more data to the training we believe that we will get even becer results.

November 18, 2014

Asked whether two unfamiliar photos of faces show the same person, a human being will get it right 97.53 percent of the 5me. New sodware developed by researchers at Facebook can score 97.25 percent on the same challenge, regardless of varia5ons in ligh5ng or whether the person in the picture is directly facing the camera.

Feb 26, 2015

•  Isotherm is to temperature as isobar is to? (i) atmosphere, (ii) wind, (iii) pressure, (iv) la*tude, (v) current.

•  Iden*fy two words (one from each set of brackets) that form a connec*on (analogy) when paired with the words in capitals: CHAPTER (book, verse, read), ACT (stage, audience, play).

•  Which is the odd one out? (i) calm, (ii) quiet, (iii) relaxed, (iv) serene, (v) unruffled.

•  Which word is closest to IRRATIONAL? (i) intransigent, (ii) irredeemable, (iii) unsafe, (iv) lost, (v) nonsensical.

•  Which word is most opposite to MUSICAL? (i) discordant, (ii) loud, (iii) lyrical, (iv) verbal, (v) euphonious.

Ref: arxiv.org/abs/1505.07909 : Solving Verbal Comprehension Ques5ons in IQ Test by Knowledge-‐ Powered Word Embedding

The future?

“I am in the camp that is concerned about super intelligence. First the machines will do a lot of jobs for us and not be super intelligent. That should be posi*ve if we manage it well. A few decades a[er that, though, the intelligence is strong enough to be a concern. I agree with Elon Musk and some others on this and don't understand why some people are not concerned.”

Stephen Hawking (hcp://www.bbc.com/news/technology-‐30290540)

"The development of full ar*ficial intelligence could spell the end of the human race […] It would take off on its own, and re-‐design itself at an ever increasing rate […] Humans, who are limited by slow biological evolu*on, couldn't compete, and would be superseded.”

•  Oren Etzioni (Computer science, Univ. Washington, CEO of the Allen Ins5t. for Ar5ficial Intelligence):

“The popular dystopian vision of AI is wrong for one simple reason: it equates intelligence with autonomy. That is, it assumes a smart computer will create its own goals, and have its own will, and will use its faster processing abili*es and deep databases to beat humans at their own game. It assumes that with intelligence comes free will, but I believe those two things are en*rely different”

•  Michael Licman (AI, Brown Univ., former program chair for the Ass. of the Advancmnt of AI):

“There are indeed concerns about the near-‐term future of AI — algorithmic traders crashing the economy, or sensi*ve power grids overreac*ng to fluctua*ons and shucng down electricity for large swaths of the popula*on. [...] These worries should play a central role in the development and deployment of new ideas. But dread predic*ons of computers suddenly waking up and turning on us are simply not realis*c.”

•  Yann LeCun (Facebook’s director of research, one of the world’s top experts in deep learning):

“Some people have asked what would prevent a hypothe*cal super-‐intelligent autonomous benevolent A.I. to “reprogram” itself and remove its built-‐in safeguards against gecng rid of humans. Most of these people are not themselves A.I. researchers, or even computer scien*sts.”

•  Andrew Ng (founded Google’s Google Brain project, now Chief Scien5st at Baidu):

“Computers are becoming more intelligent and that’s useful as in self-‐driving cars or speech recogni*on systems or search engines. That’s intelligence,” he said. “But sen*ence and consciousness is not something that most of the people I talk to think we’re on the path to.”

Assump5on: Deeper level neurons are more “abstract”

However, what was discovered:

-‐  A single neuron's feature is no more interpretable as a meaningful feature than a random set of neurons.

-‐  NN’s do not "unscramble" the data by mapping features to individual neurons in say the final layer. The informa5on that the network extracts is just as much distributed across all of the neurons as it is localized in a single neuron.

-‐  Furthermore, Every deep neural network has "blind spots" in the sense that there are inputs that are very close to correctly classified examples that are misclassified.

The Symbol Grounding Problem

0100001101010100111101010101001101001010101001011010101111…

Jpeg coding

01000001 01000011 01010100

ASCII coding

CAT

Deep NN

Harnad, S. (1990)


0100001101010100111101010101001101001010101001011010101111…

Jpeg coding

01000001 01000011 01010100

ASCII coding

CAT Human coding

Human coding

Deep NN

Human Qualifica5on or Semiosis

Harnad, S. (1990)


•  Categories (signs and meanings) are ar5facts •  The rela5on between them is arbitrary •  They are realized by agents performing semiosis

Diagram of Self-‐regula5on

The future?

The future?

“Collec*ve intelligence is the opposite of ar*ficial intelligence”

Ø  Outer world onto inner world (human neuronal coding)

Ø  Inner worlds onto each other (collec5ve intelligence) Ø  Collec5ve intelligence onto inner

world

•  Semiosis (life) = self-‐regula5on (produc5on and consump5on of variety, closure)

Self-‐regulatory system (Agent)

•  Semiosis (life) = self-‐regula5on (produc5on and consump5on of variety, closure) •  Tool usage (supplementa5on of variety)

•  Semiosis (life) = self-‐regula5on (produc5on and consump5on of variety, closure) •  Tool usage (supplementa5on of variety) •  Extension and specializa5on (constraints)

“Now, as the Internet revolu*on unfolds, we are seeing not merely an extension of mind but a unity of mind and machine, two networks coming together as one.”

[Deepstuff, May 25, 2015]

•  Semiosis (life) = self-‐regula5on (produc5on and consump5on of variety, closure) •  Tool usage (supplementa5on of variety) •  Extension and specializa5on (constraints) •  Coordina5on (conven5onal codes)

Agent 1

Agent 2

•  Semiosis (life) = self-‐regula5on (produc5on and consump5on of variety, closure) •  Tool usage (supplementa5on of variety) •  Extension and specializa5on (constraints) •  Coordina5on (conven5onal codes)

à Metasystem or “Major Transi5on”

Agent 1

Agent 2

Meta agent

“In a sense, deep learning is what happened when machine learning hit big data” “Two kinds of data: raw data (pictures, music, …) and symbolic data (text)” “With deep learning, we can bridge the gap between the physical world and the world of compu5ng”

-‐-‐ Adam Berenzweig, founding CTO of Clarifai

The Next Major Transi5on?

Symbolic

Collec5ve intelligence (deep learning)

Physical

Collec5ve ac5ng (da5ng, vo5ng, …)

Informa5on seeking ac5ng

Tagging and training

Replica*on always involves coding!

3, 15 or 33 numbers?

The Symbol Grounding Problem Harnad, S. (1990)

ü  A robot may not injure a human being or, through inac5on, allow a human being to come to harm.

ü  A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.

ü  A robot must protect its own existence as long as such protec5on does not conflict with the First or Second Laws.

MIT Technology review, Robert D. Hof, April 23, 2014

Professor Geoff Hinton, who was hired by Google two years ago to help develop intelligent opera5ng systems, said that the company is on the brink of developing algorithms with the capacity for logic, natural conversa5on and even flirta5on. “Basically, they’ll have common sense” “Thought vectors, Hinton explained, work at a higher level by extrac5ng something closer to actual meaning”