artificial intelligence is back, deep learning networks and quantum possibilities
TRANSCRIPT
80s - Genesis
When I started my career at MIT AI was thought to be on the verge of discovering the secret of human intelligence. My MIT programming contest victory, mini-max rethought
The initial successes were things like blocks world, some initial neural network work, Mathematica
The idea was that if you combined enough smart things like mathematica and blocks world you would eventually get intelligence
Neural networks didn’t go very far. Lots of problems getting the networks to converge.
I saw through this and abandoned the study quickly. I called it the chicken problem. The problem of deeplearning had not really been attempted
Marvin Minsky called the bluff publicly and the industry collapsed quickly after that
90s – Rule based systems
In the late 80s a revitalization of AI occurred and a new heyday based on rule based systems that allowed you to operate on knowledge systems. Primary among these was KEE (Knowledge Engineering Environment) from Intellicorp
This ultimately failed again and seemed to put AI into a deep funk
00s - Statistics
Machine Learning - Brute force statistical approach
This is a statistical approach to learning
Find an “algorithm” or “model” or “formula” which mimics the data close enough to get paid, >10 different statistical techniques in use including neural nets
Pass lots of “labeled” data through the system
Watson
Like Mathematica a hard-core machine learning approach
2010 Convoluted Neural Nets
Neural Net called Convoluted Neural Network(CNN) behind the scenes advancements Invariance built-in (Convolution)
Back propagation and various gradient descent algorithms
2 layers at a time “learn” abstraction and filter – filter layer reduces computation
Brain is composed of layers and vision system seems to be similar in learning abstractions in layers
First 2 layers learn lip, ear, nose, brow, tire, window, … Next 2 layers combinations make face, truck
Next 2 Layers – Female, Male, Mustang, ..
Large data sets of “labeled data” is same problem as Machine Learning
Deep Belief NetworksCNN +
The holy grail of learning is not having to use “labeled” datasets. DBN gets around this by using a Markov probabilistic approach to neuron evolution. No “labeled” data initially makes it much easier to use.
After initial training we pass “labeled” data through to reinforce learned pathways and do better selection of best abstractions.
Training individual layers is much faster and results in better abstractions
Recurrent (Feed lowest layer back into top) Unlimited layers
Addition of Memory (New Neuron type) – secret sauce Made cursive much better
Achievements of DeepLearning
The best at cursive recognition (DBN have beaten all others)
The best at text recognition
The best at object recognition (54% to 64%, google+)
The best at facial recognition (facebook 97% accurate better than FBI, 9-layer DBN)
The best at voice recognition (100% penetration DBN)
Used in Watson now
Possibly other purposes at Google
DeepMind acquisition gives Google 2 of the 4 geniuses in the field, facebook 1 and only 1 left in academia
Why is the AI problem hard?
Brain is remarkable at invariance: translation, scale, distortion, blocking, color, quality, background Very hard to replicate anything close to flexibility, uses many clues
Brain applies abstractions across different disciplines There is almost never an attempt to learn more than one “thing”
because different input tend to create instability and less precise recognition
Brain creates abstractions upon abstractions in a stable way across dozens/hundreds of layers or levels With little problems with back propagation problems or unlearning
Have not been able to go beyond a few layers
Fears of DeepLearning
Elon Musk, Stephan Hawking, Bill Gates
All fear DeepLearning from DeepMind
Elon says what he knows/has seen of DeepLearning gives him fear
Google did establish a “ethics panel” to make sure to use AI safely and was key to DeepMind agreeing to be bought by Google
It is a love/hate relationship. We fear AI being smarter and malevolent taking our jobs away or killing like we fear aliens instinctively. We fear losing our “special” status and place. We also want to like and be fair to any sentient creatures. There is a natural curiosity that seems inevitably will get us there.
Fear of accidentally creating intelligence, some simple thing replicates like a virus in a computer and takes over. In Hyperion (Dan Simmons) an 88-byte code fragment that replicates and evolves eventually becomes AI.
Isaac Asimov established the 3 rules of Robot AI… Cannot harm a human … How would you put such a rule in a CNN based AI?
Why is AI finding interest again?
Social is very much about pictures, voice, video, etc… which is greatly enhanced by image and facial recognition
Making things intelligent or at least a little intelligent, for instance recognizing voice better – siri, skyvi makes things easier to use, provide much greater value.
Self-driving cars recognizing signs, people, etc…
Smarter is better even a little bit smarter … see google
BIG DATA SkyNetworks, H2O.ai, 16z, Azure Machine Learning Studio,
Google…
Can DeepLearning become “really intelligent?”
Required
1) Plasticity
2) Many levels of abstraction
3) Planning
4) Memory in general
5) Self-perception
6) Creativity
Unsure but associated
7) Common sense
8) Emotions
9) Self-preservation
10) Qualia
11) Experiential Tagged Memory
12) Consciousness
CNN lack critical features of “intelligence”, “sentience” or “consciousness”
Quantum Computers could be a pathway to AI
Performance could be log the performance of traditional computers, i.e instead of a million computers 6 would be enough, instead of a trillion computers 12 would be enough. For some problems simply x ^ (n/2) improvement so instead of 10^60 operations, 10^30 operations.
Many of the things we think a brain does seem to be the kinds of things quantum computers would be good at. Pattern recognition, searching databases, optimization, i.e. neural weighting optimization
The applications of Quantum computers would be in optimization, recognition problems and security applications.
Quantum MechanicsRichard Feynman: If you think you understand quantum mechanics, you don't understand quantum mechanics.
Wave/Particle Duality and the Measurement Problem:
A particle acts like a self-interfering wave which when you look at it collapses from being anywhere in space to one location breaking the speed of light.
12 current theories of “why” collapse seems to happen including the newest one called quantum darwinism in which space itself has memory and evolution mimicking genetic evolution. Many Worlds, Copenhagen, Quantum Gravity
Quantum foam : virtual particles the infinite possibilities, Higgs Particle, non-zero vacuum state
Quantum Superposition: Particles when not being observed seem to occupy ALL possible states and take all possible paths simultaneously. Yet when “measured” they choose with probability varying by the energy consumed in the whole process. All paths are possible and appear but the least energy path is predominant. This seems simple but how does NATURE figure this out? It’s nontrivial.
Calculate the solution to a 3 body problem in quantum mechanics is nearly impossible nearly an infinite number of possible loops a task that takes a year of supercomputer time … yet nature does this 10^40 times a second for 10^75 particles Many worlds is popular because nature doesn’t compute anything, it just splits for every possible choice and the highest probability is the universe we happen to statistically find ourselves in. Also eliminates collapse problems but introduces the problem of an infinite number of universes created every second.
Quantum Computers are way different than regular computers
Basic Quantum Operations:
Hadamard Operation: Put a qubit into multiple states with equal probability
Result a true random number generator
Do Hadamard operation twice get a NOT operation
Schor Algorithm – optimization, factorization
A particle traverses a puzzle we set up
Grovers algorithm – database search
QCL – Quantum computing language (D-Wave)
In a quantum computer we set up the experiment, then run it, the answer is whatever nature does.
How practical are Quantum Computers?
D-Wave-2 512 qubit used by Google to demonstrate superior performance to any existing computer (up to 5x or more) … However not suitable to all problems
D-Wave-3 releases 1152 qubit computer in March, doubling capacity every year and improving entanglement.
2 ^ 1152 patterns of 1152 bits in superposition simultaneously.
NOT doubling computing SQUARING computing capability!
D-Wave 6 might only by 8000 qubit computer but more powerful than all computers on the earth existent today
Brain is a quantum computer?
Roger Penrose believes this. If he believes it then it is worth looking at.
Evidence that nature “Uses” quantum mechanics to solve difficult problems
Taking a single photon from the sun to build plants
For Birds to sense direction
In the eyes, ears, nose creating ultra-sensitive senses
If nature uses it for these functions it could also use it for many other functions
We see that quantum mechanics is good at solving problems a brain has to do: pattern recognition, efficient operation, searching for information, neuron weighting calculations
Brain science has not discovered how the brain stores all the experiential information we consume, nor the process of pattern recognition, reasoning or incredibly complex weighting optimization characteristic of learning
Recent evidence of quantum effects in microtubules of dendrites of nerves. Recent evidence of molecular process of phosphorylating microtubules to encode memories.
If Brain uses QM we are far away from “intelligence”
The human brain could be composed of up to 1 trillion quantum computers or a trillion trillion qubit quantum computer or some combination
Compared to a 1152 qubit D-wave at $10 million quite a deal
We know that Elephants with the largest neural structure compared to humans (1/2 the size) are not as intelligent as humans.
The largest CNN we have built is 650,000 virtual neurons, which is between 1/10,000 -> 1/1,000,000 of the brain
AI still probably a distance away but extremely useful
AI is extremely useful and getting better They can and will be able to do some things better than humans
They will be able to make our lives easier
They will in some cases remove people’s jobs
They make programs more intuitive, helpful, efficient
Fears of AI overtaking humans are overblown They make stupid errors, they have no common sense
They will not be gaining consciousness soon
They show no creativity
They show no planning ability
They show no ability to learn multiple disciplines
How Can WSO2 benefit
Machine Learning Adapter and Connectivity
Provide easy interfacing to machine learning systems
Wizard-like simplicity to setup bigdata systems
More and more in bigdata usage
D-wave adapters to funnel data and programming to and from d-waves to solve problems
WSO2 Deep Learning Server
Configurable Layers and parameters
Autoscaling
Roger Penrose and Consciousness
Roger Penrose invented Spinors and Twistor Space.
He was slow learner. His high school teacher had to give him two classes to finish tests. He worked everything out from basics. He was extremely visual.
He did what physicists told him he shouldn’t
Twistor Space is discrete NOT continuous. When you calculate from one vertex to another you get a result with different space-time coordinates. Intermediate space-time coordinates don’t exist.What seems like “fuzzy spooky foam probabilistic action at a distance ” in space-time appears as simply non-existant points in a Twistor lattice.Collapse is not a problem (Measurement problem) because the motion of particles is simply moving from vertex to vertex with different probabilities.
All Physicists say: “You can’t possibly imagine the quantum world so give up. Just study the math, forget trying to visualize it.”
Roger invented Spinors to visualize spinning particles involved “making real” imaginary numbers.
Imaginary numbers are central to quantum physics. They appear everywhere. So, Roger created Twistor space: 5 dimensions, 2 complex
Quantum ConsciousnessOrchestrated Objective Reduction
Strong Evidence of Quantum processes in nature and in microtubules of dendrites in brains
Penrose calculates decoherence time at approx 40 times/second which corresponds to brain waves – otherwise completely unmotivated
Using Turing machines and some Godel theorems Penrose shows the human brain does things no computer can EVER DO.
He says the human brain and consciousness is not only a quantum computer but that there are capabilities of this quantum computer that exceed even what we know about quantum mechanics. I.e. New physics.
The Theory says that human consciousness is in the quantum fuzz similar to Quantum Darwinism and the brain is a transducer. Evidence that something makes decisions before the human brain is even aware of it. Which means that our consciousness may transcend our bodies