![Page 1: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/1.jpg)
Focus on Unsupervised Learning
![Page 2: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/2.jpg)
No teacher specifying right answer
![Page 3: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/3.jpg)
No teacher specifying right answerTechniques for autonomous SW or
robots to learn to characterize their sensations
![Page 4: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/4.jpg)
“Competitive” learning algorithm
![Page 5: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/5.jpg)
“Competitive” learning algorithm Winner-take-all
![Page 6: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/6.jpg)
![Page 7: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/7.jpg)
![Page 8: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/8.jpg)
![Page 9: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/9.jpg)
Learning Rule: Iterate
![Page 10: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/10.jpg)
Learning Rule: IterateFind “winner”
![Page 11: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/11.jpg)
Learning Rule: IterateFind “winner”Delta = learning rate * (sample –
prototype)
![Page 12: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/12.jpg)
Example: Learning rate = .05 Sample = (122, 180) Winner = (84, 203)
DeltaX = learning rate * (sample x – winner x) DeltaX = .05 * (122 – 84) DeltaX = 1.9 New prototype x value = 84 + 1.9 = 85.9
DeltaY = .05 * (180 - 203) DeltaY = -1.15 New prototype y value = 203 -1.15 = 201.85
![Page 13: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/13.jpg)
![Page 14: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/14.jpg)
Python Demo
![Page 15: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/15.jpg)
Sound familiar?
![Page 16: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/16.jpg)
ClusteringDimensionality ReductionData visualization
![Page 17: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/17.jpg)
![Page 18: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/18.jpg)
![Page 19: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/19.jpg)
![Page 20: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/20.jpg)
![Page 21: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/21.jpg)
Yves Amu Klein’s Octofungi uses a kohonen neural network to react to its environment
![Page 22: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/22.jpg)
Associative learning method
![Page 23: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/23.jpg)
Associative learning methodBiologically inspired
![Page 24: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/24.jpg)
Associative learning methodBiologically inspiredBehavioral conditioning and
Psychological models
![Page 25: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/25.jpg)
activation = sign(input sum)
![Page 26: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/26.jpg)
activation = sign(input sum)+1 and -1 inputs
![Page 27: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/27.jpg)
activation = sign(input sum)+1 and -1 inputs2 layers
![Page 28: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/28.jpg)
![Page 29: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/29.jpg)
weight change = learning constant *
neuron A activation * neuron B activation
![Page 30: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/30.jpg)
weight change = learning constant *
desired output * input value
![Page 31: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/31.jpg)
![Page 32: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/32.jpg)
![Page 33: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/33.jpg)
![Page 34: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/34.jpg)
Long-term memory
![Page 35: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/35.jpg)
Long-term memory Inspired by Hebbian learning
![Page 36: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/36.jpg)
Long-term memory Inspired by Hebbian learningContent-addressable memory
![Page 37: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/37.jpg)
Long-term memory Inspired by Hebbian learningContent-addressable memoryFeedback and convergance
![Page 38: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/38.jpg)
Attractor – “a state or output vector in a system
towards which the system consistently evolves toward given a specific input vector.”
![Page 39: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/39.jpg)
Attractor Basin – “the set of input vectors surrounding
a learned vector which will converge to the same output vector.”
![Page 40: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/40.jpg)
![Page 41: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/41.jpg)
Bi-directional Associative MemoryAttractor network with 2 layers
![Page 42: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/42.jpg)
Smell Taste
![Page 43: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/43.jpg)
Bi-directional Associative MemoryAttractor network with 2 layers Information flows in both directions
![Page 44: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/44.jpg)
Bi-directional Associative MemoryAttractor network with 2 layers Information flows in both directionsMatrix worked out in advance
![Page 45: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/45.jpg)
Hamming vector – vector composed of
+1 and -1 only
Ex. [1,-1,-1,1] [1,1,-1,1]
![Page 46: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/46.jpg)
Hamming distance – number of components by which 2 vectors differ
Ex. [1,-1,-1,1] and [1,1,-1,1]Differ in only one element (index 1)Hamming distance = 1
![Page 47: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/47.jpg)
Weights are a matrix based on memories we want to store
To associate X = [1,-1,-1,-1] With Y = [-1,1,1]XY
1 -1 -1 -1
-1 -1 1 1 1
1 1 -1 -1 -1
1 1 -1 -1 -1
![Page 48: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/48.jpg)
[1,-1,-1,-1] -> [1,1,1] and [-1,-1,-1,1] -> [1,-1,1]
+
=
1 -1 -1 -1
1 -1 -1 -1
1 -1 -1 -1
-1 -1 -1 1
1 1 1 -1
-1 -1 -1 1
0 -2 -2 0
2 0 0 -2
0 -2 -2 0
![Page 49: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/49.jpg)
![Page 50: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/50.jpg)
![Page 51: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/51.jpg)
![Page 52: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/52.jpg)
![Page 53: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/53.jpg)
AutoassociativeRecurrent
![Page 54: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/54.jpg)
![Page 55: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/55.jpg)
To remember the pattern [1,-1,1,-1,1]
1 -1 1 -1 11 1 -1 1 -1 1-1 -1 1 -1 1 -11 1 -1 1 -1 1-1 -1 1 -1 1 -11 1 -1 1 -1 1
![Page 56: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/56.jpg)
Demo
![Page 57: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/57.jpg)
Complements of a vector also become attractors
![Page 58: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/58.jpg)
Complements of a vector also become attractors
Ex. Installing [1,-1, 1] [-1, 1, -1] also
“remembered”
![Page 59: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/59.jpg)
Complements of a vector also become attractors
Crosstalk
![Page 60: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/60.jpg)
![Page 61: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/61.jpg)
George Christos “Memory and Dreams”
![Page 62: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/62.jpg)
Ralph E. Hoffman models of schizophrenia
![Page 63: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/63.jpg)
Spurious Memories
![Page 64: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/64.jpg)
![Page 65: Focus on Unsupervised Learning. No teacher specifying right answer](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bf4/html5/thumbnails/65.jpg)