self organizing maps a visualization technique with data dimension reduction
DESCRIPTION
Self organizing maps A visualization technique with data dimension reduction Juan López González University of Oviedo Inverted CERN School of Computing, 24-25 February 2014. General overview. Lecture 1 Machine learning Introduction Definition Problems Techniques. Lecture 2 - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/1.jpg)
Self organizing maps
1 iCSC2014, Juan López González, University of Oviedo
Self organizing mapsA visualization technique with data dimension
reduction
Juan López GonzálezUniversity of Oviedo
Inverted CERN School of Computing, 24-25 February 2014
![Page 2: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/2.jpg)
Self organizing maps
2 iCSC2014, Juan López González, University of Oviedo
General overview
Lecture 1 Machine learning
Introduction Definition Problems Techniques
Lecture 2 ANN introduction SOM Simulation SOM based models
![Page 3: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/3.jpg)
Self organizing maps
3 iCSC2014, Juan López González, University of Oviedo
LECTURE 1
Self organizing maps.
A visualization technique with data dimension reduction.
![Page 4: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/4.jpg)
Self organizing maps
4 iCSC2014, Juan López González, University of Oviedo
1. Artificial neural networks (ANN)
![Page 5: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/5.jpg)
Self organizing maps
5 iCSC2014, Juan López González, University of Oviedo
1.1. Introduction
![Page 6: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/6.jpg)
Self organizing maps
6 iCSC2014, Juan López González, University of Oviedo
1.2.1. Feedforward NN1.2.2. Recurrent NN1.2.3. Self organizing NN1.2.4. Others
1.2. Types
![Page 7: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/7.jpg)
Self organizing maps
7 iCSC2014, Juan López González, University of Oviedo
1.2.1. Feedforward NN
![Page 8: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/8.jpg)
Self organizing maps
8 iCSC2014, Juan López González, University of Oviedo
Single layer feedforward
1.2.1. Feedforward NN
![Page 9: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/9.jpg)
Self organizing maps
9 iCSC2014, Juan López González, University of Oviedo
Multi-layer feedforward
1.2.1. Feedforward NN
Supervised learning Backpropagation learning algorithm
![Page 10: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/10.jpg)
Self organizing maps
10 iCSC2014, Juan López González, University of Oviedo
1.2.2. Recurrent neural networks
![Page 11: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/11.jpg)
Self organizing maps
11 iCSC2014, Juan López González, University of Oviedo
Elman networks ‘Context units’ Maintain state
1.2.2. Recurrent neural networks
![Page 12: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/12.jpg)
Self organizing maps
12 iCSC2014, Juan López González, University of Oviedo
Hopefield network Symmetric connections Associative memory
1.2.2. Recurrent neural networks
![Page 13: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/13.jpg)
Self organizing maps
13 iCSC2014, Juan López González, University of Oviedo
Modular neural networks The human brain is not a massive network
but a collection of small networks
1.2.2. Recurrent neural networks
![Page 14: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/14.jpg)
Self organizing maps
14 iCSC2014, Juan López González, University of Oviedo
Self-organizing networks A set of neurons learn to map points in
an input space to coordinates in an output space
1.2.3. Self-organizing networks
![Page 15: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/15.jpg)
Self organizing maps
15 iCSC2014, Juan López González, University of Oviedo
Holographic associative memory
Instantaneously trained networks
Learning vector quantization
Neuro-fuzzy networks
…
1.2.4. Others
![Page 16: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/16.jpg)
Self organizing maps
16 iCSC2014, Juan López González, University of Oviedo
2.1. Motivation2.2. Goal2.3. Main properties2.4. Elements2.5. Algorithm
2. Self-organizing maps
![Page 17: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/17.jpg)
Self organizing maps
17 iCSC2014, Juan López González, University of Oviedo
2.1. Motivation
Topographic maps Different sensory inputs (motor, visual,
auditory…) are mapped in areas of the cerebral cortex in an orderly fashion
![Page 18: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/18.jpg)
Self organizing maps
18 iCSC2014, Juan López González, University of Oviedo
2.1. Motivation
“The spatial location of an output neuron in a topographic map corresponds to a particular domain or feature drawn from the input space”
Auditory cortical fieldsMotor-somatotopic maps
![Page 19: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/19.jpg)
Self organizing maps
19 iCSC2014, Juan López González, University of Oviedo
2.2. Goal
Transform incoming signal of arbitrary dimension into a 1-2-3 dimensional discrete map in a topologically ordered fashion
![Page 20: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/20.jpg)
Self organizing maps
20 iCSC2014, Juan López González, University of Oviedo
2.3. Main properties
Transform continuos input space to discrete output space Dimension reduction
winner-takes-all neuron
Ordered feature map
Input with similar characteristics produce similar ouput
![Page 21: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/21.jpg)
Self organizing maps
21 iCSC2014, Juan López González, University of Oviedo
2.3.1 Dimension reduction
Curse of dimensionality (Richard E. Bellman) The amount of data needed grows exponentially with the
dimensionality
Types Feature extraction
Reduce input data (features vector)
Feature selection Select subset (remove redundant and irrelevant data)
![Page 22: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/22.jpg)
Self organizing maps
22 iCSC2014, Juan López González, University of Oviedo
2.4. Elements
…of machine learning A pattern exists We don’t know how to solve it mathematically A lot of data
(a1,b1,..,n1), (a2,b2,..,n2) … (aN,bN,..,nN)
![Page 23: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/23.jpg)
Self organizing maps
23 iCSC2014, Juan López González, University of Oviedo
2.4. Elements
Lattice of neurons Size? Weights
![Page 24: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/24.jpg)
Self organizing maps
24 iCSC2014, Juan López González, University of Oviedo
2.4. Elements
Learning rate Neighborhood function
Learning rate function
![Page 25: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/25.jpg)
Self organizing maps
25 iCSC2014, Juan López González, University of Oviedo
2.5. Algorithm
Initialization Input data preprocessing
Normalizing Discrete-continuous variables?
Weight initialization Random weights
![Page 26: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/26.jpg)
Self organizing maps
26 iCSC2014, Juan López González, University of Oviedo
2.5. Algorithm (2)
Sampling Take sample from input space
Matching Find BMU: i.e. min of
Update weights i.e.
![Page 27: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/27.jpg)
Self organizing maps
27 iCSC2014, Juan López González, University of Oviedo
3. Practical exercise
![Page 28: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/28.jpg)
Self organizing maps
28 iCSC2014, Juan López González, University of Oviedo
4.1. Digit recognition4.2. Finish phonetics4.3. Semantic map of word context
4. Map examples
![Page 29: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/29.jpg)
Self organizing maps
29 iCSC2014, Juan López González, University of Oviedo
4.1. Digit recognition
![Page 30: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/30.jpg)
Self organizing maps
30 iCSC2014, Juan López González, University of Oviedo
4.2. Finnish phonetics
![Page 31: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/31.jpg)
Self organizing maps
31 iCSC2014, Juan López González, University of Oviedo
4.3. Semantic map of word context
![Page 32: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/32.jpg)
Self organizing maps
32 iCSC2014, Juan López González, University of Oviedo
5.1. TASOM5.2. GSOM5.3. MuSOM
5. Other SOM based models
![Page 33: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/33.jpg)
Self organizing maps
33 iCSC2014, Juan López González, University of Oviedo
5.1. TASOM
Time adaptative self-organizing maps Deals with non-stationary input distributions Adaptative learning rates: n(w,x) Adaptative neighborhood rates: T(w,x)
![Page 34: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/34.jpg)
Self organizing maps
34 iCSC2014, Juan López González, University of Oviedo
5.2. GSOM
Growing self-organizing maps Deals with identifying sizes for SOMs
Spread factor New nodes in boundaries
Good when unknown clusters
![Page 35: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/35.jpg)
Self organizing maps
35 iCSC2014, Juan López González, University of Oviedo
5.3. MuSOM
Multimodal SOM High level classification from
sensory integration
![Page 36: Self organizing maps A visualization technique with data dimension reduction](https://reader036.vdocuments.site/reader036/viewer/2022062410/5681637c550346895dd45abc/html5/thumbnails/36.jpg)
Self organizing maps
36 iCSC2014, Juan López González, University of Oviedo
Q & A