morphological multi-scale decomposition and efficient...
TRANSCRIPT
![Page 1: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/1.jpg)
Morphological Multi-scale Decomposition and efficient representations with Auto-Encoders
April 23th - September 28th
SupervisorsMVA supervisor:
Bastien PONCHON Internship Defense - september 21th 2018
![Page 2: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/2.jpg)
Agenda01. Introduction
02. Part-based Representation using Non-Negative Matrix Factorization
03. Part-based Representation using Auto-Encoders
04. Using a Deeper Architecture
05. Conclusion
2
![Page 3: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/3.jpg)
01 - Introduction
3
![Page 4: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/4.jpg)
Representation Learning and Part-Based representation
○
○
○atom images,
○4
![Page 5: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/5.jpg)
A few recaps on flat mathematical morphology
Dilation by a structuring element SE:
commutes with supremum.5
SE
![Page 6: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/6.jpg)
Erosion by a structuring element SE:
A few recaps on flat mathematical morphology
6
![Page 7: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/7.jpg)
Max-Approximation to Morphological Operators
7
![Page 8: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/8.jpg)
Motivation for Non-Negative and Sparse representation
8
![Page 9: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/9.jpg)
Objectives and Motivations of the Internship
○○
○
○ Universal approximator theorem:
9
![Page 10: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/10.jpg)
Evaluation and Data of the Proposed Models
○ Approximation error of the representation
○ Max-approximation error to the dilation
○ Sparsity of the encoding○ Classification Accuracy
10
![Page 11: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/11.jpg)
02 - Non-Negative Matrix Factorization
11
![Page 12: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/12.jpg)
General Presentation02 -
12
○ Matrix factorization algorithm:
data matrixdictionary matrixencoding matrix
○ separable factorial articulation family:●
●●
![Page 13: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/13.jpg)
Addition of sparsity constraints (Hoyer 2004)02 -
○
Sparsity measure of vector :
After each update of and in the NMF algorithm, the encodings and atoms are projected on the space verifying:
13
![Page 14: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/14.jpg)
Results - Sh = 0.602 -
14
Original images and reconstruction - Reconstruction error: 0.0109
Histogram of the encodings - Sparsity metric: 0.650
![Page 15: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/15.jpg)
15
Atom images of the representation
![Page 16: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/16.jpg)
Results - Max-Approximation to dilation02 -
16
Dilation of the original images by a disk of radius 1
Max-approximation to the dilation by a disk of radius 1
![Page 17: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/17.jpg)
03 - Part-Based Representation using Auto-Encoders
17
![Page 18: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/18.jpg)
Auto-encoder loss function, minimized during training:
Shallow Auto-Encoders 03 -
18
ReconstructionInput image Encoder Latent representation
Max-approximation
Decoder
The rows of are the atom images of the learned representation !
“Dilated” Decoder
![Page 19: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/19.jpg)
Enforcing the Sparsity of the Encoding03 -
Regularization of the auto-encoder:
Various choices for the sparsity-regularization function:
19
expected activation of each hidden unit fixed level
![Page 20: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/20.jpg)
Enforcing Non-Negativity of the Atoms of the Dictionary03 -
Two common approaches:○
○●●
20
Stronger decay of the negative weights
![Page 21: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/21.jpg)
Results - Reconstructions03 -
21
Original images
p=0.05, beta=0.001
p=0.01, beta=0.005
No Constraint
p=0.2, beta=0.001
p=0.1, beta=0.01
![Page 22: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/22.jpg)
Results - Encodings03 -
22
Original images
p=0.05, beta=0.001
p=0.01, beta=0.005
No Constraint
p=0.2, beta=0.001
p=0.1, beta=0.01
![Page 23: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/23.jpg)
Results - Atoms03 -
23
p=0.01, beta=0.005
No Constraint
p=0.1, beta=0.01
![Page 24: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/24.jpg)
Results - Max-approximations to dilation03 -
24
Original images
p=0.05, beta=0.001
p=0.01, beta=0.005
No Constraint
p=0.2, beta=0.001
p=0.01, beta=0.01
![Page 25: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/25.jpg)
25
![Page 26: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/26.jpg)
04 - Using a Deeper Architecture
26
![Page 27: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/27.jpg)
An Asymmetric Auto-Encoder04 -
Motivations:○○○○
27
ReconstructionInput image infoGANLatent
representation
Max-approximation
Decoder
“Dilated” Decoder“InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets”, Chen et al. 2016
○○
![Page 28: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/28.jpg)
Results - Reconstructions04 -
28
No Constraint
p=0.05, beta=0.005
p=0.01, beta=0.01
![Page 29: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/29.jpg)
Results - Encodings04 -
29
No Constraint
p=0.05, beta=0.005
p=0.01, beta=0.01
![Page 30: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/30.jpg)
Results - Atoms04 -
30
p=0.01, beta=0.01
No Constraint
p=0.05, beta=0.005
![Page 31: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/31.jpg)
Results - Max-Approximations to dilation04 -
31
No Constraint
p=0.05, beta=0.005
p=0.01, beta=0.01
![Page 32: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/32.jpg)
06 - Conclusion and Future Works
32
![Page 33: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/33.jpg)
Conclusion - Reconstructions06 -
33
NMF with Sparsity Constraint Sh=0.6
Sparse, Non-Negative Shallow AE with p=0.05, beta=0.001
Sparse, Non-Negative Asymmetric AE with p=0.05, beta=0.005
Original Images
![Page 34: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/34.jpg)
Conclusion - Encodings06 -
34
NMF with Sparsity Constraint Sh=0.6
Sparse, Non-Negative Shallow AE with p=0.05, beta=0.001
Sparse, Non-Negative Asymmetric AE with p=0.05, beta=0.005 -
![Page 35: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/35.jpg)
Conclusion - Atoms06 -
35
NMF with Sparsity Constraint Sh=0.6
Sparse, Non-Negative Asymmetric AE with p=0.05, beta=0.005
Sparse, Non-Negative Shallow AE with p=0.05, beta=0.001
![Page 36: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/36.jpg)
Conclusion - Max-Approximations to dilation06 -
36
NMF with Sparsity Constraint Sh=0.6
Sparse, Non-Negative Shallow AE with p=0.05, beta=0.001
Sparse, Non-Negative Asymmetric AE with p=0.05, beta=0.005 -
Dilation of Original Images
![Page 37: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/37.jpg)
Conclusion and possible improvements06 -
○
○
○
37
![Page 38: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/38.jpg)
05 - Multi-Scales Morphological Decompositions
38
![Page 39: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/39.jpg)
Additive Morphological Decomposition05 -
39
One of the considered Morphological Decomposition:○
○
○
○○
![Page 40: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/40.jpg)
40
![Page 41: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/41.jpg)
Positive Additive Decomposition Using Openings by Reconstruction05 -
41
![Page 42: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/42.jpg)
05 -
42
![Page 43: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/43.jpg)
Results05 -
43
![Page 44: Morphological Multi-scale Decomposition and efficient ...perso.telecom-paristech.fr/gonthier/pdf/DLWG/Morphological Multi-scale...infoGAN Latent representation Max-approximation Decoder](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f86076d2f15b865c01d8aaa/html5/thumbnails/44.jpg)
○
○
○atom images,
○
Representation (latent features)
Representation Learning and Part-Based representation
44
Image