d31 entity recognition results with auto- associative memories nicolas gourier inria prima team...
TRANSCRIPT
![Page 1: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/1.jpg)
D31 Entity RecognitionResults with Auto-
associative Memories
Nicolas GourierINRIAPRIMA TeamGRAVIR
Laboratory
CAVIAR Project
![Page 2: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/2.jpg)
Entity recognition
Can be performed either by local or global approaches
Local approaches Use information contained in the
neighboorhood of pixels
Global approaches Use the entire appearance of the
image
![Page 3: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/3.jpg)
Why a global approach ?
No landmarks have to be detected
No model has to be constructed
Can handle Low resolution Partial occlusions
=> Only the object has to be detected
![Page 4: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/4.jpg)
Existing Global Approaches
PCA, KDA,… [Pentland91] Sensitive to alignment Number of dimensions ?
Neural networks Number of cells in the hidden layer ? Recovery of prototypes of image
classes ?
=> Auto-associative Memories [Abdi94]
![Page 5: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/5.jpg)
Plan of the talk
1) Our approach
2) Results
3) Comparison with other techniques
![Page 6: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/6.jpg)
1. Normalized object imagette Grey scale face imagette normalized in
size and slant: 25x25 pixels
=> Computation time reduction=> Size and slant robustness
![Page 7: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/7.jpg)
1. Auto-associative memories
Linear auto-associative memory
Input patterns associated with themselves
Connection between input unitsPortion of an input =>Complete pattern
X’ = W.X X : Source image X’ : Output image W : Weights
![Page 8: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/8.jpg)
1. Hebbian learning rule
W = Xk.XkT
Faces not well discriminated [Valentin94]
![Page 9: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/9.jpg)
1. Widrow-Hoff learning rule (1)
Learned images are reconstructed
Other images are degraded [Valentin94]
![Page 10: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/10.jpg)
1. Widrow-Hoff learning rule (2)
Creation of prototypes
Eigenvalues egalization [Abdi & Valentin94]
We adapt Widrow-Hoff learning to entity recognition
![Page 11: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/11.jpg)
1. Entity Recognition
Compare the input image to all responses=> Score between 0 and 1
Winner-takes-all process
-> ½ videos for training,½ videos for test
![Page 12: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/12.jpg)
1. Training and Test
Training ->Test V
![Page 13: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/13.jpg)
2. Experiments
3 Experiments :
1) Classes 0 / 1 person Without training a 0 person class
2) Classes 0 / 1 person
3) Classes 0 / 1+ person
![Page 14: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/14.jpg)
2. Result of the first experiment (1)
![Page 15: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/15.jpg)
2. Result of the first experiment (2) Not sufficient for reliable
classification
0 person class imagettes have non-uniform variations in appearance
=> Learn a 0 person class from random images of the background
![Page 16: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/16.jpg)
2. Result of the second experiment
![Page 17: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/17.jpg)
2. Result of the third experiment
![Page 18: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/18.jpg)
2. Recall and precision
Experiment
Classes
11/0
21/0
31+/0
1st class recall
- 99 % 99 %
2nd class recall
- 68 % 70 %
1st class precision
- 95 % 93 %
2nd class precision
- 93 % 90 %
![Page 19: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/19.jpg)
3. Discussion
Training the 0 person class improves discrimination
Some 0 person class images are misclassified :
![Page 20: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/20.jpg)
3. Advantages Varying the size of the imagette do not
have much influence -> 25x25 pixels
Normalization + Classification is done at video-rate
Prototypes can be saved and reused
Can be adapted to entity recognition
![Page 21: D31 Entity Recognition Results with Auto- associative Memories Nicolas Gourier INRIA PRIMA Team GRAVIR Laboratory CAVIAR Project](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649eda5503460f94be9bd2/html5/thumbnails/21.jpg)
Conclusion
+ Invariance to scale, slant and alignment
+ Not disrupted by local changes
- Needs to train a non-person class
Adapted to the project Low resolution Changes of viewpoint Fast