![Page 1: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/1.jpg)
Barcelona June 2013 1
![Page 2: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/2.jpg)
II- Ground truth energies and Saliency Transform
B. Ravi Kiran Jean Serra
LIMG ESIEE Université Paris-Est
Barcelona June 2013 2
UPC / Pompeu Fabra Barcelona, June 13 2013
![Page 3: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/3.jpg)
Problem context
Barcelona June 2013 3
![Page 4: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/4.jpg)
A Problem: Inputs
Barcelona June 2013 4
![Page 5: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/5.jpg)
Ground truth: Evaluation of Hierarchies
Barcelona June 2013 5
![Page 6: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/6.jpg)
Problems
1. Given a hierarchy H and ground truth partition G find the partition in H closest to G.
1. Closest from H -> G
2. Closest from G -> H
2. Compare any hierarchy H with multiple ground truth partitions of the same image
3. Compare any two hierarchies H1, H2, with respect to a common ground truth partition G
6 Barcelona June 2013
![Page 7: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/7.jpg)
Hausdorff distance and associated problems
Y
X
i.e. smallest disc dilation of X that contains Y and of X to contain Y
- Global Measure - Large variations when object are asymmetric w.r.t each other
7 Barcelona June 2013
![Page 8: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/8.jpg)
Local Hausdorff distances
• Local measures: Each class S in H is assigned 2 radii: ,
• Both are h-increasing energies
• Local optimization to obtain a globally optimal solution
8
minimum radius of dilation of ground truth contour that covers the contour of S.
minimum radius of dilation of the contour of S to cover GT within S. Barcelona June 2013
![Page 9: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/9.jpg)
Example
Barcelona June 2013 9
![Page 10: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/10.jpg)
Energy at various levels of H
10 Barcelona June 2013
![Page 11: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/11.jpg)
Energy at various levels of H
11 Barcelona June 2013
![Page 12: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/12.jpg)
Problems
1. Given a hierarchy H and ground truth partition G find the partition in H closest to G.
1. Closest from H -> G
2. Closest from G -> H
2. Compare any hierarchy H with multiple ground truth partitions of the same image
3. Compare any two hierarchies H1, H2, with respect to a common ground truth partition G
12 Barcelona June 2013
![Page 13: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/13.jpg)
Optimal Cuts
Initial Image
GT2 13 Barcelona June 2013
![Page 14: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/14.jpg)
Optimal Cuts
Initial Image
GT7 14 Barcelona June 2013
![Page 15: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/15.jpg)
Problems
1. Given a hierarchy H and ground truth partition G find the partition in H closest to G.
1. Closest from H -> G
2. Closest from G -> H
2. Compare any hierarchy H with multiple ground truth partitions of the same image
3. Compare any two hierarchies H1, H2, with respect to a common ground truth partition G
15 Barcelona June 2013
![Page 16: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/16.jpg)
Composition of ground truths:
GT5_1 GT5_2 GT5 = GT5_1 + GT5_2
The distance function of the union(sum) is the inf of the distance functions 16 Barcelona June 2013
![Page 17: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/17.jpg)
Composition of ground truths:
GT5_1 GT5_2 GT5 = GT5_1 + GT5_2
17 Barcelona June 2013
![Page 18: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/18.jpg)
Problems
1. Given a hierarchy H and ground truth partition G find the partition in H closest to G.
1. Closest from H -> G
2. Closest from G -> H
2. Compare any hierarchy H with multiple ground truth partitions of the same image
3. Compare any two hierarchies H1, H2, with respect to a common ground truth partition G
18 Barcelona June 2013
![Page 19: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/19.jpg)
Global Precision-Recall Energies
Local dissimilarity measure Counterpart Global similarity measures
19
The two half distances yield two local and then two global energies: • Precision (P) : How close is on average the ground truth to the class (G->S) • Recall (R) : How close is on average the Class contour to the Ground truth (S->G)
Barcelona June 2013
![Page 20: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/20.jpg)
Comparing Hierarchies (saliencies) with Precision-recall similarity measures
UCM Cousty (floodings of watershed)
UCM random hierarchy Cousty random hierarchy
GT6 GT5 GT4 GT3 GT2 GT7 GT1 20 Barcelona June 2013
![Page 21: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/21.jpg)
Comparing Hierarchies (saliencies) with Precision-recall similarity measures
Image 25098
UCM UCM random
Cousty Cousty random
Precision energy
4.4 0.27 0.13 0.09
Recall energy
3.9 0.28 0.16 0.10
Integrals from PR equations expressed per 1000 pixels in the image
21 Barcelona June 2013
![Page 22: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/22.jpg)
Problem context
Barcelona June 2013 22
![Page 23: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/23.jpg)
Ground truth: Evaluation of Hierarchies
Barcelona June 2013 23
![Page 24: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/24.jpg)
A problem: Transforming hierarchies
Barcelona June 2013 24
![Page 25: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/25.jpg)
Representations of Hierarchies: Saliency function
Barcelona June 2013 25
Saliency Ultrametric contour Map (UCM)
![Page 26: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/26.jpg)
Introducing an external function
Barcelona June 2013 26
![Page 27: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/27.jpg)
Introducing an external function
Barcelona June 2013 27
![Page 28: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/28.jpg)
Introducing an external function
Barcelona June 2013 28
![Page 29: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/29.jpg)
Binary Class Opening
Barcelona June 2013 29
![Page 30: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/30.jpg)
Numerical (grayscale) class opening
Barcelona June 2013 30
![Page 31: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/31.jpg)
Example: Class Opening
Barcelona June 2013 31
![Page 32: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/32.jpg)
Properties of class opening I
Barcelona June 2013 32
![Page 33: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/33.jpg)
Properties of class opening II
Barcelona June 2013 33
![Page 34: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/34.jpg)
Saliency degeneracy
Barcelona June 2013 34
1 2 3 4 5 6 7 8 9 100
2
4
6
8
10
12
14
16
18
20
linear
power law x > 1 (compression)
power law x 0 < < 1(expansion)
Logarithmic
![Page 35: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/35.jpg)
Evaluating Hierarchies w.r.t Ground Truth
Barcelona June 2013 35
![Page 36: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/36.jpg)
Evaluating Hierarchies w.r.t Ground Truth
Barcelona June 2013 36
![Page 37: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/37.jpg)
Barcelona June 2013 37
![Page 38: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/38.jpg)
Composing two external functions
Barcelona June 2013 38
![Page 39: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/39.jpg)
Composing two external functions
Barcelona June 2013 39
![Page 40: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/40.jpg)
Generating Random Hierarchies
Barcelona June 2013 40
![Page 41: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/41.jpg)
Hierarchy Fusion: Matching hierarchies
Barcelona June 2013 41
![Page 42: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/42.jpg)
Hierarchy Fusion: Matching hierarchies
Barcelona June 2013 42
![Page 43: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/43.jpg)
Hierarchy Fusion: Matching hierarchies
Barcelona June 2013 43
![Page 44: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/44.jpg)
Hierarchy Fusion: Matching hierarchies
Barcelona June 2013 44
![Page 45: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/45.jpg)
Conclusion
Barcelona June 2013 45
http://www.esiee.fr/~kiranr/HierarchEvalGT.html
![Page 46: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/46.jpg)
Future work
Barcelona June 2013 46
![Page 47: Barcelona June 2013 1kiranr/Barca-2013-II.pdf · Barcelona June 2013 1 . II- Ground truth energies and Saliency Transform B. Ravi Kiran Jean Serra LIMG ESIEE Université Paris-Est](https://reader035.vdocuments.site/reader035/viewer/2022063008/5fbd3c581a7ee40c0b19e541/html5/thumbnails/47.jpg)
47 Barcelona June 2013