image operator learning - vision.ime.usp.br · huge advances in image processing, computer vision,...
TRANSCRIPT
![Page 1: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/1.jpg)
SIBGRAPI 2016 – TUTORIAL
Image Operator Learningand Applications
Igor S. MontagnerNina S. T. Hirata
Roberto Hirata Jr.
Department of Computer ScienceInstitute of Mathematics and Statistics
University of São Paulo (USP)
October / 2016
![Page 2: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/2.jpg)
Observed input
Expected output
Given pairs of images as the oneson the left...
Question:
– how to design an imageoperator Ψ that transformsthe input image to theexpected output image?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 2
![Page 3: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/3.jpg)
Observed input
Expected output
Ψ =?
Given pairs of images as the oneson the left...
Question:
– how to design an imageoperator Ψ that transformsthe input image to theexpected output image?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 3
![Page 4: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/4.jpg)
What about this transformation ?
Ψ = ?=⇒
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 4
![Page 5: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/5.jpg)
And this one?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 5
![Page 6: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/6.jpg)
This one?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 6
![Page 7: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/7.jpg)
Or this one?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 7
![Page 8: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/8.jpg)
Historical overview
1960s: Mathematical morphology was introduced by G.Matheron and J. Serra
1980s: Book by J. Serra (1982)Nonlinear filters (median, order statistic, stack), morphologicalfilters
1990s: E. R. Dougherty - Design of optimal morphologicalfilters
1994: Junior Barrera - Vision Lab created at IME/USPWe started working on designing morphological operators fromtraining images at that time !!
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 8
![Page 9: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/9.jpg)
Today (almost 25 years since then)
Huge advances in image processing, computer vision,machine learning, hardware and computing technologies
High-level × low-level processing
– Deep learning (high level image understanding)
– Image operator learning (low level processing)
The second is a building block for the first. Thus, veryimportant.
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 9
![Page 10: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/10.jpg)
Plan for this tutorial
1. Concepts and fundamentals
– you will know what an image operator is, and importantproperties that define a class of operators adequate for thetype of processing we are interested on
2. Optimal operators
– you will know how to characterize what a good imageoperator is
3. Learning image operators
– you will know how to learn image operators from trainingdata
4. Practice
– you will have the opportunity to do exercises and run code
5. Application examples
– you will see some application examples
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 10
![Page 11: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/11.jpg)
Plan for this tutorial
1. Concepts and fundamentals
– you will know what an image operator is, and importantproperties that define a class of operators adequate for thetype of processing we are interested on
2. Optimal operators
– you will know how to characterize what a good imageoperator is
3. Learning image operators
– you will know how to learn image operators from trainingdata
4. Practice
– you will have the opportunity to do exercises and run code
5. Application examples
– you will see some application examples
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 11
![Page 12: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/12.jpg)
Plan for this tutorial
1. Concepts and fundamentals
– you will know what an image operator is, and importantproperties that define a class of operators adequate for thetype of processing we are interested on
2. Optimal operators– you will know how to characterize what a good image
operator is
3. Learning image operators
– you will know how to learn image operators from trainingdata
4. Practice
– you will have the opportunity to do exercises and run code
5. Application examples
– you will see some application examples
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 12
![Page 13: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/13.jpg)
Plan for this tutorial
1. Concepts and fundamentals
– you will know what an image operator is, and importantproperties that define a class of operators adequate for thetype of processing we are interested on
2. Optimal operators– you will know how to characterize what a good image
operator is
3. Learning image operators– you will know how to learn image operators from training
data
4. Practice
– you will have the opportunity to do exercises and run code
5. Application examples
– you will see some application examples
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 13
![Page 14: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/14.jpg)
Plan for this tutorial
1. Concepts and fundamentals
– you will know what an image operator is, and importantproperties that define a class of operators adequate for thetype of processing we are interested on
2. Optimal operators– you will know how to characterize what a good image
operator is
3. Learning image operators– you will know how to learn image operators from training
data
4. Practice– you will have the opportunity to do exercises and run code
5. Application examples
– you will see some application examples
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 14
![Page 15: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/15.jpg)
Plan for this tutorial
1. Concepts and fundamentals
– you will know what an image operator is, and importantproperties that define a class of operators adequate for thetype of processing we are interested on
2. Optimal operators– you will know how to characterize what a good image
operator is
3. Learning image operators– you will know how to learn image operators from training
data
4. Practice– you will have the opportunity to do exercises and run code
5. Application examples– you will see some application examples
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 15
![Page 16: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/16.jpg)
Fundamentals on imageoperators
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 16
![Page 17: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/17.jpg)
How to model image transformations ?
Image domain
E (subset of Z2)
Image
f : E→ K (K is the set of gray levels)
KE set of all images defined on E with gray-levels in K
Image operator
Ψ : KE → KE
[Ψ(f )](p) value of the processed image at p
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 17
![Page 18: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/18.jpg)
Image operators (mathematical morphology view)
Morphological operators are built by composing basicelementary operators.
Examples of basic operators– erosion
– dilation
– hit-miss, ...
Structuring elements: parameter of the operatorsSimple shapes such as squares, discs, line segments, to locallyprobe an image and extract geometric structural information
Fundamentals: We will present concepts restricted to binaryimages, but they hold for gray-scale images too
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 18
![Page 19: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/19.jpg)
Binary image operators = set operators
Binary image
f : E→ {0, 1}
Binary image = set
p ∈ Sf ⇐⇒ f (p) = 1
Operations on images = operations on sets
Set relations and operations: ⊆, ∪, ∩, ·cSet translation: Ap = {a + p : a ∈ A}
Set reflection: A = {−a : a ∈ A}
Set interval: [A, B] = {X : A ⊆ X ⊆ B}
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 19
![Page 20: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/20.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)
S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 20
![Page 21: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/21.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 21
![Page 22: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/22.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 22
![Page 23: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/23.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 23
![Page 24: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/24.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 24
![Page 25: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/25.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 25
![Page 26: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/26.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 26
![Page 27: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/27.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 27
![Page 28: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/28.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 28
![Page 29: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/29.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 29
![Page 30: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/30.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 30
![Page 31: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/31.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 31
![Page 32: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/32.jpg)
Basic operator: erosion
εB(S) = {p ∈ E : Bp ⊆ S}
S εB(S)
S εB(S)
How does the erosion work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 32
![Page 33: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/33.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)
S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 33
![Page 34: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/34.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 34
![Page 35: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/35.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 35
![Page 36: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/36.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 36
![Page 37: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/37.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 37
![Page 38: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/38.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 38
![Page 39: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/39.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 39
![Page 40: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/40.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 40
![Page 41: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/41.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 41
![Page 42: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/42.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 42
![Page 43: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/43.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 43
![Page 44: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/44.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 44
![Page 45: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/45.jpg)
Basic operator: dilation
δB(S) = {p ∈ E : Bp ∩ S 6= ∅}
S δB(S)
S δB(S)
How does the dilation work ?
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 45
![Page 46: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/46.jpg)
Basic operator: erosion and dilation
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 46
![Page 47: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/47.jpg)
Another basic operator: hit-miss
H(A,Bc)(S) = {p ∈ E : Ap ⊆ S and Bcp ⊆ Sc} = εA(S) ∩ εBc(Sc)
A Bc
places where A hits S
places where Bc hits thebackground (or, equiva-lently, misses S)
places detected by the hit-miss operator
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 47
![Page 48: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/48.jpg)
Short break for an exercise
Exercise 1
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 48
![Page 49: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/49.jpg)
Solution to exercise 1 – structuring element A
A Bc
In red, positions x at which Ax fits in the foreground
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 49
![Page 50: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/50.jpg)
Solution to exercise 1 – structuring element Bc
A Bc
In red, positions x at which Bcx fits in the background
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 50
![Page 51: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/51.jpg)
Solution to exercise 1 – Intersection
A Bc
Result: the holes
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 51
![Page 52: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/52.jpg)
Many useful operators can be built by composing these andother simple operators
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 52
![Page 53: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/53.jpg)
Example of an operator: Contour detection
f − εB(f )
S εB(S) S − εB(S)
Which properties of this operator
are interesting ?
• Translation invariance• Local definition
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 53
![Page 54: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/54.jpg)
Example of an operator: Contour detection
f − εB(f )
S εB(S) S − εB(S)
Which properties of this operator
are interesting ?
• Translation invariance• Local definition
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 54
![Page 55: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/55.jpg)
Example of an operator: Contour detection
f − εB(f )
S εB(S) S − εB(S)
Which properties of this operator
are interesting ?
• Translation invariance• Local definition
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 55
![Page 56: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/56.jpg)
Translation invariance
[Ψ(f )]p = Ψ(fp)
Ψ(S)
Ψ(S)
[Ψ(S)]p
Ψ(S)
[Ψ(S)]p
Sp
Ψ(S)
[Ψ(S)]p
Sp
Ψ(Sp)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 56
![Page 57: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/57.jpg)
Translation invariance
[Ψ(f )]p = Ψ(fp)
Ψ(S)
Ψ(S)
[Ψ(S)]p
Ψ(S)
[Ψ(S)]p
Sp
Ψ(S)
[Ψ(S)]p
Sp
Ψ(Sp)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 57
![Page 58: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/58.jpg)
Translation invariance
[Ψ(f )]p = Ψ(fp)
Ψ(S)Ψ(S)
[Ψ(S)]p
Ψ(S)
[Ψ(S)]p
Sp
Ψ(S)
[Ψ(S)]p
Sp
Ψ(Sp)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 58
![Page 59: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/59.jpg)
Translation invariance
[Ψ(f )]p = Ψ(fp)
Ψ(S)Ψ(S)
[Ψ(S)]p
Ψ(S)
[Ψ(S)]p
Sp
Ψ(S)
[Ψ(S)]p
Sp
Ψ(Sp)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 59
![Page 60: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/60.jpg)
Local definition
Ψ is locally defined if there is a window W such that:
p ∈ Ψ(S)⇐⇒ p ∈ Ψ(S ∩W ′)
for every p ∈ E, S ∈ KE, and W ′ ⊇W
The red pixel is a contour point
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 60
![Page 61: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/61.jpg)
Local definition
Ψ is locally defined if there is a window W such that:
p ∈ Ψ(S)⇐⇒ p ∈ Ψ(S ∩W ′)
for every p ∈ E, S ∈ KE, and W ′ ⊇W
The red pixel is a contour point
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 61
![Page 62: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/62.jpg)
Local definition
Ψ is locally defined if there is a window W such that:
p ∈ Ψ(S)⇐⇒ p ∈ Ψ(S ∩W ′)
for every p ∈ E, S ∈ KE, and W ′ ⊇W
The red pixel is a contour point
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 62
![Page 63: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/63.jpg)
Local definition
Ψ is locally defined if there is a window W such that:
p ∈ Ψ(S)⇐⇒ p ∈ Ψ(S ∩W ′)
for every p ∈ E, S ∈ KE, and W ′ ⊇W
The red pixel is a contour point
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 63
![Page 64: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/64.jpg)
Window operator or W -operator
W -operator: translation invariance + local definition
There is a local function ψ that uniquely characterizes Ψ
S Ψ(S)
zz
Ψ(S)(z) = ψ( )
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 64
![Page 65: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/65.jpg)
Back to contour detection
Contour detection operator: S − εB(S)– translation-invariant– locally defined
Local function:To decide if a pixel is a(n internal) contour point ornot, it suffices to check if it is in the foreground and ifthere is at least one pixel adjacent to it in thebackground.
Considering 4-adjacency, a cross-window is sufficient
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 65
![Page 66: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/66.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input Output
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 66
![Page 67: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/67.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input Output
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 67
![Page 68: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/68.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 68
![Page 69: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/69.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 69
![Page 70: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/70.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 70
![Page 71: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/71.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 71
![Page 72: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/72.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 72
![Page 73: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/73.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 73
![Page 74: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/74.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 74
![Page 75: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/75.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 75
![Page 76: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/76.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 76
![Page 77: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/77.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 77
![Page 78: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/78.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 78
![Page 79: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/79.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 79
![Page 80: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/80.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 80
![Page 81: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/81.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 81
![Page 82: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/82.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 82
![Page 83: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/83.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 83
![Page 84: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/84.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 84
![Page 85: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/85.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 85
![Page 86: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/86.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 86
![Page 87: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/87.jpg)
Local function of the contour detection operator
x1
x1
x2x2 x3x3 x4x4 x5
x5
window array
Input OutputInput Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00101
Input Output
00101
Input Output
01001
Input Output
01001
Input Output
00000
Input Output
00000
Input Output
00011
Input Output
00011
Input Output
00111
Input Output
00111
Input Output
11111
Input Output
11111
Input Output
01101
Input Output
01101
Input Output
01001
Input Output
01001
Input Output
and so on
X ψ(X)00000 000001 000010 000011 000100 100101 100110 100111 1· · · · · ·
01001 0· · · · · ·
01101 1· · · · · ·
11011 0· · · · · ·
11110 111111 0
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 87
![Page 88: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/88.jpg)
Short break for anotherexercise
Exercise 2
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 88
![Page 89: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/89.jpg)
Solution to exercise 2 – structuring element A
A Bc
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 89
![Page 90: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/90.jpg)
Solution to exercise 2 – structuring element Bc
A Bc
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 90
![Page 91: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/91.jpg)
Solution to exercise 2 – Intersection
A Bc
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 91
![Page 92: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/92.jpg)
Representation of an operator
How to represent an image operator ?
If Ψ is a W -operator it suffices to know ψ : {0, 1}W → {0, 1}.
Kernel of ΨK(Ψ) = {X ⊆W : ψ(X) = 1}
• What is the importance of the kernel ?
• Do we need to explicitly represent the kernel ?
Kernels are related to a canonical
representation of W -operators
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 92
![Page 93: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/93.jpg)
Representation of an operator
How to represent an image operator ?
If Ψ is a W -operator it suffices to know ψ : {0, 1}W → {0, 1}.
Kernel of ΨK(Ψ) = {X ⊆W : ψ(X) = 1}
• What is the importance of the kernel ?
• Do we need to explicitly represent the kernel ?
Kernels are related to a canonical
representation of W -operators
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 93
![Page 94: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/94.jpg)
Representation of operators
Hit-miss in an important piece in the representation ofW -operators
A hit-miss operator is equivalent to an intervalX is detected by hit-miss operator H(A,Bc) ⇐⇒ X ∈ [A, B]
W A Bc B
The window images detected by H(A,Bc) or in [A, B] are thesame:
(all templates that correspond top contour points)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 94
![Page 95: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/95.jpg)
Decomposition theoremExample: intervals for the contour detection operator
, , , ,,,,{ }[[[[ ]]]]
Interval operator:
Λ(A,B)(X) = 1⇐⇒ X ∈ [A, B]
Basis of Ψ: B(Ψ) = maximal intervals in K(Ψ)
Minimal decomposition theorem
ψ(X) = max[A,B]∈B(Ψ)
Λ(A,B)(X)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 95
![Page 96: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/96.jpg)
What would be the basis for the extreme point case ?
left endpoint
right endpoint
bottom endpoint
top endpoint
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 96
![Page 97: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/97.jpg)
What would be the basis for the extreme point case ?
left endpoint
right endpoint
bottom endpoint
top endpoint
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 97
![Page 98: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/98.jpg)
Summary of the fundamentals
You acquired knowledge on
– image operators
– W -operators, a broad class of image operators
– characterization by local functions
– basis representation: supremum of interval operators
– geometrical interpretation
We should also mention:
– binary case: equivalent to Boolean functions
– Similar results hold for gray-scale image operators
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 98
![Page 99: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/99.jpg)
Learning image operators
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 99
![Page 100: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/100.jpg)
Goal of image operator learning
Given observed images f and respective expectedtransformation g, we would like to find Ψ such thatΨ(f ) is a good approximation of g
MAE (mean absolute error)
MAE〈Ψ〉 = E[|[Ψ(f )](p) − g(p)|
]
Empirical MAE
Err =1|E|
∑p∈E
∣∣∣[Ψ(f )](p) − g(p)∣∣∣
Average of pixel-wise absolute difference
Binary images: pixel error rateSIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 100
![Page 101: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/101.jpg)
Optimal W -operators
Window: W
Input-output pairs of images: (f , g)
Observations: (X , y)X is a window imagey is the corresponding output value
Optimal operator: (in the case of binary operators)
ψ(X) =
1, if P(1|X) > P(0|X),
0, if P(0|X) > P(1|X),
0 or 1 if P(0|X) = P(1|X) = 0.5
Probabilities P(y|X) can be estimated from training images
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 101
![Page 102: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/102.jpg)
Counting occurrences of (X , y)
(0
0000
, 0)(0
0011
, 0)(0
0101
, 1)(0
1001
, 0)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 102
![Page 103: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/103.jpg)
Counting occurrences of (X , y)
(0
0000
, 0)
(0
0011
, 0)(0
0101
, 1)(0
1001
, 0)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 103
![Page 104: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/104.jpg)
Counting occurrences of (X , y)
(0
0000
, 0)
(0
0011
, 0)
(0
0101
, 1)(0
1001
, 0)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 104
![Page 105: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/105.jpg)
Counting occurrences of (X , y)
(0
0000
, 0)(0
0011
, 0)
(0
0101
, 1)
(0
1001
, 0)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 105
![Page 106: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/106.jpg)
Counting occurrences of (X , y)
(0
0000
, 0)(0
0011
, 0)(0
0101
, 1)
(0
1001
, 0)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 106
![Page 107: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/107.jpg)
Example
X ψ(X)
27 09 03 30 77 10 30 80 11
00
0/110111
Optimal decision
ψ(X) =
0, if p(0|X) > p(1|X),
1, if p(0|X) < p(1|X),
×, if p(0|X) = p(1|X).
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 107
![Page 108: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/108.jpg)
Short break for a new exerciseExercise 3
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 108
![Page 109: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/109.jpg)
Solution to exercise 3
2 8
2
10 2
440
36 0
10
9 0
11
2 12
2
X0 1
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 109
![Page 110: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/110.jpg)
Image operator designin the framework of
Machine Learning
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 110
![Page 111: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/111.jpg)
Machine learning – Supervised classification
http://www.nltk.org/book/ch06.html
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 111
![Page 112: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/112.jpg)
Learning the local function
S Ψ(S)
zz
Ψ(S)(z) = ψ( )
The local function can be understood as a classifier
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 112
![Page 113: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/113.jpg)
Training samples
Positive example
Negative example
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 113
![Page 114: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/114.jpg)
Training samples
Positive example
Negative example
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 114
![Page 115: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/115.jpg)
Training samples
Positive example
Negative example
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 115
![Page 116: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/116.jpg)
Feature extraction (to form the training set)
(0
0000
, 0)(0
0011
, 0)(0
0101
, 1)(0
1001
, 0)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 116
![Page 117: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/117.jpg)
Feature extraction (to form the training set)
(0
0000
, 0)
(0
0011
, 0)(0
0101
, 1)(0
1001
, 0)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 117
![Page 118: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/118.jpg)
Feature extraction (to form the training set)
(0
0000
, 0)
(0
0011
, 0)
(0
0101
, 1)(0
1001
, 0)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 118
![Page 119: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/119.jpg)
Feature extraction (to form the training set)
(0
0000
, 0)(0
0011
, 0)
(0
0101
, 1)
(0
1001
, 0)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 119
![Page 120: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/120.jpg)
Feature extraction (to form the training set)
(0
0000
, 0)(0
0011
, 0)(0
0101
, 1)
(0
1001
, 0)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 120
![Page 121: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/121.jpg)
Overview
Training W−operator
S
I
Window
W
• W defines the feature space
• Training data is extracted from input-output images
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 121
![Page 122: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/122.jpg)
The problem is not solved yet ...
Challenge: tradeoff between window size and generalizationerror
0.01 0.012 0.014 0.016 0.018
0.02 0.022 0.024 0.026 0.028
0.03
0 20 40 60 80 100 120 140
MA
E
Window size
1 training image5 training images
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 122
![Page 123: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/123.jpg)
Two-level training
1 2 3W W W
same target pixel
distinct neighborhood
three results
pattern for the secondlevel classifier
target pixel
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 123
![Page 124: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/124.jpg)
Input
=⇒
Results of 7operators
=⇒Superimposed
results(darker = less votes)
⇓
Combinationresult
(better than the
individual ones)
![Page 125: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/125.jpg)
![Page 126: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/126.jpg)
Why two-level training scheme is good?
Windows in the first level training
Their union is a larger window
Two-level training (indirectly) considers a larger neighborhood,without increasing window size.
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 126
![Page 127: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/127.jpg)
Why two-level training scheme is good?
Windows in the first level training
Their union is a larger window
Two-level training (indirectly) considers a larger neighborhood,without increasing window size.
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 127
![Page 128: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/128.jpg)
Application examples
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 128
![Page 129: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/129.jpg)
Application examples
Input Expected output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 129
![Page 130: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/130.jpg)
Application examples
Input Test output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 130
![Page 131: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/131.jpg)
Application examples
Input Expected output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 131
![Page 132: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/132.jpg)
Application examples
Input Test output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 132
![Page 133: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/133.jpg)
Application examples
Input Expected output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 133
![Page 134: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/134.jpg)
Application examples
Input Test output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 134
![Page 135: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/135.jpg)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 135
![Page 136: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/136.jpg)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 136
![Page 137: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/137.jpg)
Input Expected output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 137
![Page 138: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/138.jpg)
Input Test output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 138
![Page 139: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/139.jpg)
Input Expected output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 139
![Page 140: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/140.jpg)
Input Test output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 140
![Page 141: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/141.jpg)
Input Expected output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 141
![Page 142: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/142.jpg)
Input Test output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 142
![Page 143: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/143.jpg)
Input Expected output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 143
![Page 144: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/144.jpg)
Input Test output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 144
![Page 145: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/145.jpg)
Input Expected output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 145
![Page 146: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/146.jpg)
Input Test output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 146
![Page 147: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/147.jpg)
Input Expected output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 147
![Page 148: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/148.jpg)
Input Test output
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 148
![Page 149: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/149.jpg)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 149
![Page 150: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/150.jpg)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 150
![Page 151: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/151.jpg)
Handwritten musical scores
Staff line removal:
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 151
![Page 152: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/152.jpg)
Synthetically generated musical scores
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 152
![Page 153: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/153.jpg)
PRACTICAL SESSION
TRIOSLibLaboratory session exercises 1 to 4
http://bit.ly/2dtL5en
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 153
![Page 154: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/154.jpg)
TRIOSlib
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 154
![Page 155: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/155.jpg)
TRIOSlib: research library containing implementations ofmany WOperator learning techniques.
– Integrates with Numpy, Scipy, scikit-learn.
– Extensible design: new techniques implemented withoutchanging the library.
– Error measuring, persistence on disk, parallelimplementations, ...
– Used in many published papers.
Available at https://sourceforge.net/projects/trioslib andinstallable using pip and conda.
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 155
![Page 156: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/156.jpg)
Important classes:
– trios.Imageset: list of triples (input,output,mask).mask=None means that every pixel is processed.
– trios.WOperator: executes learning process;– trios.FeatureExtractor:
– raw pixels (RAWFeatureExtractor andRAWBitFeatureExtractor)
– two-level patterns (CombinationPattern)– aperture
– trios.Classifier:– Boolean function minimization (ISI)– models from scikit-learn (SKClassifier)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 156
![Page 157: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/157.jpg)
Recent advances
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 157
![Page 158: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/158.jpg)
Window determination
Empirical approach: Based on user experience; trial and error
Feature selection based methods
Window points ∼ features / Window design ∼ feature selection
– Interaction information: C. S. Santos, N. S. Hirata, and R. Hirata Jr., “An information
theory framework for two-stage binary image operator design,” Pattern Recognition Letters, 2010
– RELIEF (feature selection algorithm): I. Montagner, R. Hirata Jr., and N. S. T.
Hirata, Ä machine learning based method for staff removal,” ICPR 2014
Measures to rank windows: discriminative power of windows
– Entropy based ranking: M. Dornelles and N. S. T. Hirata, “Selection of Windows for
W-Operator Combination from Entropy Based Ranking,” SIBGRAPI 2015
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 158
![Page 159: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/159.jpg)
Operator selection for combination
– Given a domain window W defined as a m × n = Nrectangle, there are 2N possible subwindows.
– Train a very strongly regularized sparse combinationusing all possible windows.
– Fast algorithms for linear sparse models(LASSO, L1-SVM).
Iterative solution is possible by checking optimality conditionsand random sampling of the operator space.
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 159
![Page 160: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/160.jpg)
Processing gray scale images
n = |W |
There are:
– 22nbinary W-operators
– kkngray-level W-operators
Computational cost (memory, processing) and statisticalprecision are challenging issues
Some possible approaches:
– constrain operator space: two-level, stack filters, aperture
– reduce window image variability
Some details next
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 160
![Page 161: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/161.jpg)
Gray-scale from binary
Threshold decomposition
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 161
![Page 162: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/162.jpg)
Example of stack filter
Gray-scale median can be computed as the sum of themedians of the cross-sections
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 162
![Page 163: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/163.jpg)
Test image and result
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 163
![Page 164: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/164.jpg)
Aperture Operators
Class of operators to transform grayscale images.
Number of possible binary operators: 22|W |.
Number of possible grayscale operators: KK |W |.
Now we have to restrict W and K.
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 164
![Page 165: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/165.jpg)
Aperture Operators
Besides translation invariance in the domain
o xt
h(x)
W
−t
xo
h (x)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 165
![Page 166: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/166.jpg)
Aperture Operators
we add translation invariance in grayscale
−t
Wu
−z
x
h (x)
o
o
u−z
| K
x
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 166
![Page 167: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/167.jpg)
Aperture Operators
Translation invariance in the domain and in the range, locallydefined by a window W and a range K = {−k, . . . , 0, . . . , k}
Ψ(f )(t) = ζO(ut) +ψ(ut−ζI(ut)|K)
where ζO and ζI are functions defined from KW to N .
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 167
![Page 168: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/168.jpg)
Kernel approximation
Main issue in training is generalization. Kernel SVMs offergood generalization, but require building a Gram matrixK ∈ RM×M . In WOperator learning, M >> 100, 000.
Nyström method:
– approximate K using a small set of m examples, m << M.
– works for any kernel;
– exact solution for m = M.
More details: Friday 14:00-15:45 at TS10!
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 168
![Page 169: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/169.jpg)
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 169
![Page 170: Image Operator Learning - vision.ime.usp.br · Huge advances in image processing, computer vision, machine learning, hardware and computing technologies High-level low-level processing](https://reader035.vdocuments.site/reader035/viewer/2022062604/5fbfa9b381d2b507b4769ad6/html5/thumbnails/170.jpg)
Trios - training image operators from samples
Available: https://sourceforge.net/projects/trioslib
Igor S. Montagner [email protected] S. T. Hirata [email protected]
Roberto Hirata Jr. [email protected]
SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 170