![Page 1: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/1.jpg)
Computer Vision : CISC 4/689
Gradients and edges
• Points of sharp change in an image are interesting:– change in reflectance
– change in object
– change in illumination
– noise
• Sometimes called edge points
• General strategy– determine image gradient
– now mark points where gradient magnitude is particularly large wrt neighbours (ideally, curves of such points).
![Page 2: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/2.jpg)
Computer Vision : CISC 4/689
The Gradient and Edges
• Consider image intensities as a 2-D height function I(x, y). Then the image gradient is the vector field defined by:
• Definition of an edge– Line segment separating regions of contrasting intensity– Location: Where gradient magnitude is high – Direction: Orthogonal to the gradient
![Page 3: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/3.jpg)
Computer Vision : CISC 4/689
Edge Causes
• Depth discontinuity
• Surface orientation discontinuity
• Reflectance discontinuity (i.e., change in surface material properties)
• Illumination discontinuity (e.g., shadow)
![Page 4: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/4.jpg)
Computer Vision : CISC 4/689
Edge Detection• An edge point can be regarded as a point in an image
where a discontinuity (in gradient) occurs across some line. A discontinuity may be classified as one of five types
• Searching for Edges:– Filter: Smooth image– Enhance: Apply numerical derivative approximation– Detect: Threshold to find strong edges– Localize/analyze: Reject spurious edges, include
weak but justified edges
Gradient Discontinuity -- where the gradient of the pixel values changes across a line. This type of discontinuity can be classed as roof edges ramp edges convex edges concave edges by noting the sign of the component of the gradient perpendicular to the edge on either side of the edge. Ramp edges have the same signs in the gradient components on either side of the discontinuity, while roof edges have opposite signs in the gradient components.
A Jump or Step Discontinuity -- where pixel values themselves change suddenly across some line. A Bar Discontinuity -- where pixel values rapidly increase then decrease again (or vice versa) across some line.
Source: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/MARSHALL/node28.html
![Page 5: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/5.jpg)
Computer Vision : CISC 4/689
Step edge detection: First Derivative Operators
• Method: Differentiate and find extrema
• Examples– Sobel operator (Matlab: edge(I, ‘sobel’))
– Prewitt, Roberts cross
– Derivative of Gaussian
-1-2-1
000
121
-101
-202
-101
Sobel x Sobel y
Book uses thisformat
![Page 6: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/6.jpg)
Computer Vision : CISC 4/689
Sobel Edge Filtering Example
1 0 -1
2 0 -2
1 0 -1
0 0 2 2
0 0 2 2
0 0 2 2
0 0 2 2
Rotate
10-1
20-2
10-1
![Page 7: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/7.jpg)
Computer Vision : CISC 4/689
Step 1
0
0
1
2
2
1
2
2
22
20
20
22 0
0
0
0
0
2
2
2
2
20
20
20
20
00-1
00-2
10-1
10-1
20-2
10-1
![Page 8: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/8.jpg)
Computer Vision : CISC 4/689
Step 2
0
0
1
2
2
2
3
2
22
20
20
22 60
0
0
0
0
2
2
2
2
20
20
20
20
200
400
10-1
10-1
20-2
10-1
![Page 9: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/9.jpg)
Computer Vision : CISC 4/689
Step 3
0
0
1
2
2
2
3
2
30
20
20
30 6 60
0
0
0
0
2
2
2
2
20
20
20
20
200
400
10-1
10-1
20-2
10-1
![Page 10: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/10.jpg)
Computer Vision : CISC 4/689
Step 4
0
0
0
0
2
2
3
2
30
20
20
30 6 6 -60
0
0
0
0
2
2
2
2
20
20
20
20
10-2
20-4
10-1
10-1
20-2
10-1
edgeeffectfrom zero-padding
![Page 11: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/11.jpg)
Computer Vision : CISC 4/689
Sobel Edge Filtering Example: Result
6
8
8
6
6
8
8
6
-80
-60
-80
-60
(pad with zeroes again, the boundary)and then we threshold…
![Page 12: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/12.jpg)
Computer Vision : CISC 4/689
Sobel Edge Detection: Gradient Approximation
Horizontal diff. Vertical diff.
-1-2-1
000
121
-101
-202
-101
Note anisotropy of edge finding
![Page 13: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/13.jpg)
Computer Vision : CISC 4/689
Sobel
• These can then be combined together to find the absolute magnitude of the gradient at each point and the orientation of that gradient. The gradient magnitude is given by:
• an approximate magnitude is computed using:
which is much faster to compute.
• The angle of orientation of the edge (relative to the pixel grid) giving rise to the spatial gradient is given by:
In this case, orientation 0 is taken to mean that the direction of maximum contrast from black to white runs from left to right on the image, and other angles are measured anti-clockwise from this.
![Page 14: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/14.jpg)
Computer Vision : CISC 4/689
Derivative of Gaussian
![Page 15: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/15.jpg)
Computer Vision : CISC 4/689
Smoothing and Differentiation
• Issue: noise– smooth before differentiation
– two convolutions: to smooth, then differentiate?
– actually, no - we can use a derivative of Gaussian filter
• because differentiation is convolution, and convolution is associative
![Page 16: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/16.jpg)
Computer Vision : CISC 4/689
The Laplacian of Gaussian
• Another way to detect an extremal first derivative is to look for a zero second derivative– the Laplacian
• Bad idea to apply a Laplacian without smoothing– smooth with Gaussian, apply
Laplacian
– this is the same as filtering with a Laplacian of Gaussian filter
• Now mark the zero points where there is a sufficiently large (first) derivative, and enough contrast
![Page 17: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/17.jpg)
Computer Vision : CISC 4/689
Marr-Hildreth operator
• The Laplacian is linear and rotationally symmetric. Thus, we search for the zero crossings of the image that is first smoothed with a Gaussian mask and then the second derivative is calculated; or we can convolve the image with the Laplacian of the Gaussian, also known as the LoG operator;
• This defines the Marr-Hildreth operator.
• One can also get a shape similar to G'' by taking the difference of two Gaussians having different standard deviations. A ratio of standard deviations of 1:1.6 will give a close approximation to .This is known as the DoG operator (Difference of Gaussians), or the Mexican Hat Operator.
• Still sensitive to noise.
![Page 18: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/18.jpg)
Computer Vision : CISC 4/689
Step edge detection: 2nd-Derivative Operators
• Method: 2nd derivative is 0 for 1st-derivative extrema, so find “zero-crossings”– Laplacian
Isotropic (finds edges regardless of orientation.
Three commonly used discrete approximations to the Laplacian filter. (Note, we have defined the Laplacian using a negative peak because this is more common, however, it is equally valid to use the opposite sign convention.) Source: http://www.cee.hw.ac.uk/hipr/html/log.html
![Page 19: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/19.jpg)
Computer Vision : CISC 4/689
Laplacian of Gaussian
• Matlab: fspecial(‘log’,…)
Below: Discrete approximation to LoG function with Gaussian 1.4
![Page 20: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/20.jpg)
Computer Vision : CISC 4/689
Sobel vs. LoG Edge Detection:Matlab Automatic Thresholds
Sobel LoG
![Page 21: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/21.jpg)
Computer Vision : CISC 4/689
There are three major issues: 1) The gradient magnitude at different scales is different; which should we choose? 2) The gradient magnitude is large along thick trail (for 3rd fig); how do we identify the significant points? 3) How do we link the relevant points up into curves?
= 1 = 2
![Page 22: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/22.jpg)
Computer Vision : CISC 4/689
We wish to mark points along the curve where the magnitude is biggest.We can do this by looking for a maximum along a slice normal to the curve(non-maximum suppression). These points should form a curve. There arethen two algorithmic issues: at which point is the maximum, and where is thenext one?
![Page 23: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/23.jpg)
Computer Vision : CISC 4/689
Non-maximumsuppression
At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values.
![Page 24: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/24.jpg)
Computer Vision : CISC 4/689
Predictingthe nextedge point
Assume the marked point is an edge point. Then we construct the tangent (along) to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either r or s).
![Page 25: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/25.jpg)
Computer Vision : CISC 4/689
Remaining issues
• Check that maximum value of gradient value is sufficiently large– drop-outs? use hysteresis
• use a high threshold to start edge curves and a low threshold to continue them.
![Page 26: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/26.jpg)
Computer Vision : CISC 4/689
![Page 27: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/27.jpg)
Computer Vision : CISC 4/689
fine scalehigh Threshold(be strict inAcceptingEdge points)
![Page 28: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/28.jpg)
Computer Vision : CISC 4/689
coarse scale,high threshold
![Page 29: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/29.jpg)
Computer Vision : CISC 4/689
coarsescalelowthreshold
![Page 30: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/30.jpg)
Computer Vision : CISC 4/689
Canny Edge Detection
• Steps1. Apply derivative of Gaussian (not Laplacian!)
2. Non-maximum suppression
• Thin multi-pixel wide “ridges” down to single pixel
3. Thresholding
• Low, high edge-strength thresholds
• Accept all edges over low threshold that are connected to edge over high threshold (in the stage of predicting next edge point)
• Matlab: edge(I, ‘canny’)
![Page 31: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/31.jpg)
Computer Vision : CISC 4/689
Edge “Smearing”
from Forsyth & Ponce
0
0
0
0
2
2
2
2
20
20
20
20
2
2
2
2
Sobel filter example: Yields2-pixel wide edge “band”
We want to localize the edge to within 1 pixel
6
8
8
6
6
8
8
6
00
00
00
00
-8
-6
-6
-8
Input
Result
![Page 32: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/32.jpg)
Computer Vision : CISC 4/689
Non-Maximum Suppression: Steps
1. Consider 9-pixel neighborhood around each edge candidate (i.e., already over a threshold)
2. Interpolate edge strengths E at neighborhood boundaries in negative & positive gradient directions from the center pixel
3. If the pixel under consideration is not greater than these two values (i.e. not a maximum), it is suppressed
Interpolating the E value:
E(r) = (1 ¡ a)E(x, y) + aE(x + 1, y)
a 1 ¡ a
(x, y) (x + 1, y)r
![Page 33: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/33.jpg)
Computer Vision : CISC 4/689
Example: Non-Maximum Suppression
courtesy of G. Loy
Original image Gradient magnitudeNon-maxima suppressed
![Page 34: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/34.jpg)
Computer Vision : CISC 4/689
Edge “Streaking”
• Can predict next pixel in edge orthogonal to gradient to make edge chain– Can also just use 8-connectedness to define chains
• Streaking: Gaps in edge chain due to edge strength dipping below threshold
courtesy of G. Loy
Original image Strong edges
gap
![Page 35: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/35.jpg)
Computer Vision : CISC 4/689
Edge Hysteresis
• Hysteresis: A lag or momentum factor
• Idea: Maintain two thresholds khigh and klow
– Use khigh to find strong edges to start edge chain
– Use klow to find weak edges which continue edge chain
• Usual ratio of thresholds is roughly
khigh / klow = 2 or 3
![Page 36: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/36.jpg)
Computer Vision : CISC 4/689
Example: Canny Edge Detection
courtesy of G. Loy
gap is gone
Originalimage
Strongedges
only
Strong +connectedweak edges
Weakedges
![Page 37: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/37.jpg)
Computer Vision : CISC 4/689
Example: Canny Edge Detection
(Matlab automatically set thresholds)
![Page 38: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/38.jpg)
Computer Vision : CISC 4/689
Image Pyramids
• Observation: Fine-grained template matching expensive over a full image – Idea: Represent image at smaller
scales, allowing efficient coarse- to-fine search
• Downsampling: Cut width, height in half at each iteration:
from Forsyth & Ponce
![Page 39: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/39.jpg)
Computer Vision : CISC 4/689
Gaussian Pyramid
• Let the base (the finest resolution) of an n-level Gaussian pyramid be defined
as P0 = I. Then the ith level is reduced from the level below it by:
• Upsampling S"(I): Double size of image, interpolate missing pixels
courtesy of Wolfram
Gaussian pyramid
![Page 40: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/40.jpg)
Computer Vision : CISC 4/689
Reconstruction
![Page 41: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/41.jpg)
Computer Vision : CISC 4/689
Laplacian Pyramids
• The tip (the coarsest resolution) of an n-level Laplacian pyramid is the same as the Gaussian pyramid at that level: Ln(I) = Pn(I)
• The ith level is obtained from the level above according to Li(I) = Pi(I) ¡ S"(Pi+1(I))
• Synthesizing the original image: Get I back by summing upsampled Laplacian pyramid levels
![Page 42: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/42.jpg)
Computer Vision : CISC 4/689
Laplacian Pyramid
• The differences of images at successive levels of the Gaussian pyramid define the Laplacian pyramid. To calculate a difference, the image at a higher level in the pyramid must be increased in size by a factor of four prior to subtraction. This computes the pyramid.
• The original image may be reconstructed from the Laplacian pyramid by reversing the previous steps. This interpolates and adds the images at successive levels of the pyramid beginning with the coarsest level.
• Laplacian is largely uncorrelated, and so may be represented pixel by pixel with many fewer bits than Gaussian.
courtesy of Wolfram
![Page 43: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/43.jpg)
Computer Vision : CISC 4/689
Splining
• Build Laplacian pyramids LA and LB for A & B images
• Build a Gaussian pyramid GR from selected region R
• Form a combined pyramid LS from LA and LB using nodes of GR as weights:
LS(I,j) = GR(I,j)*LA(I,j)+(1-GR(I,j))*LB(I,j)
Collapse the LS pyramid to get the final blended image
![Page 44: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/44.jpg)
Computer Vision : CISC 4/689
Splining (Blending)
• Splining two images simply requires: 1) generating a Laplacian pyramid for each image, 2) generating a Gaussian pyramid for the bitmask indicating how the two images should be merged, 3) merging each Laplacian level of the two images using the bitmask from the corresponding Gaussian level, and 4) collapsing the resulting Laplacian pyramid.
• i.e. GS = Gaussian pyramid of bitmask LA = Laplacian pyramid of image "A" LB = Laplacian pyramid of image "B" therefore, "Lout = (GS)LA + (1-GS)LB"
![Page 45: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/45.jpg)
Computer Vision : CISC 4/689
Example images from GTech
Image-1 bit-mask image-2
Direct addition splining bad bit-mask choice
![Page 46: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/46.jpg)
Computer Vision : CISC 4/689
Outline
• Corner detection
• RANSAC
![Page 47: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/47.jpg)
Computer Vision : CISC 4/689
Matching with Invariant Features
Darya Frolova, Denis Simakov
The Weizmann Institute of Science
March 2004
![Page 48: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/48.jpg)
Computer Vision : CISC 4/689
Example: Build a Panorama
M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003
![Page 49: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/49.jpg)
Computer Vision : CISC 4/689
How do we build panorama?
• We need to match (align) images
![Page 50: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/50.jpg)
Computer Vision : CISC 4/689
Matching with Features
•Detect feature points in both images
![Page 51: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/51.jpg)
Computer Vision : CISC 4/689
Matching with Features
•Detect feature points in both images
•Find corresponding pairs
![Page 52: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/52.jpg)
Computer Vision : CISC 4/689
Matching with Features
•Detect feature points in both images
•Find corresponding pairs
•Use these pairs to align images
![Page 53: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/53.jpg)
Computer Vision : CISC 4/689
Matching with Features
• Problem 1:– Detect the same point independently in both images
no chance to match!
We need a repeatable detector
![Page 54: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/54.jpg)
Computer Vision : CISC 4/689
Matching with Features
• Problem 2:– For each point correctly recognize the corresponding one
?
We need a reliable and distinctive descriptor
![Page 55: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/55.jpg)
Computer Vision : CISC 4/689
More motivation…
• Feature points are used also for:– Image alignment (homography, fundamental matrix)
– 3D reconstruction
– Motion tracking
– Object recognition
– Indexing and database retrieval
– Robot navigation
– … other
![Page 56: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/56.jpg)
Computer Vision : CISC 4/689
Corner Detection
• Basic idea: Find points where two edges meet—i.e., high gradient in two directions
• “Cornerness” is undefined at a single pixel, because there’s only one gradient per point– Look at the gradient behavior over a small window
• Categories image windows based on gradient statistics– Constant: Little or no brightness change– Edge: Strong brightness change in single direction– Flow: Parallel stripes– Corner/spot: Strong brightness changes in orthogonal directions
![Page 57: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/57.jpg)
Computer Vision : CISC 4/689
Corner Detection: Analyzing Gradient Covariance
• Intuitively, in corner windows both Ix and Iy should be high– Can’t just set a threshold on them directly, because we want rotational invariance
• Analyze distribution of gradient components over a window to differentiate between types from previous slide:
• The two eigenvectors and eigenvalues ¸1, ¸2 of C (Matlab: eig(C)) encode the predominant directions and magnitudes of the gradient, respectively, within the window
• Corners are thus where min(¸1, ¸2) is over a threshold courtesy of Wolfram
![Page 58: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/58.jpg)
Computer Vision : CISC 4/689
Contents
• Harris Corner Detector
– Description
– Analysis
• Detectors
– Rotation invariant
– Scale invariant
– Affine invariant
• Descriptors
– Rotation invariant
– Scale invariant
– Affine invariant
![Page 59: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/59.jpg)
Computer Vision : CISC 4/689
Harris Detector: Mathematics
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
Change of intensity for the shift [u,v]:
IntensityShifted intensity
Window function
orWindow function w(x,y) =
Gaussian1 in window, 0 outside
Taylor series:F(x+dx,y+dy) = f(x,y)+fx(x,y)dx+fy(x,y)dy+…http://mathworld.wolfram.com.TaylorSeries.html
![Page 60: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/60.jpg)
Computer Vision : CISC 4/689
Harris Detector: Mathematics
( , ) ,u
E u v u v Mv
For small shifts [u,v] we have a bilinear approximation:
2
2,
( , ) x x y
x y x y y
I I IM w x y
I I I
where M is a 22 matrix computed from image derivatives:
![Page 61: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/61.jpg)
Computer Vision : CISC 4/689
Harris Detector: Mathematics
( , ) ,u
E u v u v Mv
Intensity change in shifting window: eigenvalue analysis
1, 2 – eigenvalues of M
direction of the slowest change
direction of the fastest change
(max)-1/2
(min)-1/2
Ellipse E(u,v) = const
![Page 62: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/62.jpg)
Computer Vision : CISC 4/689
Harris Detector: Mathematics
1
2
“Corner”1 and 2 are large,
1 ~ 2;
E increases in all directions
1 and 2 are small;
E is almost constant in all directions
“Edge” 1 >> 2
“Edge” 2 >> 1
“Flat” region
Classification of image points using eigenvalues of M:
![Page 63: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/63.jpg)
Computer Vision : CISC 4/689
Harris Detector: Mathematics
Measure of corner response:
2det traceR M k M
1 2
1 2
det
trace
M
M
(k – empirical constant, k = 0.04-0.06)
![Page 64: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/64.jpg)
Computer Vision : CISC 4/689
Harris Detector: Mathematics
1
2 “Corner”
“Edge”
“Edge”
“Flat”
• R depends only on eigenvalues of M
• R is large for a corner
• R is negative with large magnitude for an edge
• |R| is small for a flat region
R > 0
R < 0
R < 0|R| small
![Page 65: Computer Vision : CISC 4/689 Gradients and edges Points of sharp change in an image are interesting: –change in reflectance –change in object –change in](https://reader034.vdocuments.site/reader034/viewer/2022051417/56649f035503460f94c1733c/html5/thumbnails/65.jpg)
Computer Vision : CISC 4/689
Harris Detector
• The Algorithm:– Find points with large corner response function R (R >
threshold)
– Take the points of local maxima of R