Introduction

Edge detectionWinter in Krakw photographed by Marcin Ryczek

Edge detectionGoal: Identify sudden changes (discontinuities) in an imageIntuitively, most semantic and shape information from the image can be encoded in the edgesMore compact than pixels

Ideal: artists line drawing (but artist is also using object-level knowledge)Source: D. Lowe2Origin of edgesEdges are caused by a variety of factors:

depth discontinuitysurface color discontinuityillumination discontinuitysurface normal discontinuitySource: Steve Seitz3Review: Linear filteringImage blurringSeparabilityMedian filtering

Edge detectionAn edge is a place of rapid change in the image intensity function

imageintensity function(along horizontal scanline)first derivativeedges correspond toextrema of derivative5Derivatives with convolutionFor 2D function f(x,y), the partial derivative is:

For discrete data, we can approximate using finite differences:

To implement the above as convolution, what would be the associated filter?

Source: K. Grauman

Partial derivatives of an image

Which shows changes with respect to x?

-1 11 -1or-1 1

7Finite difference filtersOther approximations of derivative filters exist:

Source: K. Grauman8The gradient points in the direction of most rapid increase in intensity

Image gradientThe gradient of an image:

The gradient direction is given by

Source: Steve SeitzThe edge strength is given by the gradient magnitudeHow does this direction relate to the direction of the edge?9Application: Gradient-domain image editingGoal: solve for pixel values in the target region to match gradients of the source region while keeping background pixels the same

P. Perez, M. Gangnet, A. Blake, Poisson Image Editing, SIGGRAPH 2003Effects of noiseConsider a single row or column of the image

Where is the edge?Source: S. Seitz11How to fix?Solution: smooth firstTo find edges, look for peaks in

f

g

f * g

Source: S. Seitz12Differentiation is convolution, and convolution is associative:

This saves us one operation:

Derivative theorem of convolution

f

Source: S. Seitz13Derivative of Gaussian filtersWhich one finds horizontal/vertical edges?

x-directiony-direction14Derivative of Gaussian filtersAre these filters separable?

x-directiony-direction15Recall: Separability of the Gaussian filter

Source: D. Lowe16Smoothed derivative removes noise, but blurs edge. Also finds edges at different scales

1 pixel3 pixels7 pixelsScale of Gaussian derivative filterSource: D. Forsyth17Review: Smoothing vs. derivative filtersSmoothing filtersGaussian: remove high-frequency components; low-pass filterCan the values of a smoothing filter be negative?What should the values sum to?One: constant regions are not affected by the filter

Derivative filtersDerivatives of GaussianCan the values of a derivative filter be negative?What should the values sum to? Zero: no response in constant regionsHigh absolute value at points of high contrast

The Canny edge detectorFilter image with derivative of Gaussian Find magnitude and orientation of gradientNon-maximum suppression:Thin wide ridges down to single pixel widthLinking and thresholding (hysteresis):Define two thresholds: low and highUse the high threshold to start edge curves and the low threshold to continue them

MATLAB: edge(image, canny);J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986. The Canny edge detectororiginal image

Slide credit: Steve SeitzThe Canny edge detector

norm of the gradientThe Canny edge detector

thresholdingThe Canny edge detector

thresholdingHow to turn these thick regions of the gradient into curves?

Non-maximum suppressionCheck if pixel is local maximum along gradient direction, select single max across width of the edgerequires checking interpolated pixels p and r

The Canny edge detectorthinning(non-maximum suppression)

Problem: pixels along this edge didnt survive the thresholdingHysteresis thresholdingUse a high threshold to start edge curves, and a low threshold to continue them.Source: Steve SeitzHysteresis thresholding

original image

high threshold(strong edges)

low threshold(weak edges)

hysteresis thresholdSource: L. Fei-Fei27Recap: Canny edge detectorCompute x and y gradient images Find magnitude and orientation of gradientNon-maximum suppression:Thin wide ridges down to single pixel widthLinking and thresholding (hysteresis):Define two thresholds: low and highUse the high threshold to start edge curves and the low threshold to continue them

MATLAB: edge(image, canny);J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986. Image gradients vs. meaningful contoursBerkeley segmentation database:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/

imagehuman segmentationgradient magnitude29Data-driven edge detectionP. Dollar and L. Zitnick, Structured forests for fast edge detection, ICCV 2013

Training data

Input imagesGround truthOutput