local image preprocessing (cont’d) - uvic.caaalbu/computer vision 2010/l11-12... · 8 edge...

1

Local Image preprocessing (cont’d)

2

Outline

-  Edge detectors -  Corner detectors

-  Reading: textbook 5.3.1-5.3.5 and 5.3.10

3

What are edges?

  Edges correspond to relevant features in the image.

  An edge corresponds to a change in intensity, surface orientation, shadow etc.

  Object recognition and detection use often times edge information (in both CV and biological vision)

4

Why study edges?

  Collecting information about edges will help us to identify contours in the image and to retrieve regions enclosed by those contours.

  An ‘edge’ image represents a higher level of abstraction (i.e. less information to process)

  Edges are features invariant to absolute illumination (as opposed to colour information)

5

Linear filters for edge detection

  We need linear filters for:   Edge enhancement – image sharpening   Removal of false edges (noise

generated)

  Edge localization   Needs other operations (such as

thresholding)

6

Types of 1D edge profiles

From Forsyth and Ponce

7

Edge representation

From Forsyth and Ponce

8

Edge enhancement   Computing the image gradient : vector

composed of first-order partial derivatives

  The gradient magnitude gives the amount of the difference between pixels in the neighborhood (the strength of the edge).

  The gradient orientation gives the direction of the greatest change, which presumably is the direction across the edge (the edge normal).

  Derivatives are linear and shift invariant, thus the gradient can be computed with convolution

9

First order differences of an image

Estimates of derivatives produced by 1st order differences

10

Convolution kernels: Roberts Idea: Approximate the gradient using finite differences

Forward differences

- The Roberts kernels correspond to derivatives with respect to the two diagonal directions

Pros: Only four pixels are needed for the gradient computation

Cons: Masks are too small to obtain reliable results in the presence of noise

11

Convolution kernels: Prewitt

  Based on idea of central difference:

very sensitive to noise

Averaging for reducing noise sensitivity:

In y when computing

In x when computing

12

Convolution kernels: Sobel

  Also rely on central differences, but give greater weight to central pixels when averaging:

13

From gradient magnitude to edges

  The gradient magnitude gives a measure at every pixel of the “edginess” of each pixel:

  We still have to find and connect the “best” edge points:   Local maxima of gradient magnitude   Connected maxima (ridges)   Maximum-magnitude contours/paths

Adapted from Brian Morse, http://morse.cs.byu.edu/450/home/index.php

14

From gradient magnitude to edges

  The gradient magnitude gives a measure at every pixel of the “edginess” of each pixel:

  We still have to find and connect the “best” edge points:   Local maxima of gradient magnitude   Connected maxima (ridges)   Maximum-magnitude contours/paths

Adapted from Brian Morse, http://morse.cs.byu.edu/450/home/index.php

15

From Tomas Svoboda: Image preprocessing in the spatial domain

16

Marr-Hildreth Edge Detection

  approximate finding maxima of gradient magnitude (edges) by finding pixels where the Laplacian is zero valued.

  Problem: we cannot always find discrete pixels where the Laplacian is exactly zero.   Zero-crossings instead

17

Combining first and second order derivatives

  Laplacian zero crossings:

  Problem: Tells us the gradient magnitude is at a maximum, not how strong it is—lots of spurious edges.

  Idea: Combine two measures

From Brian Morse, http://morse.cs.byu.edu/450/home/index.php

18

Edge detection and noise

  Partial derivatives are very sensitive to noise   image noise results in pixels that look very

different from their neighbours   Solution: Gaussian smoothing before filtering

Response of finite differences to noise

σ=0.03 σ=0.09

20

Major issues: 1) The gradient magnitude at different values of σ is different; which one should we choose? 2) Large values of the gradient magnitude form thick trails; how do we extract 1 pixel-wide boundaries?

Original image Gradient magnitudes using the derivatives of a Gaussian with σ=1 pixel

σ=2 pixels

21

Laplacian of Gaussian

  To include a smoothing Gaussian filter, combine the Laplacian and Gaussian functions to obtain a single equation:

A discrete kernel of the LoG with σ=1.4

22

Laplacian of Gaussian - example

From Efford

23

Limits of LoG

Boundaries of sharp corners are inaccurate; Trihedral vertices are not detected – loops.

Problem with LOG: adds the principal curvatures together; does not determine a gradient of maximum amplitude in any one direction

The Canny Edge Detector defines edges as zero-crossings of second derivatives in the direction of greatest first derivative.

24

The Canny edge detector

  Also known as the optimal edge detector   Published in IEEE Trans. on Pattern

Analysis and Machine Intelligence, 1986   Intended to enhance the already existent

edge detectors   Formulated a list of criteria to enhance

the current methods of edge detection

25

Canny edge detector: criteria

  detection: edges occurring in images should not be missed; there should be no responses to non-edges.

  edge points have to be well localized. The distance between the edge pixels as found by the detector and the actual edge is to be minimal.

  Only one response at a single edge is allowed. Eliminates multiple detections of the same edge.

  Note: There is a trade-off between good detection and good localization. Why?

26

Canny edge detector: main steps

  1. Filter out noise in the original image before trying to locate and detect any edges.

  2. Compute the image gradient to highlight regions with high spatial derivatives.

  3. Non-maximum suppression   4. Hysteresis

27

Canny edge detector: Step 1+2

  We can implement both steps by filtering with a Difference of Gaussian (DoG) filter

or:   Step 1: convolution with a Gaussian   Step 2: computation of spatial

derivatives with Sobel masks.   Approximate edge strength with: |G| = |Gx| + |Gy|

28

Canny edge detector: Non maxima suppression

The gradient magnitude tends to be large along thick trails in an image.

- To replace this thick trail with a representative thin curve, we have to find a cut direction and to extract the intensity maxima along this direction.

- The gradient direction is a good estimate of the cut direction

29

Canny edge detector: non maximum suppression

Nonmaximum suppression obtains points where the gradient magnitude is at a maximum along the direction of the gradient.

- The gradient direction (forward and backward) does not pass through any pixels on the grid.

- Linear interpolation for computing the values of gradient magnitude at p and r (considering the pixels to the left and right of p and r)

-  if G(q)>max(Gint(p), Gint(r)) then q is retained as an edge point;

else q is suppressed.

30

Canny edge detector: hysteresis

  In spite of the smoothing step, the non maxima suppressed magnitude image will contain many false edge fragments caused by noise and fine texture.

  Solution: thresholding   One threshold is not enough, since the gradient

magnitude is variable   Thresholding with hysteresis: - retain all edge candidates with gradient

magnitude>TH

-  Edge following by linking edge candidates with gradient magnitude>TL

32

Strong edges reinforce weak edges

33

Matlab implementation of Canny BW = edge(I,'canny') specifies the Canny method.

  BW = edge(I,'canny',thresh) specifies sensitivity thresholds for the Canny method. thresh is a two-element vector in which the first element is the low threshold, and the second element is the high threshold.

  If you specify a scalar for thresh, this value is used for the high threshold and 0.4*thresh is used for the low threshold. If you do not specify thresh, or if thresh is empty ([]), edge chooses low and high values automatically.

  BW = edge(I,'canny',thresh,sigma) specifies the Canny method, using sigma as the standard deviation of the Gaussian filter. The default sigma is 1; the size of the filter is chosen automatically, based on sigma.

  [BW,thresh] = edge(I,'canny',...) returns the threshold values as a two-element vector.

34

Edge detectors: examples

Edge detectors by gradient operators (Prewitt, Roberts, Sobel)

35

Edge detectors: examples (2)

Edge detection with DOG and LOG

36

Edge detectors: examples

Edge detection with Canny: σ=1,2,3 τ1=0.3 τ2=0.7

37

What to remember

  Edge = Rapid intensity (color) change   Edge information is one of the most

important in CV and Human Vision   Three steps in edge detection:

  Noise reduction (image smoothing)   Edge enhancement   Edge localisation

  Three types were presented:   Based on first-order derivative

  Roberts, Prewitt and Sobel   Based on second-order derivative

  Laplacian of Gaussian   Based on linking edges

  Canny

38

Corner detection

  Corners are important features in the image

  Useful for matching two images of the same static scene acquired from different viewpoints (stereo vision, building mosaics)

  Useful for object recognition as well

The Moravec corner detector

Developed by Hans Moravec in 1977 in his research about autonomous navigation and obstacle avoidance

http://www.frc.ri.cmu.edu/~hpm/project.archive/robot.papers/1977/aips.txt

-  defined the concept of "points of interest” -  these interest points could be used to find matching regions in consecutive image frames -  interest points=points where there is a large intensity variation in every direction

Slides about Moravec detector use materials from - http://www.cim.mcgill.ca/~dparks/CornerDetector/moravec.htm 39

The Moravec corner detector: measuring the intensity variation

40

The Moravec corner detector: why does it detect corners?

41

Cornerness map

43

Moravec-results

44

Moravec-limitations   Anisotropic response; large response to diagonal

edges

45

The Harris corner detector

  Improves upon Moravec   Isotropic   More computationally intensive

46

47

Change

No Change

Ideal vertical edge

48

No Change

Change

Ideal horizontal edge

49

Harris corner detector

C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988

50

The Basic Idea   We should easily recognize the point by looking

through a small window   Shifting a window in any direction should change

significantly the intensity distribution inside the window

From Darya Frolova, Denis Simakov The Weizmann Institute of Science Lecture notes on Matching with Invariant features

51

Harris Detector: Basic Idea

“flat” region: no change in all directions

“edge”: no change along the edge direction

“corner”: significant change in all directions


52

Harris Detector: Mathematics

Change of intensity for the shift [Δx, Δy]:

Intensity Shifted

intensity

or Window function w(x,y) =

Gaussian 1 in window, 0 outside

€

S(x, y) = f xi , yi( ) − f xi − Δx, yi − Δy( )[ ](xi ,yi )∈W∑

2

53


We will show that

Where A captures the intensity structure of the local neighborhood. Its main modes of variation are reflected in its eigenvalues λ1 and λ2

€

S(x, y) = Δx,Δy[ ]AW (x, y)ΔxΔy⎡

⎣ ⎢

⎤

⎦ ⎥

54


λ1

λ2

“Corner” λ1 and λ2 are large, λ1 ~ λ2; E increases in all directions

λ1 and λ2 are small; E is almost constant in all directions

“Edge” λ1 >> λ2

“Edge” λ2 >> λ1

“Flat” region

Classification of image points using eigenvalues of A:

From Darya Frolova & Denis Simakov

55

Harris Detector: Mathematics (cont’d)

Measure of corner response:

(k – tunable empirical constant, k = 0.04-0.15)

€

R(A) = det(A) − k ⋅ trace2 (A)

Det(A)=λ1λ2

Trace(A)=λ1+ λ2

But: we do not need to compute λ1 and λ2 for the computation of R.

56


λ1

λ2 “Corner”

“Edge”

“Edge”

“Flat”

•  R depends only on eigenvalues of M

•  R is large for a corner

•  R is negative with large magnitude for an edge

•  |R| is small for a flat region

R > 0

R < 0

R < 0 |R| small From Darya Frolova, Denis Simakov Lecture notes on Matching with Invariant features

57

Harris Detector: the algorithm

  Filter the image with a Gaussian   Compute the horizontal and vertical

gradient images with Sobel masks or a double 1D convolution

  Define the neighborhood W (preferably gaussian, thus isotropic)

  Find points with large corner response function R(A) (R > threshold)

  Take the points of local maxima of R

58

Harris Detector: Workflow Compute corner response R

59

Harris Detector: Workflow Find points with large corner response: R>threshold

60

Harris Detector: Workflow Take only the points of local maxima of R

61

Harris Detector: Workflow

62

Harris detector - examples

63

Harris Detector: Some Properties

  Rotation invariance

Ellipse rotates but its shape (i.e. eigenvalues) remains the same

Corner response R is invariant to image rotation

From Darya Frolova, Denis Simakov Lecture notes on Matching with Invariant features

64


  Partial invariance to affine intensity change

  Only derivatives are used => invariance to intensity shift I → I + b

  Intensity scale: I → a I

R

x (image coordinate)

threshold

R

x (image coordinate)

65


  But: non-invariant to image scale!

All points will be classified as edges

Corner !


Invariance to scale: SIFT

  D. Lowe, 2004: Distinctive image features from scale-invariant keypoints

  Paper is mandatory reading for ELEC 536; optional for CENG 421.

66

local image preprocessing (cont’d) - uvic.caaalbu/computer vision 2010/l11-12... · 8 edge...

Documents