cse 185 introduction to computer vision edges. scale space reading: chapter 3 of s

50
CSE 185 Introduction to Computer Vision Edges

Upload: ferdinand-hall

Post on 14-Jan-2016

218 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

CSE 185 Introduction to Computer

VisionEdges

Page 2: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Edges

• Edges• Scale space

• Reading: Chapter 3 of S

Page 3: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Edge detection• Goal: Identify sudden

changes (discontinuities) in an image– Intuitively, most semantic

and shape information from the image can be encoded in the edges

– More compact than pixels• Ideal: artist’s line

drawing (but artist is also using object-level knowledge)

Page 4: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Why do we care about edges?• Extract information,

recognize objects

• Recover geometry and viewpoint

Vanishing point

Vanishing

line

Vanishing point

Vertical vanishing point

(at infinity)

Page 5: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Origin of Edges

• Edges are caused by a variety of factors

depth discontinuity

surface color discontinuity

illumination discontinuity

surface normal discontinuity

Page 6: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Marr’s theory

Page 7: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Example

Page 8: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Closeup of edges

Source: D. Hoiem

Page 9: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Closeup of edges

Page 10: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Closeup of edges

Page 11: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Closeup of edges

Page 12: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Characterizing edges• An edge is a place of rapid change in the

image intensity function

imageintensity function

(along horizontal scanline) first derivative

edges correspond toextrema of derivative

Page 13: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Intensity profile

Page 14: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

With a little Gaussian noise

Gradient

Page 15: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Effects of noise• Consider a single row or column of the image

– Plotting intensity as a function of position gives a signal

Where is the edge?

Page 16: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Effects of noise

• Difference filters respond strongly to noise– Image noise results in pixels that look

very different from their neighbors– Generally, the larger the noise the

stronger the response

• What can we do about it?

Page 17: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Solution: smooth first

• To find edges, look for peaks in )( gfdx

d

f

g

f * g

)( gfdx

d

Page 18: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

• Differentiation is convolution, and convolution is associative:

• This saves us one operation:

gdx

dfgf

dx

d )(

Derivative theorem of convolution

gdx

df

f

gdx

d

Page 19: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Derivative of Gaussian filter

* [1 -1] =

Page 20: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Differentiation and convolution• Recall

• Now this is linear and shift invariant, so must be the result of a convolution

• We could approximate this as

(which is obviously a convolution; it’s not a very good way to do things, as we shall see)

),(),(lim

0

yxfyxf

x

f

x

yxfyxf

x

f nn

),(),( 1

000

101

000

H

Page 21: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Discrete edge operators• How can we differentiate a discrete image?

Finite difference approximations:

1, jiI 1,1 jiI

jiI , jiI ,1

Convolution masks :

jijijiji

jijijiji

IIIIy

I

IIIIx

I

,1,,11,1

,,11,1,1

2

12

1

2

1

x

I

2

1

y

I

Page 22: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

1, jiI 1,1 jiI

jiI , jiI ,1

1,1 jiI

jiI ,1

1,1 jiI 1, jiI 1,1 jiI

• Second order partial derivatives:

• Laplacian :

Convolution masks :

or

Discrete edge operators

(more accurate)

1 4 1

4 -20

4

1 4 1

0 1 0

1 -4 1

0 1 0

1,,1,22

2

,1,,122

2

21

21

jijiji

jijiji

IIIy

I

IIIx

I

2

2

2

22

y

I

x

II

22 1

I

26

1

Page 23: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Finite differences

Partial derivative in y axis, respond strongly tohorizontal edges

Partial derivative in x axis, respond strongly to vertical edges

Page 24: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Spatial filter

• Approximation

|)||)(|||

|)||)(|||

|)||)(|||

])()[(||

||,

8695

2/18695

6585

2/1265

285

2/122

zzzzf

zzzzf

zzzzf

zzzzf

y

f

x

ff

y

fx

f

f

987

654

321

zzz

zzz

zzz

Page 25: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Roberts operator

One of the earliest edge detection algorithm by Lawrence Roberts

Page 26: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Sobel operatorOne of the earliest edge detection algorithm by Irwine Sobel

Page 27: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Image gradient• Gradient equation:

• Represents direction of most rapid change in intensity

• Gradient direction:

• The edge strength is given by the gradient magnitude

Page 28: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

2D Gaussian edge operators

Laplacian of GaussianGaussian Derivative of Gaussian (DoG)

Mexican Hat (Sombrero)• is the Laplacian operator:

Marr-Hildreth algorithm

Page 29: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

• Smoothed derivative removes noise, but blurs edge. Also finds edges at different “scales”.

1 pixel 3 pixels 7 pixels

Tradeoff between smoothing and localization

Page 30: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Designing an edge detector• Criteria for a good edge detector:

– Good detection: the optimal detector should find all real edges, ignoring noise or other artifacts

– Good localization•the edges detected must be as close as

possible to the true edges•the detector must return one point only for

each true edge point• Cues of edge detection

– Differences in color, intensity, or texture across the boundary

– Continuity and closure– High-level knowledge

Page 31: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Canny edge detector• Probably the most widely used edge

detector in computer vision• Theoretical model: step-edges

corrupted by additive Gaussian noise• Canny has shown that the first

derivative of the Gaussian closely approximates the operator that optimizes the product of signal-to-noise ratio and localization

J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986.

http://www.mathworks.com/discovery/edge-detection.html

Page 32: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Example

original image (Lena)

Page 33: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Derivative of Gaussian filter

x-direction y-direction

Page 34: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Compute gradients

X-Derivative of Gaussian Y-Derivative of Gaussian Gradient Magnitude

Page 35: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Get orientation at each pixel

• Threshold at minimum level• Get orientation theta = atan2(gy, gx)

Page 36: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Non-maximum suppression for each orientation

At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values.

Page 37: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Sidebar: Interpolation options• imx2 = imresize(im, 2,

interpolation_type)

• ‘nearest’ – Copy value from nearest known– Very fast but creates blocky edges

• ‘bilinear’– Weighted average from four nearest known

pixels– Fast and reasonable results

• ‘bicubic’ (default)– Non-linear smoothing over larger area (4x4)– Slower, visually appealing, may create

negative pixel values

Page 38: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Before non-max suppression

Page 39: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

After non-max suppression

Page 40: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Hysteresis thresholding• Threshold at low/high levels to get weak/strong edge pixels• Do connected components, starting from strong edge pixels

Page 41: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Hysteresis thresholding

• Check that maximum value of gradient value is sufficiently large– drop-outs? use hysteresis

• use a high threshold to start edge curves and a low threshold to continue them.

Source: S. Seitz

Page 42: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Final Canny edges

Page 43: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Canny edge detector

1. Filter image with x, y derivatives of Gaussian 2. Find magnitude and orientation of gradient3. Non-maximum suppression:

– Thin multi-pixel wide “ridges” down to single pixel width

4. Thresholding and linking (hysteresis):– Define two thresholds: low and high– Use the high threshold to start edge curves and

the low threshold to continue them

• MATLAB: edge(image, ‘canny’)

Page 44: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Effect of (Gaussian kernel size)

Canny with Canny with original

The choice of depends on desired behavior• large detects large scale edges• small detects fine features

Page 45: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Where do humans see boundaries?

• Berkeley segmentation database:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/

image human segmentation gradient magnitude

Page 46: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

pB boundary detector

Martin, Fowlkes, Malik 2004: Learning to Detect Natural Boundaries…http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/papers/mfm-pami-boundary.pdf

Page 47: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Brightness

Color

Texture

Combined

Human

Page 48: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Human (0.95)

Pb (0.88)

Page 49: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

State of edge detection

• Local edge detection works well– But many false positives from

illumination and texture edges

• Some methods to take into account longer contours, but could probably do better

• Poor use of object and high-level information

Page 50: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S

Sketching

• Learn from artist’s strokes so that edges are more likely in certain parts of the face.

Berger et al. SIGGRAPH 2013