Computational Vision / Ioannis StamosComputational Vision / Ioannis Stamos
Edge DetectionEdge Detection Canny DetectorCanny Detector
Line DetectionLine Detection Hough TransformHough Transform
Trucco: Chapter 4, pp. 76 – 80Trucco: Chapter 4, pp. 76 – 80
Chapter 5, pp. 95 - 100Chapter 5, pp. 95 - 100
Computational VisionComputational Vision
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Finding Corners Finding Corners
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
What Is a Corner?What Is a Corner?
Large gradients in more than one direction.
Edges vs. CornersEdges vs. Corners
Edges = maxima in intensity gradientEdges = maxima in intensity gradient
Edges vs. CornersEdges vs. Corners
Corners = lots of variation in direction of Corners = lots of variation in direction of gradient in a small neighborhoodgradient in a small neighborhood
Detecting CornersDetecting Corners
How to detect this variation?How to detect this variation?
Detecting CornersDetecting Corners
How to detect this variation?How to detect this variation? Not enough to check average andNot enough to check average and
x
f
x
f
y
f
y
f
Detecting CornersDetecting Corners
Claim: the following covariance matrix Claim: the following covariance matrix summarizes the statistics of the gradientsummarizes the statistics of the gradient
Summations over local neighborhoodsSummations over local neighborhoods
y
ff
x
ff
fff
fffC yx
yyx
yxx
,2
2
y
ff
x
ff
fff
fffC yx
yyx
yxx
,2
2
Detecting CornersDetecting Corners
Examine behavior of Examine behavior of CC by testing its effect by testing its effect in simple casesin simple cases
Case #1: Single edge in local Case #1: Single edge in local neighborhoodneighborhood
Case#1: Single EdgeCase#1: Single Edge
Let (Let (aa,,bb) be gradient along edge) be gradient along edge Compute Compute CC ((aa,,bb):):
b
aff
b
aff
b
a
fff
fff
b
aC
yyx
yxx
T
2
2
b
aff
b
aff
b
a
fff
fff
b
aC
yyx
yxx
T
2
2
y
x
f
ff
y
x
f
ff
Case #1: Single EdgeCase #1: Single Edge
However, in this simple case, the only However, in this simple case, the only nonzero terms are those where nonzero terms are those where ff = ( = (aa,,bb))
So, So, CC ((aa,,bb) is just some multiple of () is just some multiple of (aa,,bb))
Case #2: CornerCase #2: Corner
Assume there is a corner, with Assume there is a corner, with perpendicular gradients (perpendicular gradients (aa,,bb) and () and (cc,,dd))
Case #2: CornerCase #2: Corner
What is What is CC ((aa,,bb)?)? Since (Since (aa,,bb) ) ( (cc,,dd) = 0, the only nonzero terms ) = 0, the only nonzero terms
are those where are those where ff = ( = (aa,,bb)) So, So, CC ((aa,,bb) is again just a multiple of () is again just a multiple of (aa,,bb))
What is What is CC ((cc,,dd)?)? Since (Since (aa,,bb) ) ( (cc,,dd) = 0, the only nonzero terms ) = 0, the only nonzero terms
are those where are those where ff = ( = (cc,,dd)) So, So, CC ((cc,,dd) is a multiple of () is a multiple of (cc,,dd))
Corner DetectionCorner Detection
Matrix times vector = multiple of vectorMatrix times vector = multiple of vector Eigenvectors and eigenvalues!Eigenvectors and eigenvalues! In particular, if In particular, if CC has has oneone large large
eigenvalue, there’s an edgeeigenvalue, there’s an edge If If CC has has twotwo large eigenvalues, have large eigenvalues, have
cornercorner Tomasi-Kanade corner detectorTomasi-Kanade corner detector
Corner Detection Corner Detection Implementation Implementation
1.1. Compute image gradientCompute image gradient
2.2. For each For each mmmm neighborhood, neighborhood,compute matrix compute matrix CC
3.3. If smaller eigenvalue If smaller eigenvalue 22 is larger than is larger than
threshold threshold , record a corner, record a corner
4.4. Nonmaximum suppression: only keep Nonmaximum suppression: only keep strongest corner in each strongest corner in each mmmm window window
Corner Detection ResultsCorner Detection Results
CheckerboardCheckerboardwith noisewith noise
Trucco & VerriTrucco & Verri
Corner Detection ResultsCorner Detection Results
Corner Detection ResultsCorner Detection Results
Histogram of Histogram of 22 (smaller eigenvalue (smaller eigenvalue))
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
A Simple Corner DetectorA Simple Corner Detector
2
2
yyx
yxx
III
IIIC
21 Find eigenvalues of C
If the smaller eigenvalue is above a threshold then we have a corner.
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
A Simple Corner DetectorA Simple Corner Detector
2
2
yyx
yxx
III
IIIC
21 Find eigenvalues of C
If the smaller eigenvalue is above a threshold then we have a corner.
0yI
00
0aC
a
aC
0
0
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
A Simple Corner DetectorA Simple Corner Detector
Canny Edge DetectionCanny Edge Detection Can we derive an optimal detector?Can we derive an optimal detector? Good Detection:Good Detection: minimize false positives and false negatives. minimize false positives and false negatives. Good Localization:Good Localization: close as possible to the true edges. close as possible to the true edges. Single Response Constraint:Single Response Constraint: one edge should be detected one edge should be detected
for each true edgefor each true edge
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
ComparisonComparison
CannySobel
Canny Edge DetectionCanny Edge Detection
3 STEPS:3 STEPS: CANNY_ENHANCERCANNY_ENHANCER NONMAX_SUPPRESIONNONMAX_SUPPRESION HYSTERESIS_THRESHHYSTERESIS_THRESH
Localization-Detection TradeoffLocalization-Detection Tradeoff
Filter’s spatial scale Filter’s spatial scale Location &Location & Detection criteria.Detection criteria.
Good tradeoff 1-D step edge detector:Good tradeoff 1-D step edge detector:
(optimal)(optimal) Approximated by the 1Approximated by the 1stst derivative of the Gaussian. derivative of the Gaussian.
Optimal 1-D step edge detectors…
Pixel (i,j)
Keep the one that gives you maximum response.
Expensive.
Solution…
Pixel (i,j)
Compute derivatives with respect to x & y directions.
Compute edge normal.
x
y
CANNY_ENHANCERCANNY_ENHANCER Compute Compute IIxx and and IIyy the gradient of the image using a derivative of the gradient of the image using a derivative of
Gaussian filter.Gaussian filter. Compute the edge strength from the magnitude of the gradient: Compute the edge strength from the magnitude of the gradient:
EsEs
Compute the orientation of the edge from arctan(Compute the orientation of the edge from arctan(IIyy / / IIxx ): ): EoEo
[Canny ’86]n
• Compute image derivatives • if gradient magnitude > and the value is a local max. along gradient direction – pixel is an edge candidate• how to detect one pixel thin edges ?
gradient magnitude (Es)original image
Problem with detectorProblem with detector
NONMAX_SUPPRESSIONNONMAX_SUPPRESSION
0
4590
135
Locate local maxima from Es.
•Find direction d that best approximates Eo(i,j)•If Es(i,j) is smaller than at least one neighbor along d In(i,j)=0 else In(i,j)=Es(i,j).
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Non-Maximum SupressionNon-Maximum Supression
Non-maximum suppression:Select the single maximum point across the width of an edge.
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Linking to the Next Edge PointLinking to the Next Edge Point
Assume the marked point q is an edge point.
Take the normal to the gradient at that point and use this to predict continuation points (either r or p).
ThresholdingThresholding Edges are found by thresholding the output of Edges are found by thresholding the output of
NONMAX_SUPRESSIONNONMAX_SUPRESSION If the threshold is too high:If the threshold is too high:
Very few (none) edges Very few (none) edges High MISDETECTIONS, many gapsHigh MISDETECTIONS, many gaps
If the threshold is too low:If the threshold is too low: Too many (all pixels) edgesToo many (all pixels) edges
High FALSE POSITIVES, many extra edgesHigh FALSE POSITIVES, many extra edges
Edge Detection With HysteresisEdge Detection With Hysteresis
Low threshold High threshold
Hysteresis (high and low threshold)
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Edge HysteresisEdge Hysteresis
HysteresisHysteresis:: A lag or momentum factorA lag or momentum factor Idea: Maintain two thresholds Idea: Maintain two thresholds kkhighhigh and and kklowlow
Use Use kkhighhigh to find strong edges to start edge to find strong edges to start edge
chainchain Use Use kklowlow to find weak edges which to find weak edges which
continue edge chaincontinue edge chain Typical ratio of thresholds is roughlyTypical ratio of thresholds is roughly
kkhighhigh // kklowlow == 22
Edge TrackingEdge TrackingHysteresis thresholding [Canny ’86]
Strong edge
Weak edges
Strong edges reinforce weak edges.
We call a pixel an edge if it is strong.We also call a pixel an edge if it is weak but is connected to an edge.A pixel is connected to an edge if it is in a direction perpendicular to the edge normal
Weak edge removed
edges
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Canny Edge Detection (Example)Canny Edge Detection (Example)
courtesy of G. Loy
gap is gone
Originalimage
Strongedges
only
Strong +connectedweak edges
Weakedges
HYSTERESIS_THRESHHYSTERESIS_THRESH
Input: In, Eo, high threshold, low threshold.Input: In, Eo, high threshold, low threshold. Output: Lists of connected edges Output: Lists of connected edges
(contours).(contours).
Edge RelaxationEdge RelaxationParallel – iterative method to adjust edge values on the basis of neighboring edges
Crack edges:
Edge RelaxationEdge RelaxationParallel – iterative method to adjust edge values on the basis of neighboring edges
Vertex types:
Notation:Edge to be updated
Edge
No edge
Crack edges:
(0)
(1)
(2)
(3)
Edge RelaxationEdge RelaxationParallel – iterative method to adjust edge values on the basis of neighboring edges
Vertex types:
Notation:Edge to be updated
Edge
No edge
Crack edges:
(0)
(1)
(2)
(3)
Action table for edge type:
Decrease Increase Unchanged 0-0 1-1 0-1 0-2 1-2 2-2 0-3 1-3 2-3
3-3
Edge RelaxationEdge RelaxationParallel – iterative method to adjust edge values on the basis of neighboring edges
Vertex types:
Notation:Edge to be updated
Edge
No edge
Crack edges:
(0)
(1)
(2)
(3)
Action table for edge type:
Decrease Increase Unchanged 0-0 1-1 0-1 0-2 1-2 2-2 0-3 1-3 2-3
3-3
Algorithm:
0. Compute initial confidence C0(e) of each edge e C0(e) =
1. k = 1
2. Compute edge type for all e
3. Modify Ck(e) based on Ck-1(e) and edge type.
4. If all Ck(e) have converged to 1 or 0 else go to step 1
)max( I
e
Computational Vision / Ioannis StamosComputational Vision / Ioannis Stamos
Canny Edge DetectorCanny Edge Detector
Original: LenaOriginal: Lena EdgesEdges
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Towards Global FeaturesTowards Global Features
Local versus global
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
From Edges to LinesFrom Edges to Lines
Detecting LinesDetecting Lines
What is the difference between line What is the difference between line detection and edge detection?detection and edge detection? Edges = localEdges = local Lines = nonlocalLines = nonlocal
Line detection usually performed on the Line detection usually performed on the output of an edge detectoroutput of an edge detector
From Szymon Rusinkiewicz, Princeton
Detecting LinesDetecting Lines
Possible approaches ?Possible approaches ?
From Szymon Rusinkiewicz, Princeton
Detecting LinesDetecting Lines
Possible approaches:Possible approaches: Brute force: enumerate all lines, check if presentBrute force: enumerate all lines, check if present Hough transform: vote for lines to which detected Hough transform: vote for lines to which detected
edges might belongedges might belong Fitting: given guess for approximate location, refine itFitting: given guess for approximate location, refine it
Second method efficient for finding Second method efficient for finding unknown lines, but not always accurateunknown lines, but not always accurate
From Szymon Rusinkiewicz, Princeton
Hough TransformHough Transform
General idea: transform from image General idea: transform from image coordinates to parameter space of featurecoordinates to parameter space of feature Need parameterized model of featuresNeed parameterized model of features For each pixel, determine all parameter values that For each pixel, determine all parameter values that
might have given rise to that pixel; votemight have given rise to that pixel; vote
At end, look for peaks in parameter spacAt end, look for peaks in parameter spacee
From Szymon Rusinkiewicz, Princeton
Hough Transform for LinesHough Transform for Lines
Generic line: Generic line: y y = = axax++bb Parameters: Parameters: aa and and bb
From Szymon Rusinkiewicz, Princeton
Hough Transform for LinesHough Transform for Lines
1.1. Initialize table of Initialize table of bucketsbuckets, indexed by, indexed byaa and and bb, to zero, to zero
2.2. For each detected edge pixel (For each detected edge pixel (xx,,yy):):a.a. Determine all (Determine all (aa,,bb) such that ) such that y y = = axax++bb
b.b. Increment bucket (Increment bucket (aa,,bb))
3.3. Buckets with many votes indicate probable linesBuckets with many votes indicate probable lines
From Szymon Rusinkiewicz, Princeton
Hough Transform for LinesHough Transform for Lines
bb
aa
From Szymon Rusinkiewicz, Princeton
Hough Transform for LinesHough Transform for Lines
bb
aa
From Szymon Rusinkiewicz, Princeton
Difficulties withDifficulties withHough Transform for LinesHough Transform for Lines
Slope / intercept parameterization not idealSlope / intercept parameterization not ideal Non-uniform sampling of directionsNon-uniform sampling of directions Can’t represent vertical linesCan’t represent vertical lines
Angle / distance parameterizationAngle / distance parameterization Line represented as (Line represented as (rr,,) where) where
xx cos cos + + yy sin sin = = rr
rr
From Szymon Rusinkiewicz, Princeton
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Finding Lines Using the Hough Finding Lines Using the Hough TransformTransform
sincos ii yx
ii yx ,
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
AlgorithmAlgorithm Discretize the parameter spaces Discretize the parameter spaces ρ and θ.ρ and θ. Create Accumulator array A(1..R,1..T).Create Accumulator array A(1..R,1..T). Set A(k,h)=0 for all k and h.Set A(k,h)=0 for all k and h. For each image edge E(i,j)=1For each image edge E(i,j)=1
For h=1…TFor h=1…T ρ =i cosθρ =i cosθdd(h)+j sinθ(h)+j sinθd d (h)(h) Find index k: ρFind index k: ρd d is closest to ρis closest to ρ Increment A(h,k) by one.Increment A(h,k) by one.
Find all local maxima (kFind all local maxima (kpp, h, hpp) such that A (k) such that A (kpp, h, hpp)>τ)>τ
Hough Transform ResultsHough Transform Results
Forsyth & PonceForsyth & Ponce
Hough Transform ResultsHough Transform Results
Forsyth & PonceForsyth & Ponce
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Finding Lines Using the Hough Finding Lines Using the Hough TransformTransform
Strong local peaks correspond to lines.
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Finding Lines Using the Hough Finding Lines Using the Hough TransformTransform
Resolution Issues
…
Bucket SelectionBucket Selection
How to select bucket size?How to select bucket size?
From Szymon Rusinkiewicz, Princeton
Bucket SelectionBucket Selection
How to select bucket size?How to select bucket size? Too small: poor performance on noisy data, long Too small: poor performance on noisy data, long
running timesrunning times Too large: poor accuracy, possibility of false positivesToo large: poor accuracy, possibility of false positives
Large buckets + verification and refinementLarge buckets + verification and refinement Problems distinguishing nearby linesProblems distinguishing nearby lines
Be smarter at selecting bucketsBe smarter at selecting buckets Use gradient information to select subset of bucketsUse gradient information to select subset of buckets More sensitive to noiseMore sensitive to noise
From Szymon Rusinkiewicz, Princeton
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Hough Transform: ResultsHough Transform: Results
Hough TransformImage Edge detection
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Summary Hough TransformSummary Hough Transform Smart countingSmart counting
Local evidence for global featuresLocal evidence for global features Organized in a tableOrganized in a table Careful with parameterization!Careful with parameterization!
Problem: Curse of dimensionalityProblem: Curse of dimensionality Works great for simple features with 3 unknownsWorks great for simple features with 3 unknowns Will fail for complex objectsWill fail for complex objects
Problem: Not a local algorithm Problem: Not a local algorithm
Hough TransformHough Transform
What else can be detected usingWhat else can be detected usingHough transform?Hough transform?
Hough TransformHough Transform
What else can be detected usingWhat else can be detected usingHough transform?Hough transform?
Anything, but Anything, but dimensionality dimensionality is keyis key
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Finding Circles by Hough TransformFinding Circles by Hough Transform
a0
b0r
(xi,yi)
Equation of Circle: 220
20 )()( rbyax ii
x
y
Finding Circles by Hough TransformFinding Circles by Hough Transform
a0
b0r
(xi,yi)
Equation of Circle: 220
20 )()( rbyax ii
If radius r is known:
x
y
x
y
a
b
(xi,yi)
Circles!
Accumulator array A(a,b)
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Finding Circles by Hough TransformFinding Circles by Hough Transform
a0
b0r
(xi,yi)
x
y
If r is not knownUse accumulator array A(a,b,r)
For each (xi,yi) increment A(a,b,r) such that222 )()( rbyax ii
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Using Gradient InformationUsing Gradient InformationCan save lot of computations!
Given: location (xi,yi)Edge direction φi
x
y
Using Gradient InformationUsing Gradient InformationCan save lot of computations!
Given: location (xi,yi)Edge direction φi
x
y
Assume r is known:
x
y φi
(xi,yi) a=x-rcosφ b=y-rsinφNeed to incrementonly one pointin Accumulator Array.
(a,b)
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Hough Transform for CurvesHough Transform for Curves
Curve y=f(x,Curve y=f(x,aa)) a=a=[a[a11, … , a, … , app] the parameters of the curve.] the parameters of the curve. Limitation: size of parameter space wrt # of Limitation: size of parameter space wrt # of
parameters.parameters. Solution: variable-resolution parameter space.Solution: variable-resolution parameter space.
Hough TransformHough Transform
Pattern Matching.Pattern Matching. More efficient than template matching.More efficient than template matching. Handles occlusion.Handles occlusion. Finds all instances of patternFinds all instances of pattern.. Handling inaccurate edge locations?Handling inaccurate edge locations? Drawbacks?Drawbacks?
FittingFitting
Output of Hough transform often not Output of Hough transform often not accurate enoughaccurate enough
Use as initial guess for fittingUse as initial guess for fitting
Fitting LinesFitting Lines
Initial guessInitial guess
Fitting LinesFitting Lines
Least-squaresLeast-squaresminimizationminimization
Fitting LinesFitting Lines
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Finding LinesFinding Lines
Assume edge detectionAssume edge detection Each pixel is either edge or notEach pixel is either edge or not How do we find the line?How do we find the line?
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Finding LinesFinding Lines
Assume edge detectionAssume edge detection Each pixel is either edge or notEach pixel is either edge or not How do we find the line?How do we find the line?
CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos
Least Squares Fitting of LinesLeast Squares Fitting of Linescmxy
ii yx ,
cmxy ii
i
ii
N
cmxyE
2
Minimize E
0
0
c
Em
E
Least Squares Fitting of LinesLeast Squares Fitting of Lines
i
ii
N
cmxyE
2
Minimize E
0
0
c
Em
E
Problem: E must be formulated carefully!
Least Squares Fitting of LinesLeast Squares Fitting of Lines
ii yx ,
i
ii
N
yxE
2cossin
Minimize E
0
0
E
E
0cossin yx
cossin ii yx