the university of ontario cs 433/653 algorithms for image analysis segmentation (2d)...
TRANSCRIPT
The University of
Ontario
CS 433/653
Algorithms for Image Analysis
Segmentation (2D)
Acknowledgements: Alexei Efros, Steven Seitz
The University of
Ontario
CS 433/653 Algorithms for Image Analysis Segmentation (2D)
Blobs• Need for blobs• Extracting blobs
– Thresholding, region growing, mean-shift– Minimum spanning trees
Object extraction• Intelligent scissors (also known as live-wire)• Snakes• Region+boundary methods
Local, greedy, and global processing
Extra Reading: Sonka at.al. Ch. 5 Gonzalez and Woods, Ch. 10
The University of
OntarioHW assignment 1
2D image segmentation• Live-wire (based on Dijkstra algorithm for shortest
paths on a graph)• Implement and experiment on two images• Bonus for comparing with local/greedy methods• Some useful code libraries and sample images will
be posted within a couple of days
The University of
OntarioGoal: Extract “Blobs”
What are “blobs”?• Regions of an image that are somehow coherent
Why?• Object extraction, object removal, compositing, etc.• …but are “blobs” objects?• No, not in general
The University of
OntarioBlob’s coherence
Simplest way to define blob coherence is as similarity in brightness or color:
The tools become blobsThe house, grass, and sky make different blobs
The University of
OntarioThresholding
Basic segmentation operation:mask(x,y) = 1 if im(x,y) > T mask(x,y) = 0 if im(x,y) < T
T is threshold• User-defined• Or automatic
Same as histogram partitioning:
The University of
OntarioRegion growing
Is this same as global threshold?
• Start with initial set of pixels K (initial seed(s))• Add to pixels p in K their neighbors q if |Ip-Iq| < T • Repeat until nothing changes
The University of
Ontario
What can go wrongwith region growing ?
Region growth may “leak” through a single week spot
in the boundary
The University of
OntarioRegion growing
See region leaks into sky due to a weak boundary
between them
The University of
Ontario
Due to Pedro Felzenszwalb and Daniel Huttenlocher
Motivating example
This image has three perceptually distinct regions
Difference along border between A and B is less then differences
within C
A CB Q: Where would image thresholding fail?
Q: Where would region growing fail?
A: Region A would be divided in two sets and region C will be split into a large number of arbitrary small subsets
A: Either A and B are merged or region C is split into many small subsets Also, B and C are merged
The University of
OntarioColor-Based Blob Segmentation
Automatic Histogram Partitioning• Given image with N colors, choose K• Each of the K colors defines a region
– not necessarily contiguous
• Performed by computing color histogram, looking for modes
• This is what happens when you downsample image color range, for instance in Photoshop
The University of
OntarioFinding Modes in a Histogram
How Many Modes Are There?• Easy to see, hard to compute
The University of
OntarioMean Shift [Comaniciu & Meer]
1. Initialize random seed, and fixed window
2. Calculate center of gravity ‘x’ of the window (the“mean”)
3. Translate the search window to the mean
4. Repeat Step 2 until convergence
Iter
ativ
e M
ode
Sea
rch
x
o
x x
mode
The University of
OntarioMean-shift results
More Examples: http://www.caip.rutgers.edu/~comanici/segm_images.html
The University of
OntarioIssues:
Although often useful, all these approaches work only some of the time, and are considered rather “hacky”.
Can’t even handle our tiger:
Problem is that blobs != objects!
The University of
OntarioExtracting objects
Many approaches proposed• color cues• region cues• contour cues
We will consider a few of these Today:
• Intelligent Scissors (contour-based)– E. N. Mortensen and W. A. Barrett, Intelligent Scissors for Image Composition, in
ACM Computer Graphics (SIGGRAPH `95), pp. 191-198, 1995
The University of
OntarioIntelligent Scissors
Approach answers a basic question• Q: how to find a path from seed to mouse that follows object boundary
as closely as possible?• A: define a path that stays as close as possible to edges
The University of
OntarioIntelligent Scissors Basic Idea
• Define edge score for each pixel– edge pixels have low cost
• Find lowest cost path from seed to mouse
seed
mouse
Questions• How to define costs?• How to find the path?
The University of
OntarioPath Search (basic idea) Graph Search Algorithm
• Computes minimum cost path from seed to all other pixels
Note: diagonal “paths” are scaled by a factor .
2 … Why?
The University of
Ontario
Good segmentation method shouldhave Invariance to Image Rotation
After object rotation
L
L
Path’s cost along the top boundary Path’s cost along the top-left boundary
147*2 105*2 After diagonal links are adjusted by2
145*2*2
2 2 2 2 2 2 2
2
2
2
2
2
image gradient scores
The University of
Ontario
Instead of nodes, image gradient scores can be assigned directly to graph edges
Graph• node for every pixel p• link between every adjacent pair of pixels, p,q• cost c for each link
Note: each link has a cost• this is a little different than the figure before
where each pixel (graph node) had a cost
p
qc
Treat the image as a graph
The University of
OntarioDefining the costs
Treat the image as a graph
Want to hug image edges: how to define cost of a link?
p
qc
• the link should follow the intensity edge– want intensity to change rapidly to the link
• c - |difference of intensity to the link|
T
T
The University of
OntarioDefining the costs
p
qc
c can be computed using a cross-correlation filter• assume it is centered at p
Also typically scale c by its length• set c = (max-|filter response|)
– where max = maximum |filter response| over all pixels in the image
The University of
OntarioDefining the costs
p
qc 1
-1w-1-1
11
c can be computed using a cross-correlation filter• assume it is centered at p
Also typically scale c by its length• set c = (max-|filter response|)
– where max = maximum |filter response| over all pixels in the image
Why no scalar?2
The University of
Ontario
(see Cormen et.al. “Introduction to Algorithms”, p.595) Dijkstra’s shortest path algorithm
0531
33
4 9
2
link cost
1. init node costs to , set p = seed point, cost(p) = 02. expand p as follows:
for each of p’s neighbors q that are not expanded• set cost(q) = min( cost(p) + cpq, cost(q) )
ALG
OR
ITH
M
The University of
OntarioDijkstra’s shortest path algorithm
4
1 0
5
3
3 2 3
9
531
33
4 9
2
11
1. init node costs to , set p = seed point, cost(p) = 02. expand p as follows:
for each of p’s neighbors q that are not expanded• set cost(q) = min( cost(p) + cpq, cost(q) )
– if q’s cost changed, make q point back to p• put q on the ACTIVE list (if not already there)
ALG
OR
ITH
M
The University of
OntarioDijkstra’s shortest path algorithm
4
1 0
5
3
3 2 3
9
531
33
4 9
2
15
233
3 2
4
1. init node costs to , set p = seed point, cost(p) = 02. expand p as follows:
for each of p’s neighbors q that are not expanded• set cost(q) = min( cost(p) + cpq, cost(q) )
– if q’s cost changed, make q point back to p• put q on the ACTIVE list (if not already there)
3. set r = node with minimum cost on the ACTIVE list4. repeat Step 2 for p = r
ALG
OR
ITH
M
The University of
OntarioDijkstra’s shortest path algorithm
3
1 0
5
3
3 2 3
6
531
33
4 9
2
4
3 1
4
52
33
3 2
4
1. init node costs to , set p = seed point, cost(p) = 02. expand p as follows:
for each of p’s neighbors q that are not expanded• set cost(q) = min( cost(p) + cpq, cost(q) )
– if q’s cost changed, make q point back to p• put q on the ACTIVE list (if not already there)
3. set r = node with minimum cost on the ACTIVE list4. repeat Step 2 for p = r
ALG
OR
ITH
M
The University of
OntarioDijkstra’s shortest path algorithm
3
1 0
5
3
3 2 3
6
531
33
4 9
2
4
3 1
4
52
33
3 2
4
2
1. init node costs to , set p = seed point, cost(p) = 02. expand p as follows:
for each of p’s neighbors q that are not expanded• set cost(q) = min( cost(p) + cpq, cost(q) )
– if q’s cost changed, make q point back to p• put q on the ACTIVE list (if not already there)
3. set r = node with minimum cost on the ACTIVE list4. repeat Step 2 for p = r
ALG
OR
ITH
M
The University of
OntarioPath Search (basic idea)
A
B
Dijkstra algorithm
- processed nodes (distance to A is known)- active nodes (front)
- active node with the smallest distance value
The University of
OntarioDijkstra’s shortest path algorithm
Properties• It computes the minimum cost path from the seed to every
node in the graph. This set of minimum paths is represented as a tree
• Running time, with N pixels:– O(N2) time if you use an active list– O(N log N) if you use an active priority queue (heap)– takes < second for a typical (640x480) image
• Once this tree is computed once, we can extract the optimal path from any point to the seed in O(N) time.
– it runs in real time as the mouse moves
• What happens when the user specifies a new seed?
The University of
OntarioLivewire extensions
Directed graphs Restricted search space
• Restricted domain (e.g. near a priori model)• Restricted backward search
Different edge weight functions• Image-Edge strength• Image-Edge Curvature • Proximity to known approximate model/boundary
Multi-resolution processing
The University of
OntarioResults
http://www.cs.washington.edu/education/courses/455/03wi/projects/project1/artifacts/index.html