video segmentation
Post on 08-Aug-2015
210 Views
Preview:
TRANSCRIPT
Video SegmentationLabeling independently moving image regions
2
Motion SegmentationForeground (object) and Background (noise)
Result could be a Binary image, containing foreground onlyProbability image, containing the likelihood of each
pixel being foreground
ApproachesMotion-based (optical flow)Color-basedTexture-based
Motion (Change) Detection
Static camera : changed and unchanged regions.Moving camera : global and local motion regions.Limitations associated with motion estimationAperature problem : pixels in a flat image region
may appear stationary even if they are moving as a result of an aperture problem (hence the need for hierarchical methods)
Occlusion Problem : erroneous labels may be assigned to pixels in covered or uncovered image regions as a result of an occlusion problem.
Motion Detection: using two frames : Image Differencing
5
Subtract ImageCompute pixel-wiseSubtract previous image from input image:
Usually the absolute distance is applied
),(),(),( yxByxIyxF
),( yxF
1. Save image in last frame2. Capture camera image3. Subtract image4. Threshold5. Delete noise
1
2
3
4
5
6
7
8
1
2
5
4
3
6
7
8
9
10
7
ThresholdDecide, when a pixel is supposed to be
considered as a background pixel, or when it is to be considered as a foreground pixel:
Pixel is foreground pixel, if
Pixel is background pixel, if
Problem: What TH?!?
),( yx
THyxF ),(),( yx
THyxF ),(
1. Save image in last frame2. Capture camera image3. Subtract image4. Threshold5. Delete noise
1
2
3
4
5
6
7
8
125 4 3678
Frame no. 1 2 3 4 5 6 7 8 9
x 154 160 160 160 157 130 112 79 0
y 13 19 19 19 16 19 19 19 0
w s s w w w w w
Dominant Motion : no optical flowSpatio temporal intensity gradient.Dominant motion segmentation
fitting a single parametric motion modelpartition the frame in two pixel groupsRepeat step 1 only to well represented pixels
group.
ThresholdingNoise Removal Median Filter
Image subtraction
Bounding Box Centroid
Multiple Motion SegmentationMultiple motion models compete against each
other at each decision site. They consist estimating:Motion within each region (motion model)spatial support of each regionnumber of regions.
The problem : associate each pixel to the right motion model, while simultaneously estimating motions and supports. Clustering (K-means, hough transform),Maximum Likelihood (ML) (pixel based) and Maximum A Posteriori probability (MAP).Region based label assignment
Optical flow estimationmotion vectors at each frame
Thresholding Morphological closing on the motion
vectors
Object tracking using kalman filters (prediction)
15
Motion ModelPredicted position at time t:
Brownian Motion: According to a Gaussian model
0’th order:
1’th order:Similar for y
2’th order Similar for y
),( tt yx
),(),( 11 tttt yxyx
1 at time in velocity :1
11
t-xx
xtxx
t
ttt
1 at time in on accelerati :2
1
1
112
1
t-xx
xtxtxx
t
tttt
16
Color Based segmentation : Background Subtraction
Use Neighborhood relation!! Compare pixel with its neighbors!! Weight them!!
Learn the background and its variations!!E.g. Gaussian models (mean,var) for each pixel!!!E.g. a Histogram for each Pixel The more images you train on the better!! Algorithm:
Consider each pixel (x,y) in the input image and check, how much it varies with respect to the mean and variance of the learned Gaussian models?
1. Calculate mean and variance for each pixel
2. Capture camera image3. Subtract image (= motion)4. Weight the distances (new)5. Threshold according to variance6. Delete noise
Color Segmentation with Histograms
18
Color Segmentation with Histogramsbrightness
19
Color Segmentation with Gaussian Distribution
N(m, C)
Thank you
top related