8/15/2019 Dense optical flow expansion based on polynomial basis approximation

1/12

Dense Motion Estimation based

on Polynomialexpansion Pi19404

March 5, 2014

8/15/2019 Dense optical flow expansion based on polynomial basis approximation

2/12

8/15/2019 Dense optical flow expansion based on polynomial basis approximation

3/12

Dense Motion Estimation based on Polynomial expansion

Dense Motion Estimation basedon Polynomial expansion

0.1 IntroductionIn this article we will look at dense motion estimation based onpolymonial repsentation of image.The polynomial basis representationof the image is obtained by approximating the local neighborhoodof image using quadratic polynomial basis.The displacement between adjacent frames can be obtained by equating the coefficients of thebasis.

0.2 Introduction This article describes a fast dense optical flow computation al-

gorithm by Farnebäck, 2002 . In the earlier articles it was seen that a local neighborhood of image can be represented usingpolynomial basis. Using this representation estimation of denseoptical flow is obtained at each point in the image.

Assuming how a polynomial transforms under translation us- ing the polynomial expansion coefficients derived from current and previous frames and estimate of displacement vector is ob- tained.

The idea of polynomial expansion is to approximate a neighbor- hood of a point in a 2D function with a polynomial.Considering quadratic polynomial basis ,pixel values in a neigh-borhood of image is represented by

where A is a symmetric matrix,b is a vector and c is a scalar

The coefficients can be estimated by weighted least square es- timate of pixel values about neighborhood as seen in the earlier

8/15/2019 Dense optical flow expansion based on polynomial basis approximation

4/12

Dense Motion Estimation based on Polynomial expansion

(a) 2D basis function

Figure 1: Polynomial basis

8/15/2019 Dense optical flow expansion based on polynomial basis approximation

5/12

Dense Motion Estimation based on Polynomial expansion

article. As with all optical flow algorithm,the brightness con-stancy assumption is made. The brighness of a path of image in adjacent frames is constant.

Consider a translational motion

ecountered at point

in the image.

equating coefficients in the two polynomials Due to brightness constancy assumption

Assuming is non-singular

Thus by equating the coefficients of the polynomial the dis-placement vector can be obtained at each point in the image assuming there is overlap between the region of interest ie image neighborhood in adjacent frames.

Let us say we have the estimate of displacement We extract the ROI about the neighborhood at point and point

The polynomial basis are extracted and thecomputation that is shown above is performed.

The total displacement can be estimated as

we know , ,

Thus an iterative scheme can be used where in every succes-sive iteration a better estimate of displacement vector is ob- tained.The iterations can be terminated when change is displace-ment vector is below is threshold in successive iterations or

8/15/2019 Dense optical flow expansion based on polynomial basis approximation

6/12

8/15/2019 Dense optical flow expansion based on polynomial basis approximation

7/12

Dense Motion Estimation based on Polynomial expansion

8/15/2019 Dense optical flow expansion based on polynomial basis approximation

8/12

Dense Motion Estimation based on Polynomial expansion

The method EstimateFlow computes the coefficients

required for displacentt field computation.The EstimateFlow func- tions call the method UpdatePoly for each pixel in the image.

The displacement field obtained may be discontinuous and contain noise and other atrifacts.Since it is reasonable to assume that if motion is encounted at a point, the neighborhood pixels also encounter the same motion. The displacement vector can be averaged over a neighborhood to get a better estimate of the displacement field.

8/15/2019 Dense optical flow expansion based on polynomial basis approximation

9/12

Dense Motion Estimation based on Polynomial expansion

The method AverageFlow computes the average of coefficients

and then computes the displacement flow field.

8/15/2019 Dense optical flow expansion based on polynomial basis approximation

10/12

Dense Motion Estimation based on Polynomial expansion

This approach may in case of large displaement.Hence a multi scale estimation is performed. The estimation of flow field is per- formed as the smallest resolution.The displacement computed at the lower resolution is used as estimate for peform displace-ment field computation at higher resolution.

Dense optial flow computed for two frames is shown in figure 2c

(a) Frame 1 (b) Frame 2

(c) Optical Flow

Figure 2: Displacement field

8/15/2019 Dense optical flow expansion based on polynomial basis approximation

11/12

Dense Motion Estimation based on Polynomial expansion

0.3 Conclusion

This article describes the theory and implementation details of the dense optical flow algorithm based on paper by Farnebäck, 2002 .This code for the algorithm can be found at github repos- itory in files DenseOf.cpp and DenseOf.hpp files. In the future article we will look at op- timizing the code using SSE,NEOM and OpenCL optimizations to enable real time computation of the dense optical flow fields

https://github.com/pi19404/OpenVision

8/15/2019 Dense optical flow expansion based on polynomial basis approximation

12/12