measuring flow in digital video containing smoke …424496/fulltext01.pdfhave many dynamic features...

Code: ________________

Faculty of Engineering and Sustainable Development

Measuring flow in digital video containing smoke

and gas

Yunyi Huang

June 2011

Bachelor Thesis, 15 credits, C

Computer Science

Computer Science Program

Examiner: Stefan Seipel

Supervisor: Julia Åhlén

Measuring flow in digital video containing smoke and gas

by

Yunyi Huang

Faculty of Engineering and Sustainable Development

University of Gävle

S-801 76 Gävle, Sweden

Email:

[email protected]

Abstract

Optical flow is widely used in image and video processing. This paper describes the

definition of optical flow and some basic methods of estimating optical flow. There

are three main optical flow methods described in this paper: Horn–Schunck method,

Lucas–Kanade method and Anandan method. It will select the most appropriate

method to measure the flow in videos containing smoke and gas by comparing those

three methods with different criteria. The results are as below 1) Horn–Schunck

method is good at measuring flow in infrared video and in horizontal direction,

when 2) Anandan method is adept in estimating upward optical flow but not suitable

for infrared video. 3) Lucas-Kanade method can be used not only in smoke videos,

but also in methane gas videos, and it can detect flow in both horizontal and vertical

direction.

Keywords: optical flow, video processing, smoke or gas detection

mailto:[email protected]

Contents 1 Introduction ....................................................................................................... 1 1.1 Aim .................................................................................................................................. 2 1.2 Background ..................................................................................................................... 2 1.2.1 Optical flow constraint equation ............................................................................ 2 1.2.2 Aperture problem ................................................................................................... 3 1.2.3 Horn–Schunck method ........................................................................................... 3 1.2.4 Lucas–Kanade method ........................................................................................... 4 1.2.5 Pyramid Lucas-Kanade method ............................................................................. 4 1.2.6 Anandan method .................................................................................................... 5 1.2.7 Angular error ......................................................................................................... 5

2 Comparative method......................................................................................... 6 2.1 Selection of videos .......................................................................................................... 6 2.2 Criteria ............................................................................................................................. 6 2.2.1 Feature Correspondency ....................................................................................... 6 2.2.2 Threshold setting .................................................................................................... 7 2.2.3 Running Time ......................................................................................................... 8 2.2.4 Error measurement ................................................................................................ 8 2.3 Implementation ................................................................................................................ 8

3 Result .................................................................................................................. 8 3.1 Smoke videos .................................................................................................................. 9 3.1.1 Smoke from burning kerosene (video 1) ................................................................. 9 3.1.2 Smoke from burning plastic (video 2) .................................................................. 10 3.1.3 Smoke from burning wood (video 3) ................................................................... 12 3.2 People video (video 4) ................................................................................................... 13 3.3 Methane gas videos ....................................................................................................... 15 3.3.1 Methane gas video (video 5) ............................................................................... 15 3.3.2 Methane gas and people video (video 6) .............................................................. 18

4 Discussion ......................................................................................................... 21

5 Conclusion ........................................................................................................ 22

Acknowledgements ................................................................................................ 22

References ............................................................................................................... 22

1

1 Introduction

In recent years, video surveillance has got the extensive application in many fields

such as law enforcement, security, immigration, correctional institutions, and

protection of the environment [10], etc. Using video surveillance to monitor and detect

the smoke and gas is becoming more and more popular. Conventional smoke detectors

normally detect the presence of combustion products by ionization or photometry

based sensors [7]. But such detectors have some shortcomings. For one thing, sensors

are limited to the placement because they just can be placed on the ceiling. For another,

it is time consuming in outdoor or open spaces because it takes a long time from the

combustion products start burning to the smoke reaches the sensors [7]. Therefore,

instead of using conventional smoke sensor, we can use video surveillance because it

can be placed anywhere indoor or outdoor and do early detection of the smoke from

forest fires.

In addition, video equipment technologies can be used to detect invisible gas as

well as visible smoke. For example, Infrared (IR) high sensitivity equipment can

operate invisible light spectrum, thus IR cameras are used to detect the leakage of gas

and volatile organic compounds (VOCs) including a range of hazardous and

greenhouse gases [6].

Smoke and gas are the flowing characteristic of the suspended aerosol, so they

have many dynamic features and the motion of smoke or gas plays an important role

in video smoke or gas detection, furthermore, motion provides a clue of image

understanding much than intensity, color, texture and others [16]. In order to detect

this motion object, we usually use the approaches of Background Subtraction,

Temporal Difference and Optical Flow. Background subtraction can extract the

dynamic object information easily when the scene of video is static, temporal

difference has strong self-adaptive ability for motion environment, but it cannot obtain

all related characteristic pixels [13].

Optical flow method is widely used as it can get the relation or information of

most pixels in both static and moving video, and the motion objects like smoke and

gas can be detected without any scene information. Nevertheless, optical flow method

is usually used with other approaches in real time video surveillance system on

account of a majority of real time video surveillance basically monitors statically and

optical flow method need complex calculations.

Figure 1. Optical flow [14]

Figure 1 illustrates the concept that 2D representation of optical flow is a perspective

projection of the 3D velocities of surface points onto the image plane, and we will

calculate the optical flow from spatiotemporal patterns of image intensity [9]. There

are many optical flow techniques, and S. S. Beauchemin and J. L. Barron [15] divided

them into six classes, intensity-based differential methods, multi-constraint methods,

http://upload.wikimedia.org/wikipedia/commons/5/55/Opticfloweg.png

http://upload.wikimedia.org/wikipedia/commons/5/55/Opticfloweg.png

2

frequency-based methods, correlation-based methods, multiple motion methods and

temporal refinement methods. Among of them, the intensity-based differential

methods are used mostly and the typical differential methods are Horn–Schunck

method [5] and Lucas–Kanade method [4]. In this paper, I will analyze those two

methods and one of the correlation-based methods called Anandan method for

determining optical flow.

1.1 Aim

The aim of this research is to make a comparison of three main methods of estimating

optical flow, included Horn–Schunck method, Lucas–Kanade method and Anandan

method, with different criteria and test which algorithm is suited to detect smoke or

gas in specific videos in a best way.

1.2 Background

In the background part, the relevant academic background for three estimating optical

flow method – Horn Schunck method, Lucas Kanade method and Anandan method,

and one of the comparison methods will be described in detail.

1.2.1 Optical flow constraint equation

As Andrés Bruhn, et al [1] said many differential methods of optic flow estimation

base on brightness constancy and they suppose that the grey values of image objects

(surface radiance) in subsequent frames is constant. As shown in Figure 2, gray value

of the tracked pixel is unchanged over time.

Figure 2. Brightness constancy

The image constraint equation can be given:

( ) ( ) Eq. (1)

Where it is for a dimensional case, a voxel at location ( ) with gray

value ( ) moves , and between two frames of image, and after

movement, the gray value of that voxel is ( ). If , the

gray values of those two voxels regard as constant.

Then we assume the displacement of the image between two nearby instants is

small and perform a first order Taylor expansion to Equation (1):

( ) ( )

Eq. (2)

3

Due to in small movement, we will get:

Eq. (3)

Let

and

represent the optical flow of ( ) in x and y direction,

and

,

and

represent the derivatives of the image at ( ) in

the corresponding directions. Then the optical flow constraint equation is created

Eq. (4)

1.2.2 Aperture problem

Figure 3. Aperture problem

The aperture problem normally arises from a moving object viewed through a small

window, and it is impossible to determine the true orientation unless the end of the

contour of object becomes visible in the aperture [8]. For example, the object seems to

move from left to right in the first row of Figure 3. However, we can see in the second

row, the true velocity is toward lower right and the component velocity in vertical

direction is ignored. Obviously, the aperture problem has appeared in the Equation (4),

two unknown constants and cannot be computed in single equation, so we need

one more constraint equation.

1.2.3 Horn–Schunck method

Optical flow constraint equation cannot determine the unique optical flow of the

images. Therefore, Horn and Schunck [5] put forward a smoothness constraint, which

is a particular regularization term expressing the dependence and the expected

smoothness of the flow field within the whole image.

From the smoothness constraint Equation (5) and optical flow constraint Equation (6):

∬(

) Eq. (5)

∬( ) Eq. (6)

We can know that optical flow should satisfy the global energy function (7):

∬ ( ) (| | | | ) Eq. (7)

Where the smoothness weight α > 0 is the regularization parameter and the larger the

values α, the stronger penalization of large flow gradients, thus the smoother the flow

fields. In this paper, equals to 1.

4

Equation (7) can be minimized by solving the associated Euler–Lagrange

equations, and then we will obtain the component velocity (8) and (9) at last by

iterating the minimized equation. In this paper, I use 100 iterations:

( )

Eq. (8)

( )

Eq. (9)

Where the superscript point to the next iteration, and k is the result from last

calculated.

1.2.4 Lucas–Kanade method

Different from global constraint in Horn–Schunck method, Lucas and Kanade [4]

solve the aperture problem based on a local constraint. It assumes that the flow is

essentially constant within a local area, and then it will implement a weighted least

square fit by minimizing cost function (10):

∑( ) ∑( ) , Eq. (10)

In order to minimize the cost function, get the derivative of Equation (10) and make it

equal to 0:

∑(

) Eq. (11)

∑( ) Eq. (12)

Equation (11) and (12) can be formulated as a matrix (13)

(∑

∑ ∑ ∑

)( ) (

∑ ∑

) Eq. (13)

Let (∑

∑ ∑ ∑

) (∑

∑ ) , if M is invertible, the optical flow

(velocity) can be obtained as:

Eq. (14)

Otherwise, optical flow is calculated in flat regions where the image gradient vanishes.

1.2.5 Pyramid Lucas-Kanade method

Gaussian Pyramid is a low-pass pyramid, and it is generated by smoothing the images

with a Gaussian filter firstly, and then subsampling the smoothed image by a factor of

two along each coordinates direction, all the smoothed image are subjected to the

same processing and it results in a yet smaller image [12], as Figure 3 shown.

5

g0 = IMAGE

gL = REDUCE [gL-1] Figure 3. Gaussian Pyramid [12]

Where g0 is the original image, g1 is the image after low-pass filter and subsampling

g0. We can get the gL image by processing gL-1 image and this process for each level

is called “reduce”.

In fact Lucas-Kanade method is not efficient because we need large window

which too often breaks the coherent motion assumption to catch large motions and it

means the motions need to be small for the correct motion estimation. Therefore, we

can use Gaussian pyramids (coarse to fine estimation method) to avoid this problem.

We progress flow estimation from coarse to fine levels within Gaussian pyramids of

the two images (current frame and subsequent frame), then warp or interpolate those

fine flow back to the images level by level. In this paper, I will use the pyramid Lucas-

Kanade method instead of Lucas-Kanada method and the pyramid level is three.

1.2.6 Anandan method

Anandan’s method [11] is based on a Laplacian pyramid and coarse-to-fine SSD (sum

of squared difference)-based matching strategy. The Laplacian pyramid is very similar

to Gaussian Pyramid, but it uses a Laplacian transform instead of a Gaussian filter,

because it allows large displacement of object and enhances edges [11]. In this paper,

it still implements the Gaussian Pyramid in Anandan’s method for making good

comparison with pyramid Lucas-Kanade method and the pyramid level is two. The

coarse-to-fine SSD measure is defined as

( ) ∑ ∑ ( )

[ ( ( )) ( ( ))]

( )

( ) ( ) Eq. (15)

Where ( ) represents a discrete two-dimensional window function, ( )

is the velocity, the velocity is limited in the neighboring square region which size is

( ) and centered at two-dimensional value x.

1.2.7 Angular error

In most articles with regard to optical flow estimation [1] and [9], angular deviations

are used to measure the errors. As written in [9], we present the velocity as 3-D

direction vectors instead of 2-D, thereby the velocities are written as:

√( )( ) Eq. (16)

The angular error between the correct velocity and an estimate velocity is:

( ) Eq. (17)

6

Barron et al. [8] also point out this error measure is convenient because it can handle

both high and very low speeds without the amplification inherent in a correlative

measure of vector differences, but it still has some bias.

2 Comparative method

In this chapter, I describe the selection of videos which contain smoke and gas. In

addition, it is going to show a series of criteria or standards for comparisons among

three different estimating optical flow methods, because the comparisons are based on

primarily empirical test, and then concentrate on the accuracy reliability.

2.1 Selection of videos

Six videos will be selected as input material for comparison. Three videos contain

smoke, two contain methane gas and one contains only people. The people video and

three smoke videos are in color and the scenes are static. The smoke in smoke videos

is from burning different combustion products and which means that the conditions of

combustion are different. Two methane gas video are in gray scale because they are

infrared videos. The scenes are moving and one video has not only methane gas but

also moving people. In every video, I will choose ten frames to do the comparison in

order to obtain more accurate values.

2.2 Criteria

There are four criteria for comparison among three different optical flow estimation

methods and they are: feature correspondency, threshold setting, running time and

error measurement. The reasons for choosing those criteria are as follows, firstly, the

comparisons are based on observation and the direction of optical flow can be seen

easily. Thus we can compare different methods by observing if the optical flows

which are estimated by different methods are corresponding with the features.

Secondly, as there are many moving objects in the videos, some optical flows are not

from the smoke or gas. In order to select the right optical flows, it is necessary to set

the threshold. Furthermore, the running speed is also important when comparing

algorithms. We will calculate the running time of each method in different videos.

Last but not least, following the scientific researchers who estimate or perform the

optical flow methods [1], [9], I will also calculate the angular error between the

estimated optical flow and correct optical flow. Better method with less error,

therefore, the error measurement will be the last criteria in my comparison.

2.2.1 Feature Correspondency

Generally speaking, smoke and gas move upward and slowly, the shape of them will

change with the passing of time. We will use these characteristic to separate the smoke

from other motion object. For example, people will move back and forth and usually

move faster than smoke. Moreover, the shape of people will not change. Thus, the

features of smoke and gas will serve as the standard, and by comparing each method

with this criterion, we can find out which method is more suitable to estimate optical

flow of irregularly shaped object, which method is in reverse good at estimating

optical flow for unchanged shape object. We can also find out the method is more

appropriate for detecting object moves upward and in a slow pace. However, the

judgment which is based on this criterion may not correct when the scenes is moving.

Because when the scene is static, the static objects in videos should have not optical

flow, the apparent optical flow must from the moving object such as smoke or gas.

7

But if the scene is moving downward, all the things inside the scene are moving

upward and optical flows are not totally from smoke or gas. In this case, we cannot

make a correct judgment for which method is better. Therefore, a new criterion

appears to solve the problem that estimating optical flow in dynamic scene videos, it is

threshold.

2.2.2 Threshold setting

In dynamic scene videos, the threshold value will be set for removing the interference

from moving scene. Interference denotes the optical flows which are produced by the

moving scene and it will hinder the comparison. I have set two thresholds, one is used

to determine which flow is interference, and the other is used to reduce the

interference.

Eq. (18)

Eq. (19)

Where R is the reduce value. Usually is between 2 to 3, it means if the

current flow is larger than half or one third of max flow, it should be the interference.

is from 0.5 to 0.9, we reduce the interference by subtract a part of max

value.

The details of how to reduce interference are as follow. First of all, calculate the

average value among all optical flows. Because most optical flows presented in

moving scene are interferences and average value tends to most value, so the direction

of average value are the same as the directions of interferences. And if the direction of

one optical flow is not the same as the direction of average value, we can eliminate

this optical flow as interference. Otherwise, it can be the interference and goes to the

next step.

We find out the maximum value among the optical flows which has the same

direction as average value. Here is an example to calculate the maximum value of

component velocity u:

if ((av_u/Ax(i,j)) > 0 && abs(Ax(i,j)) > abs(max_u))

max_u = Ax(i,j);

Where av_u is the average value of component velocity u, Ax (i, j) is current

component velocity u in i row and j column, max_u is the maximum value of

component velocity u and abs is method to get the absolute value.

Use maximum velocity in Equation (18), if the currently velocity is satisfied with

Equation (18), it should be regarded as interference. The maximum value will be used

in classification, because I find that most values of interferences are close to the value

of maximum velocity which has the same direction as average velocity.

Lastly, we can reduce the interference in two cases. One is the easier case that the

interference is just in one direction (horizontal or vertical direction) and one of the

component velocities is equal to zero. The other is more difficult that the interference

is slanting and the values of both component velocities are not equal to zero. And then

use Equation (19) in two different cases. In case one, we just use Equation (19) to

reduce the interference in one component velocity. In case two, we need to reduce the

interference in both two component velocities. Here is the example to reduce the

interference in horizontal direction and it just need to calculate the correct component

velocity u, velocity v is equal to zero:

8

if ((Ax(i,j)-sub_u)/Ax(i,j)<0)

Ax(i,j)=0;

else

Ax(i,j)=Ax(i,j)-sub_u;

Where sub_u is R in Equation (19). If after reduce sub_u, current component velocity

Ax(i,j) will change its direction, Ax(i,j) will be set to zero. Otherwise, Ax(i,j) is equal

to the difference between current component velocity and reduce number.

2.2.3 Running Time

In real time video surveillance, quick detection of results pays an important role.

Calculate the running time in an easy way for each frame, get the CPU (central

processing unit) time before and after running the optical flow method, then subtract

the former time from the latter time and we will get the running time, the average

running time is the sum of each frame divided by the number of frames, the pseudo

code is following.

Running_time = 0;

for k = 1 : nFrames

gettime = cputime;

…… (run the function)

Running_time = Running_time + cputime - gettime;

end

Runnning_time = Running_time / (nFrames);

Where nFrames is the number of frames.

2.2.4 Error measurement

Angular measure of error will be used as a criterion in this paper, the average angular

error and standard deviation between the correct velocity and an estimate velocity will

be calculated. A smaller standard deviation represents those angular errors that close

to the average angular error. It is a good way to compare three different optical flow

estimation methods for better estimation method will provide less angular error.

However, it is very difficult to find the correct velocity. Therefore, the estimate

velocity from each method will be considered as correct velocity once, and calculate

the average angular error with the estimate velocity from another method and itself.

The reason why to calculate the average angular error between the method and itself is

that the result of it is not equals to zero and it can be a reference. But during the

comparison, this result should be ignored because it holds lowest angular error.

2.3 Implementation

All the optical flow methods and the comparative methods will be implemented by

using MATLAB®.

3 Result

In this section, I will classify the result into three different types - the smoke type,

people type and methane gas type. In smoke type, there are three different videos with

different kind of smokes, in methane gas type, there are two different videos, and in

9

people type, just one video, but one of the methane gas videos also contains people. In

addition, only the methane gas videos use threshold values for dealing with the

moving scene.

In each result part of videos, it has at least one table and three images. The tables

will show the average running time of each method for different methods. At the same

time, the average error and standard deviation between the reference method and the

other method will be presented in the tables. Optical flows which are estimated by

three methods will be displayed by images.

3.1 Smoke videos

3.1.1 Smoke from burning kerosene (video 1)

Table 1. Comparison based on video 1, frame from 53 to 62

Reference Method Average Running

Time/s Method

Average

Error/degree

Standard

Deviation/degree

Horn–Schunck method

(iteration = 100) 1.398438

Horn–Schunck

method 37.9095 29.7522

Lucas–Kanade

method 54.2633 24.7589

Anandan method 57.6292 26.5610

Lucas–Kanade method

(pyramid level = 3) 22.257813

Horn–Schunck

method 62.6600 29.6502

Lucas–Kanade

method 50.6761 37.3537


Anandan method


Horn–Schunck

method 60.3821 33.1226

Lucas–Kanade

method 58.8111 35.4061


It is clear from Table 1 that Horn–Schunck method is the fastest method and Anandan

method is the slowest one even though it just implements two level of pyramid

compared with Lucas–Kanade method which has implemented three level of pyramid.

Besides, Lucas–Kanade method holds least error and standard deviation and Anandan

method holds most in this case.

Figure 4(1). Frame 62 with Horn–Schunck method in video 1

10

Figure 4(2). Frame 62 with Lucas–Kanade method in video 1

Figure 4(3). Frame 62 with Anandan method in video 1

From Figure 4(1-3), we can see that the result of both the Lucas–Kanade method and

the Anandan method are better than the result of the Horn–Schunck method in this

video.

3.1.2 Smoke from burning plastic (video 2)



Time/s Method

Average

Error/degree

Standard

Deviation/degree


(iteration = 100) 1.857813

Horn–Schunck

method 23.5764 24.2645

Lucas–Kanade

method 34.6794 25.1356




Horn–Schunck

method 49.8003 28.8337

Lucas–Kanade

method 38.6974 33.5852


Anandan method


Horn–Schunck

method 21.9047 28.8409

Lucas–Kanade

method 28.8409 31.8254


11

Different from Table 1, Lucas–Kanade method has most error and standard deviation

in Table 2. And Horn-Schunck method has least error and standard deviation.




12

Anandan method (Figure 5(3)) gives a better result than any other method in video 2.

3.1.3 Smoke from burning wood (video 3)



Time/s Method

Average

Error/degree

Standard

Deviation/degree


(iteration = 100) 1.873438

Horn–Schunck

method 6.7525 14.3249

Lucas–Kanade

method 13.0921 19.5246




Horn–Schunck

method 26.8236 25.4911

Lucas–Kanade

method 20.4840 25.4774


Anandan method


Horn–Schunck

method 4.7083 14.0971

Lucas–Kanade

method 8.8684 19.4247




13


All the results of video 3 including Table and Figures are similar to the results of video2. I

can draw a short conclusion by analyzing the results of three videos above. Above all, Horn–

Schunck method runs fastest and usually has least error and standard deviation, but the flows

from it are not represented in a good way. On the contrary, although Anandan method needs

more time to run, the results of it are presented very well. Meanwhile, Lucas–Kanade

method is situated between of them with the mean values of both running time and results.

3.2 People video (video 4)



Time/s Method

Average

Error/degree

Standard

Deviation/degree


(iteration = 100) 1.455966

Horn–Schunck

method 40.0045 29.3470

Lucas–Kanade

method 51.5040 24.4146




Horn–Schunck

method 60.7707 30.0135

Lucas–Kanade

method 49.2712 37.1465


Anandan method


Horn–Schunck

method 58.3453 37.3441

Lucas–Kanade

method 61.0444 41.2836


The values for average error and standard deviation from Anandan method are larger

than the other two methods as indicated in Table 4.

14




Figure 7 (1 to 3) show that Horn–Schunck method and Lucas–Kanade method can

estimate the flows in horizontal direction in a good way, while Anandan method

presents some wrong flows in that direction, and we can compare the flows within the

red circle in figures.

15

3.3 Methane gas videos

Because methane gas cannot be seen with the naked eye, we use infrared videos to observe

methane gas and they are dealt with in form of gray scale images. Different from the videos

above, the scenes of infrared videos below are moving.

3.3.1 Methane gas video (video 5)



Time/s Method

Average

Error/degree

Standard

Deviation/degree


(iteration = 100) 0.837500

Horn–Schunck

method 13.7408 17.7958

Lucas–Kanade

method 21.3343 20.8581




Horn–Schunck

method 37.7151 29.2655

Lucas–Kanade

method 30.1217 31.4389


Anandan method


Horn–Schunck

method 16.7331 25.4262

Lucas–Kanade

method 22.2880 33.4000


Horn–Schunck method has less error and at the same time, Lucas–Kanade method has

most error as showed in Table 5.


16



The scene of this video is moving toward right, so it leads to the flows of most objects

point to left. However, we can focus on the Figure 8 (3) and the flows which are

estimated by Anandan method are in wrong direction. This kind of mistake occurred

in video 4 above.

Apparently, what we need to do is to reduce the disturbance from moving scene

and achieve the right optical flows. Therefore, we need to remove the obstacles by

setting the threshold value. The results below are the right flows after process with

threshold.

17

Figure 9(1). Frame 805 with Horn–Schunck method in video 5 after reduce interference

First threshold value is 2, second value is 0.9

Figure 9(2). Frame 805 with Lucas–Kanade method in video 5 after reduce interference

First threshold value is 2, second threshold value is 0.9

18

Figure 9(3). Frame 805 with Anandan method in video 5 after reduce interference


Both Horn–Schunck method and Lucas–Kanade method estimate good optical flows

except for Anandan method. The first threshold value of Anandan method is four

while the first threshold values of Horn–Schunck method and Lucas–Kanade method

are just two, it means the influence of interference is large in Anandan method.

3.3.2 Methane gas and people video (video 6)



Time/s Method

Average

Error/degree

Standard

Deviation/degree


(iteration = 100) 1.312500

Horn–Schunck

method 13.7839 14.3898

Lucas–Kanade

method 21.7770 16.9393




Horn–Schunck

method 43.5916 29.4502

Lucas–Kanade

method 35.5985 33.7140


Anandan method


Horn–Schunck

method 16.7953 19.0250

Lucas–Kanade

method 20.9404 22.9143


Table 6 indicates that Anandan method has least error and Horn–Schunck method has

most error in this video

19




20

As we can see, from frame 53 to frame 54 in video 6, the background is moving

downward, and it leads to all the objects in screen are moving upward. We use

threshold value to get the right optical flows. The results are as below:

Figure 10(1). Frame 53 with Horn–Schunck method in video 6 after reduce interference


Figure 10(2). Frame 53 with Lucas–Kanade method in video 6 after reduce interference


21

Figure 10(3). Frame 53 with Anandan method in video 6 after reduce interference


From Figure 10 (1 to 3), we realize that Horn–Schunck method gives most correct

optical flows, but there is just one optical flow left after reduce interference in

Anandan method.

4 Discussion

After compare different optical flow method, we can realize those kinds of methods

can be used in video processing in a good way, and I would like to discuss the pros

and cons of those three methods. From the result, I find that Horn–Schunck method,

which is a fast method, is very suitable to estimate the optical flow of objects whose

gray value is close to background’s gray value. In other words, it is good to be used in

infrared videos. The advantage of Anandan method is that estimating the optical flow

from the upward moving objects very easily. However, it wastes time and it is not

suited for infrared videos. Chen Bangzhong etc. [3] held the same opinion that in

infrared image sequences, Horn-Schunck method has the best performance while

Anandan has the worst among six different optical flow methods in their paper. The

results of the Lucas–Kanade method are relatively steady and its advantage is that it

can measure the flows in every direction. However, the effect figures of Lucas–

Kanade method are not better than the figures of Anandan method in smoke video and

in infrared videos, Horn–Schunck method can be used more suitably than Lucas–

Kanade method.

The result seems to be satisfying, but the precision of it still need to be improved.

For one thing, the running time of each method highly depends on the code of

program or we can say the time complexity of each algorithm in each optical flow

method, because different algorithms can be realized in different ways and it is

difficult to find out the most timesaving way. For example, Chen Bangzhong etc. [3]

presented the calculation quantities of Lucas–Kanade method are the least, while

Anandan method are the largest and Horn–Schunck method is between those two

22

methods. But in my paper, Horn–Schunck method run fastest while Anandan method

run the slowest and Lucas–Kanade method is between them. For the other thing,

although calculate the angular error is a nice approach to compare different optical

flow method, finding out the reference of correct velocity is demanding. Scientific

researchers usually use this method when he or she has the correct velocity of the

video, such as [1] and [9].

5 Conclusion

The conclusion I draw from my research is that it seems like the best choice is the

Anandan method when measuring the flow in general digital video containing smoke.

Because the results of Anandan method show that this method is very good at

estimating the optical flow of smoke, although it runs a long time.

However, if the general digital video contains people as well as smoke, my

suggestion is to use Lucas–Kanade method instead of Anandan method. Combined the

smoke videos with the people video, it is obvious that the Anandan method can

estimate the optical flow from the upward moving objects easily, while Horn–Schunck

method is more suited to detect the objects which move back and forth and Lucas–

Kanade method can estimate the optical flow in any direction.

Lastly, if the video is an infrared video then the results in my paper leans more at

that the Horn-Schunck method should be a better method. Because Horn–Schunck

method is best with fast running time, good results and with least angle error in

methane gas videos (infrared videos). It is seems to very hard for Anandan method to

estimate the optical flow in infrared videos. And Lucas–Kanade method is between

them.

Acknowledgements

I would like to thank my supervisor Julia for her valued support and encouragement in

the past few weeks. I also appreciate my examiner Stefan who provides precious

views and suggestions for this paper.

References

[1] A. Bruhn, J. Weickert, and C. Schnörr , “Lucas/Kanade meets Horn/Schunck:

combining local and global optic flow methods”, International Journal of

Computer Vision (IJCV), 61(3), February 2005, pp. 211–231.

[2] A. Safitri, X. Gao and M. S. Mannan, “Dispersion modeling approach for

quantification of methane emission rates from natural gas fugitive leaks detected

by infrared imaging technique”, Journal of Loss Prevention in the Process

Industries, 24(2), March 2011, pp. 138-145.

[3] B. Chen, X. Wang, J. Tu and Y. Peng, “Moving Target Detection from Infrared

Image Sequences”, Journal of Detection & Control, 24(3) , 2002 ,pp. 11-13,20

[4] B. D. Lucas and T. Kanade, “An Iterative Image Registration Technique with an

Application to Stereo Vision”, Proceedings of the 1981 DARPA Image

Understanding Workshop, April 1981, pp. 121-130.

[5] B. K. P. Horn and B. G. Schunck, “Determining Optical Flow”, Artificial

Intelligence, 17(1-3), 1981, pp. 185-203.

[6] FLIR Systems AB, “IR camera detects gas leaks and measures temperature”,

Sealing Technology, 2009(8), August 2009, p. 12.

http://caod.oriprobe.com/journals/tcykzxb/%e6%8e%a2%e6%b5%8b%e4%b8%8e%e6%8e%a7%e5%88%b6%e5%ad%a6%e6%8a%a5.htm

23

[7] F. Yuan, “Video-based smoke detection with histogram sequence of LBP and

LBPV pyramids”, Fire Safety Journal, 46(3), April 2011, pp. 132-139.

[8] I. Murakami, “The aperture problem in egocentric motion”, Trends in

Neurosciences, 27(4), 2004, pp. 174-177.

[9] J. L. Barron, D. J. Fleet and S. S. Beauchemin, “Performance of optical flow

techniques”, International Journal of Computer Vision, 12(1), 1994, pp.43-77.

[10] M. Nieto, K. Johnston-Dodds and C. W. Simmons, “Public and Private

Applications of Video surveillance and Biometric Technologies, California

Research Bureau”, California State Library, California Research Bureau, 2002.

[11] P. Anandan, “A computational framework and an algorithm for the measurement

of visual motion”, International Journal of Computer Vision, 2(3), 1989, pp.283-

310.

[12] P. Burt, and E. Adelson, “The Laplacian Pyramid as a Compact Image Code”,

IEEE Transactions on Communications, 31(4), Apr 1983, pp.532 – 540.

[13] Q. Luo, N. Han, J. Kan and Z. Wang, “Effective Dynamic Object Detecting for

Video-based Forest Fire Smog Recognition”, Image and Signal Processing, 2009.

CISP '09. 2nd International Congress on, 2009, pp.1-5.

[14] S. J. Huston and H. G. Krapp, “Visuomotor Transformation in the Fly Gaze

Stabilization System”, PLoS Biol, 6(7), 2008, p.173

[15] S. S. Beauchemin and J. L. Barron, “The computation of optical flow”, Journal

ACM Computing Surveys (CSUR), 1995, pp.433-467

[16] Z. Pan and J. J. Pfeiffer, Jr., “Adaptive estimation of optical flow from general

object motion”, Proceeding SAC '92 Proceedings of the 1992 ACM/SIGAPP

Symposium on Applied computing: technological challenges of the 1990's, 1992.

http://portal.acm.org/author_page.cfm?id=81100028380&coll=DL&dl=ACM&trk=0&cfid=22387499&cftoken=37151377

measuring flow in digital video containing smoke …424496/fulltext01.pdfhave many dynamic features...

Documents