Rapid Classification Based Pedestrian Detection in

Changing Scenes Zhong Wang, XianBin Cao

College of Computer Science & Technology University of Science and Technology of China

zhongw@mail.ustc.edu.cn xbcao.ustc.edu.cn

Abstract-How to adapt to changing scenes in pedestrian detection is a difficult problem in visual monitoring. This paper

proposed a pedestrian detection method in changing scenes. Response to the requirements of high detection speed and high detection rate of pedestrian detection method in changing scenes, this paper mainly consists of two parts: (1) proposing a general

ternary classification framework. It is based on cascade classification framework and each stage is a ternary detection pattern, that is, through comparing stage threshold to exclude current pedestrians or non-pedestrians object and objects which is difficult determine will enter the next layer filtering. Such detection framework is faster than traditional method and is suitable for real time pedestrian detection system. (2) Considering the above mentioned detection framework relies on

thresholds, the parameters of cascade classifier which trained in old scene require adaptive adjustment in a new scenario. We design a pedestrian method in changing scenes, using a small amount of data in new scene to assist the old scene classifier,

taking cross entropy method to quickly optimizing these parameters combination so that the optimized classifier can be better adapt to pedestrian detection in changing scenes. The new classifier can receive high detection rate and high detection speed.

Taking AHHF dataset as an old scene and NICTA dataset as the new scene, experiments show that the proposed method can apply to pedestrian detection in new scene and obtain good results.

K�ords-pedestrian detection, cross entropy method,

classifier, changing scenes, detection speed.

I. INTRODUCTION & RELATED WORK

The key of pedestrian detection system is to detect pedestrians quickly and accurately. Owing to the complex changing scenes in pedestrian detection, adaptive pedestrian detection method in changing scenes has certain research value.

At present there are many pedestrian detection methods in particular scenes, such as image processing [2, 3], classification [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]. Specifically in classification, there are single-classifier, for example, classifier based on neural works [4], Radial Basis Function [5], Support Vector Machines [6, 7]. There is also cascaded classifier, Voila et al. [8, 9, 10] fIrstly proposed a cascaded classifier with good classification results and received wide applications, for example, [11, 12, 13] were also based on AdaBoost cascaded classifier.

A classifier which trained in a particular scene detects a new scenario, if two scenes are similar and the result is acceptable, but in practical applications, we will face with obvious changing scenes in pedestrian detection. If the old classifier without retraining, the result will be ineffective. But

978-1-4244-6588-0/10/$25.00 ©201 0 IEEE

in the new scene, a large number of labeled data is difficult to obtain, so retraining is not unrealistic. In other words, there are two kinds of approaches available for detection in changing scenes. Firstly, we use the trained classifier directly. The result will decrease such as detect rate and false positive rate when the scene change obvious as the features and pedestrian change greatly. Secondly, retraining in new scene restrict its application because of large cost.

At any rate, the classifier will get good performance only in a particular scene. It will usually be less accurate in other new scenarios and meanwhile restrict its application. In addition, a problem in training such a classifier is that an extremely large number of training examples are required to ensure good performance in the test phase and the training time is very long. Once the classifier is trained, the classifier parameters are fixed in the test stage and they are not suitable for the new scene. Therefore, the performance of classifier is related to whether classifier parameters can adapt to the scenario.

This paper presents a pedestrian detection in changing scenes, which absorb new scene features to update the classifier based on the old classifier so that the classifier can receive satisfactory results. To design such a classifier, frrst, we adopt cascaded classification structure. In the new scene, (for speed up the detection, a general ternary detection framework was proposed.) the detection speed requires fast, to this end we propose a general ternary detection framework. It is based on cascade classification framework, and at each stage, we can differentiate exclude pedestrians and non-pedestrians tlrrough comparing the stage tlrreshold, and also, indeterminate objects will move to next stage. Comparing with the traditional cascade classifier, the ternary detection framework can detect) pedestrians and non-pedestrians at the same time, and the detection speed is faster. Then taking into account the performance of classifier is related to classifier parameters, we convert it to an optimization problem in order to adapt to detection in new scene. We present a fast optimization strategy based on cross entropy method and use small number of new scene features to adjust the classifier parameters, making the trained classifier can adapt to the new scenario. The final classifier obtained in new scene can receive good performance with fast detection speed and high detection rate and low false positive rate.

We demonstrate the performance of our method in AHHF [1] pedestrians dataset and NICTA [16] pedestrians dataset. Results show that the proposed method can receive high performance in changing scene.

1591

The rest of paper is organized as follows. In Section 2, we describe the pedestrian detection method in changing scenes. In Section 3, we present the results, and give the concluding remarks in Section 4.

II. PEDESTRIAN DETECTION IN CHANGING SCENE

This section proposed a pedestrian detection method in changing scene and consists four parts. The fIrst part gives the general idea and main framework. The second part discusses the ternary detection framework. Next part explains the classifIer parameters optimization in changing scene detailed. Part IV gives pseudo-code algorithm.

A. General idea This paper proposed rapid classifIcation based on

pedestrian detection in changing scenes. It can divide into two steps, the fIrst step is to train a cascade classifIer in old scene and the cascade classifIer is a ternary detection framework. The ternary detection can exclude non-pedestrians but also exclude pedestrians at each stage. Consider the classifIer performance is related to classifIer parameters, the second step is using small numbers of new scene data to transfer new scene features to the old classifIer, and then adopt cross entropy method to optimize classifIer parameters. The fmal classifIer which optimized over some times can adapt to the new scenario.

Figure I gives the main modules.

Figure 1: main modules

B. Ternary detection framework Voila presented a simple and effective cascade classifIer,

we can consider it as binary detection. Ternary detection framework is also a simple cascade classifIer and it is a chain structure which consists of many single classifIers. Each node is also a ternary framework. The common of the two detection framework is they can exclude non-pedestrians at each stage and indeterminate objects will move to next level. The difference is ternary detection can exclude pedestrians at each stage, but binary detection classify an object as a pedestrian only all the stage think it as pedestrian. The problem with binary detection is that the algorithm is not able to classify an object as pedestrian before the whole stage, not allowing the process to quilt earlier.

Consider a cascade classifIer contains 1 layers, and each layer is ternary detection. At each layer, we introduce two

parameters ai' � ( i = 1, ... ,1 ) as stage threshold to

determine the current objects that is pedestrians or non­pedestrians. We compare the stage parameters with the output of each strong classifIer to classify the current object. The single classifIer of the cascade classifIer which we proposed in the paper is based on AdaBoost algorithm. For each AdaBoost

algorithm, let � denote the output of stage i . The ternary

detection framework is given in Algorithm 1.

Algorithm 1 Ternary detection algorithm

Input a test dataset, a trained cascade classifIer

for i=I, ... ,1 Step 1: Compute the output of AdaBoost at ith layer � ;.

Step 2: if � < � , Output non-pedestrian, break;

Step 3: if � > ai , Output pedestrian, break;

Step 4: if � < � < ai ,continue;

Output pedestrian or non-pedestrian The detection framework is very simple, while it only

expands binary detection to ternary detection. This will be faster than binary detection and can exclude more objects at each stage. In binary detection, the cascade classifIer determines an object as pedestrian only all layers is positive of the object, while ternary detection algorithm can classify the object at any stage.

We give the theoretical interpretation to the algorithm. If the current object is a pedestrian, we use binary detection and ternary detection separately. In binary detection, the object

needs compare 1 times to consider the object as a pedestrian. In the ternary detection, suppose the current object be classifIed

as a pedestrian at stage i (i = 1, 2, ... ,1) and the probability

of the object is a pedestrian denotes as Pi. It needs to compare

i times to determine the object as a pedestrian, so the current object classifIed as a pedestrian need to compare

I E = I Pi * i times in average. The probability of the object

i=1 1

is a pedestrian at each layer is Pi = - . So the total times 1

n il 1+1 E = I Pi * i = - I i = -- , that is, the current object

i=1 1 i=1 2 1+1

determined as a pedestrian need to compare -- times, while 2

the object in binary detection need to compare 1 times. If we

2xl only consider pedestrian detection, ternary detection is --

1+1 times to binary detection.

However, ternary detection has a precondition which we need to obtain each stage parameters quickly and accurately.

Ternary detection depends on parameters ai' � and the value

depends on the training set. Consider a cascade classifIer which

contains 1 layers, there are two parameters at each layer, so the

1592

total number of parameters are 2 * I . In other word, if we can achieve these parameters quickly and accurately, the classifier will perform well.

The (above mentioned) trained classifier gives the good result in a particular scene. Obviously, the specific classifier would not perform well in other scenarios and thus it will not have widespread application. Next section we will adjust the stage parameters to adapt new scenarios.

C. Parameters optimization a/pedestrian detection in changing scenes In this section, we propose the pedestrian detection

algorithm in changing scenes. Firstly, we establish a classification model in the old scene and calculate its namely

threshold namely parameters ai' /3; at each stage, and then

with aid of a small number of new scene data to assist the classifier, using global optimization method to search the parameters combination of the classifier, establish a new classification model to adapt the new scenario.

Consider a cascade classifier which contains I layers, each layer has two parameters and the total amount of parameters

are 2 * I . The combination of the parameters is a

vector Jio = (� , PI ' a2 ' P2 , ... , a[ , /3; ) . In order to adapt the

new scenario, we need to adjust the parameter thresholds, so the problem can be converted to an optimization problem that is about searching a set of parameters combination making the classifier adapt the new scenario.

There are many existing optimization algorithm, such as Genetic Algorithm [16], Ant Colony Optimization [17], Neural Networks [18], Simulated Annealing [19] and Cross Entropy (CE) method [20] etc. In this paper, we use CE method to carry out global optimization. The reason is CE method is simper than other optimization algorithms and it is more suitable for parameters optimization of pedestrian in changing scenes.

The CE method has been successfully applied to a number of difficult combinatorial optimization problems. It has good performance, fast converge and high search rate. The CE method is an iterative method, which involves the following two phases: generation of a sample of random data (trajectories, vectors, etc.) according to a specified random mechanism and updating the parameters of the random mechanism, on the basis of the data, in order to produce a better sample in the next iteration.

We designed a parameter optimization method for pedestrian detection in changing scenes. We can see from above, the CE method for optimization mainly contains two questions, how to generate random sample by means of some random mechanism and how to update parameters and generate better sample in next iteration. This paper specially designs the optimization algorithm, first is calculate the mean and variance of samples to form initially vector, generate some random vectors by means of normal distribution, and then select several better vectors using auxiliary data from new scene, update parameter vector through iteration formal.

The main object of the proposed algorithm is to optimize this set of parameter vector and make it adapted to a new scene.

When we calculate the parameters air /3; (the mean of

positive samples and negative samples), we compute the variance of the samples at the same time. The initial parameter

vector is Jio, (J 0 .

Figure 2 gives the flowchart of parameter optimization.

Initial parameter

vector

Generate random

vectors using normal

distribution

meet condition

these mean

Update parameter

vector using

iteration formal

Using auxiliary

data

Figure 2: Flowchart of parameter optimization algorithm

Parameter optimization algorithm based of pedestrian diction in changing scenes is given in Algorithm 2.

Algorithm2 Parameters Optimization Algorithm

Input a trained cascade classifier, initial parameters vector

Jio = (� , PI' a2 , P2 , ... , a[, /3; ) ,iteration times n ,

auxiliary dataset

Step 1: i = O. generate M random parameter vectors

using normal distribution N (Jii , (J i) , detect these

parameter vectors in auxiliary dataset and select m better vectors.

Step 2: calculate the mean vector of m better vectors Jii

and obtain its variance vector (Ji . Step 3: update the next mean and variance vector ,the

formal is as follows:

Jii+1 = AXJii +(1-A)XJii

(J i+l= AX(Ji +(1-A) X (Ji Step 4: if (i < n) i + +, goto Step 1;

else output the vector un' exit.

Output the detect vector un The result of parameter optimization is the fmal vector

Jin = (ai' PI , a2 , P2 , ... , a[ , /3; ) that iterated n times.

1593

Pedestrian detection method in changing scenes adopts ternary detection framework and detect the new scene using a parameter vector.

The characteristics of the pedestrian detection are only using small numbers of new scene data to assist the old classification model and adapt the new scene by adjusting strategy. This method overcomes the shortcomings of retaining in new scene cost large and detect new scene using old trained classifier cannot obtain good performance. It use old scenario model to adjust parameters to adapt new scene continuously.

D. Pedestrian detection method in changing scenes Pedestrian detection algorithm in changing scenes is given

in Algorithm 3.

Algorithm 3 Pedestrian detection algorithm in changing scenes

Input old scene dataset, new scene dataset Step I: build a cascade classifier in old scene dataset and

compute initial parameter vector Po' (j 0 at same time.

Step 2: generate the final detect vector using Algorithm 2.

Step 3: detect the final vector un using Algorithm 1.

Output detect result

III. EXPERIMENTS

A. Dataset We take AHHF [I] pedestrian dataset as the old scene. This

dataset was collected from urban traffic videos captured in city Hefei, P.R.C. which contains 1600 pedestrian samples and many high-quality negative samples that are manually selected from many similarities to pedestrians. There are 800 pedestrian samples for training and the other 800 pedestrian samples of testing. The pedestrian samples are 16x32 pixel pictures and the negative samples are 320x240 pixel pictures. We select 828685 negative samples for testing and the ratio of positive samples and negative samples is about 1:100. We trained an old cascade classifier in the AHHF dataset.

We take NICTA [15] pedestrian dataset as the new scenario. The dataset contains a large number of different sizes and dimensions pedestrian images. We select 16x40 pixel pictures as pedestrian samples and this specification of samples contain 37347 pedestrian images. The negative samples are also 320x240 high quality pictures. Because of the complex and diverse pedestrian gesture in NICTA dataset, we first cluster the dataset into five parts and each part contains various forms of pedestrian images. We take the first part which includes 7249 pedestrian samples for training and the other parts which contain 29918 samples for testing.

Figure 3 gives the pedestrian images in two datasets. The pedestrian images in two datasets are quite different. So the two datasets are suitable as changing scenes.

(a) Pedestrian samples in AHHF dataset (l6x32) •

(b) Pedestrian samples in NICTA dataset (16x40) Figure 3: Example of pedestrian samples

B. Performance comparison In order to verify the validity of the proposed algorithm, we

trained a cascade classifier named Cascade I under AHHF dataset. Cascade I contained 6 layers and we computed the initial parameter vector at the training time. Pedestrian detection algorithm in changing scenes contained four steps. First of all, we selected 100 positive samples and 100 negative samples randomly from the first part of NICTA dataset as an auxiliary dataset and then generated 100 parameters vectors randomly using normal distribution and calculate these F _measure in auxiliary dataset. Then, we selected 10 maximal F _measure with its corresponding vector and computed the mean vector and variance vector of these 10 vectors. Finally, we obtained the fmal detect vector after 25 times iteration. The fmal parameter vector was suitable for new scene. We denoted the optimization classifier as NewCascade.

F measure is denotes as:

2xTP F measure = -------

2xTP+FP+FN Where TP corresponds to the positive samples which

classified correct, FN corresponds to the positive samples which classified false, FP corresponds to the negative samples which classified false.

In the iteration optimization process, we set the parameter

A as 0.9. We also trained a cascade classifier named Cascade2 in NICTA dataset in order to compare the algorithm performance.

We denote Cascadel-AHHF, Cascadel-NICTA ,

Cascade2-NICTA, NewCascade-NITCA as the detect results of different classifiers in different scenes.

Table I shows the detect rate and false positive rate of classifier Cascade I and NewCascade in NICTA dataset and gives the detect results of classifier Cascade I in AHHF dataset and Cascade2 in NICT A dataset at the same time. As we can see from Table I, Cascadel-AHHF and Cascade2-NICTA can receive good performance when the training and test data under the same distribution. If we use Cascade I detect NICTA dataset directly, the detect rate is very low and the false positive rate is also high. However, the improved algorithm

1594

makes the classifier which trained in old scene adapt to the new scenario, and the fmal detect rate improves obviously and the false positive rate is also better.

TABLE I COMPARISON OF ALGORITHM PERFORMANCE Result TPR FPR

Cascadel-AHHF 87.875% 0.0078 Cascade2-NICT A 89.59% 0.0077

Cascadel-NICT A 68.83% 0.0778

NewCascade-NICT A 81.23% 0.0657

FIgure 4 gIves the ROC curve of Cascade 1 under AHHF dataset and Figure 5 gives the ROC curve of Cascade2 under NICT A dataset.

ROC curve

0 .9 r-�-==:::=======::;;-� O.B

0.7 � 0.6

� 0.5 ;; � 0.4 � B 0.3

0.2

0.1

Cascade1-AHHF I

false posilive rate

Figure 4: ROC curve of Cascade 1 under AHFF dataset

ROC curve

0.9 r�-�-�----::::::::::====� O.B

0.7

� 0 6

� 0.5 :ij '00.4

� 0.3

0.2 0.1

- Cascade2-NICTA

0.002 0.004 0.006 0.008 0.01 0.012 0.014 false pos�ive rate

Figure 5: ROC curve of Cascade2 under NICT A dataset

The detect algorithm uses ternary detection which can exclude pedestrians or non-pedestrians at each layer. Only considering pedestrian detection, we can see from Section 2.1,

ternary detection is faster 2 * I / (l + 1) = 1.7 times than binary

diction in theory. Test data contains 29918 pedestrian samples and 828685 negative samples. Table 2 gives the detect time of ternary detection and binary detection in the test data. Ternary diction detects the vector which has optimized after 25 times. We can see from Table 2, the ratio of the time of two detection framework is close to theoretical value.

TABLE 2 TIME COMPARISON Detection Time (ms)

framework Pedestrian Negative Samples

Binary detection 1981 19172

Ternary detection 1232 14503

Time Ratio 1.6 1.32

Pedestrian detection method in changing scenes optimizes the initial vector by Cross Entropy method, which making the vector adapt to new scene continuously. In this paper we set the iteration times as 25. Figure 6 gives the detect rate under different iteration times. As we see from the figure, detect rate of the improved algorithm reaches 81 % after 25 iteration times and tends to steady if go on iteration. Therefore, the improved algorithm can effectively adjust the performance of old classifier under new scenario.

Detect rate under different iteration times

0.85 O.B

0.75 0.7 NewCascade I

0.40'----�-�,0----,,�5 -20� ----,L25,------.,:Il�- 35�----'40 ileration times

Figure 6 Detect rate under different iteration times

IV. CONCLUSIONS

In this paper, we proposed a rapid classification method based on pedestrian detection in changing scenes. It uses small numbers of auxiliary data from new scene to assisting the old classification model and generates the fmal detect vector by Cross Entropy method. The experimental results show that the proposed is effective with high detect rate, high detect speed and low false positive rate.

The main contribution of the proposed algorithm is adjusting the classification model by means of small numbers of scene data. In the future, we will try to fmd the common feature of the different scenes.

ACKNOWLEDGMENT

This work was supported by National High Technology Research and Development Program of China (2007AAllZ240), by the Major Program of National Natural Science Foundation of China (9820007), and by the Program for New Century Excellent Talents in University (NCET-07-0787).

REFERENCES

[I] X.B. Cao, H. Qiao, and 1. Keane, A low-cost pedestrian detection system with a single optical camera, IEEE Transactions on Intelligent Transportation System, 9 (I) :58-67,2008.

[2] YP. Huang, S. Fu, and C. Thompson, "Stereovision-based object segmentation for automotive applications," Eurasip Journal on Applied Signal Processing, Vol. 2005, no. 14, pp.2322-2329, 2005.

[3] M. Bertozzi, A. Broggi, A. Lasagni, and M. Del Rose, "Infrared stereo vision-based pedestrian detection," In Proc. IEEE International Conference on Intelligent Vehicles Symposium, pp. 24-29, Jun. 2005.

[4] D.M. Gavrila, and 1. Geibel, "Shape-based pedestrian detection and tracking," In Proc. IEEE Intelligent Vehicles Symposium, pp. 8-14, 2003.

1595

[5] G. M. A. Sessler, T. Martoyo, and F. K. Jondral, "RBF based multiuser detectors for D1RA-TDD," In Proc. IEEE Vehicular Technology Conference, pp. 484-486, 2001.

[6] F.L. Xu, X. Liu, and K. Fujimura, "Pedestrian Detection and Tracking with Night Vision," IEEE Transactions on Intelligent Transportation System, vol. 6, pp. 63-71,2005.

[7] S. Maji, A. Berg, and J. Malik. Classification using intersection kenels SYM is efficient. In CVPR, 2008.

[8] Paul A. Viola, Michael J. Jones: Rapid Object Detection using a Boosted Cascade of Simple Features. CVPR, 2001: 511-518

[9] P. Viola, M. Jones, and D. Snow. Detecting pedestrians using patterns of motion and appearance. In CVPR, 2003.

[10] P.Viola and M.Jones. Robust real-time object detection. UCV, 57(2):137-154,2004.

[11] G. Grubb, A. Zelinsky, L. Nilsson, and M. Rilbe, "3D Vision Sensing for Improved Pedestrian Safety," In Proc. IEEE Intelligent Vehicle Symposium, pp.19-24, 2004.

[12] A. Shashua, Y. Gdalyabu, and G. Rayun, "Pedestrian Detection for Driving Assistance Systems - Single-frame Classification and System Level Performance," In Proc. IEEE International Conference on Intelligent Vehicles Symposium, pp.I-6, 2004.

[13] P. Sabzmeydani and G. Mori, "Detecting Pedestrians by Learning Shapelet Features," In Proc. IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2007.

[14] O.Tuzel, F.M. Porikli, and P. Meer. Pedestrians det. Via classification on Riemannian manifolds. P AMI, 30(10): 1713-1727, 2008.

[15] G. Overett, L. Petersson, N. Brewer, L. Andersson and N. Pettersson. A New Pedestrian Dataset for Supervised Learning, In IEEE Intelligent Vehicles Symposium, 2008.

[16] J.A. Walker and J.F. Miller. The Automatic Acquisition, Evolution and Re-use of Modules in Cartesian Genetic Programming. IEEE Transactions on Evolutionary Computation, 12(4):397-417,2008.

[17] M. Reimann. Guiding ACO by Problem Relaxation: A Case Study on the Symmetric TSP. In: Proceedings of HM 2007, volume 4771: 45-56, 2007.

[18] Zhu and G. Yang. New delay-dependent stability results for neural networks with time-varying delays. IEEE Trans. Neural Networks, 19(10): 1783-1791,2008.

[19] Bandyopadhyay S, Saba S, Maulik U, Deb K. A Simulated Annealing Based Multi-objective Optimization Algorithm: AMOSA. IEEE Transactions on Evolutionary Computation, 12(3):269-283, 2008.

[20] Rubinstein, R.Y., &Kroese, D.P. The Cross-Entropy Method. Information Science and Statistics, Springer, 2004.

1596