Research ArticleDiscriminative Fusion Correlation Learning for Visible andInfrared Tracking
Xiao Yun Yanjing Sun Xuanxuan Yang and Nannan Lu
China University of Mining and Technology School of Information and Control Engineering 1 Daxue Road XuzhouJiangsu 221116 China
Correspondence should be addressed to Yanjing Sun yjsuncumteducn
Received 22 January 2019 Revised 2 April 2019 Accepted 18 April 2019 Published 15 May 2019
Academic Editor Volodymyr Ponomaryov
Copyright copy 2019 Xiao Yun et alThis is an open access article distributed under the Creative Commons Attribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited
Discriminative correlation filter- (DCF-) based trackers are computationally efficient and achieve excellent tracking in challengingapplications However most of them suffer low accuracy and robustness due to the lack of diversity information extracted froma single type of spectral image (visible spectrum) Fusion of visible and infrared imaging sensors one of the typical multisensorcooperation provides complementarily useful features and consistently helps recognize the target from the background efficientlyin visual trackingTherefore this paper proposes a discriminative fusion correlation learningmodel to improveDCF-based trackingperformance by efficiently combiningmultiple features from visible and infrared images Fusion learning filters are extracted via latefusion with early estimation in which the performances of the filters are weighted to improve the flexibility of fusion Moreoverthe proposed discriminative filter selection model considers the surrounding background information in order to increase thediscriminability of the template filters so as to improve model learning Extensive experiments showed that the proposed methodachieves superior performances in challenging visible and infrared tracking tasks
1 Introduction
Visual tracking has received widespread attention for itsextensive applications in video surveillance autonomousdriving and human-machine interaction military attackrobot vision etc [1 2] Depending on the appearance modelexisting tracking algorithms can be categorized into two cat-egories generative and discriminative tracking Generativetracking algorithms build a target model and search for thecandidate image patch with maximal similarity For exampleWang et al [3] proposed a novel regression-based objecttracking framework which successfully incorporates Lucasand Kanade algorithm into an end-to-end deep learningparadigm Chi et al [4] trained a dual network with randompatches measuring the similarities between the networkactivation and target appearance to leverage the robustnessof visual tracking On the contrary the goal of discriminativealgorithms is to learn a classifier to discriminate betweenits appearance and that of the environment given an initialimage patch containing the target Yang et al [5] proposed atemporal restricted reverse-low-rank learning algorithm for
visual tracking to jointly represent target and backgroundtemplates via candidates which exploits the low-rank struc-ture among consecutive target observations and enforces thetemporal consistency of target in a global level A new peakstrengthmetric [6] is proposed tomeasure the discriminativecapability of the learned correlation filter that can effectivelystrengthen the peak of the correlation response leading tomore discriminative performance than previous methods
Besides these efforts other researchers have worked ontrackingmethods that are both generative and discriminativeFor instance Zhang et al [7] obtained an object likelihoodmap to adaptively regularize the correlation filter learningby suppressing the clutter background noises while makingfull use of the long-term stable target appearance informa-tion Qi et al [8] proposed a structure-aware local sparsecoding algorithm which encodes a target candidate usingtemplates with both global and local sparsity constraintsand also obtains a more precise and discriminative sparserepresentation to account for appearance changes In [9]an adaptive set of filtering templates is learned to alleviatedrifting problem of tracking by carefully selecting object
HindawiMathematical Problems in EngineeringVolume 2019 Article ID 2437521 11 pageshttpsdoiorg10115520192437521
2 Mathematical Problems in Engineering
candidates in different situations to jointly capture the targetappearance variations Moreover a variety of simple yeteffective features are effectively integrated into the learningprocess of filters to further improve the discriminative powerof the filters In the salient-sparse-collaborative tracker [10]an object salient feature map is built to create a salient-sparsediscriminative model and a salient-sparse generative modelto both handle the appearance variation and reduce track-ing drifts effectively A multilayer convolutional network-based visual tracking algorithm based on important regionselection [11] is proposed to build high entropy selectionand background discrimination models and to obtain thefeature maps by weighting the template filters with clusterweights which enables the training samples to be informativein order to provide enough stable information and also bediscriminative so as to resist distractors Generally speakingdiscriminative and generative methods have complementaryadvantages in appearance modeling and the success of avisual tracking method depends not only on its represen-tation ability against appearance variations but also on thediscriminability between target and background thus leadingto the requirement of a more robust training model [12]
Recently discriminative correlation filter- (DCF-) basedvisual tracking methods [13ndash18] have shown excellent per-formances on real-time visual tracking for its advantageof robustness and computational efficiency The DCF-basedmethods work by learning an optimal correlation filter usedto locate the target in the next frame The significant gainin speed is obtained by exploiting the fast Fourier transform(FFT) at both learning and detection stages [14] Bolmeet al [13] presented an adaptive correlation filter namedMinimum Output Sum of Squared Error (MOSSE) filterwhich produces stable correlation filters by optimizing theoutput sum of squared error Based on MOSSE Danelljan etal [14 15] proposed a novel scale adaptive tracking approachby learning separate discriminative correlation filters fortranslation and scale estimation which achieves accurate androbust scale estimation in a tracking-by-detection frame-work Galoogahi et al [16] proposed a computationallyefficient Background-aware correlation filter-based on hand-crafted features that can efficiently model how both the fore-ground and background of the object varies over time Thework in [17] reformulates DCFs as a one-layer convolutionalneural network composed of integrates feature extractionresponse map generation and model update with residuallearning Johnander et al [18] proposed a unified formulationfor learning a deformable convolution filter in which thedeformable filter is represented as a linear combination ofsubfilters and both the subfilter coefficients and their relativelocations are inferred jointly in our formulation Howeverthe above trackers fail when the target undergoes severeappearance changes due to limited data supplied by singlefeatures
Multiple feature fusion contains more useful informationthan single feature thus providing higher precision certaintyand reliability for visual tracking Wu et al [19] proposed adata fusion approach via sparse representation with applica-tions to robust visual tracking Uzkent et al [20] proposedan adaptive fusion tracking method that combines likelihood
maps from multiple bands of hyperspectral imagery intoone single more distinctive representation which increasesthe margin between mean value of foreground and back-ground pixels in the fused map Chan et al [21] proposed arobust adaptive fusion tracking method which incorporatesa novel complex cell into the group of object representationto enhance the global distinctiveness Feature fusion alsoachieves superior performances on correlation filter-basedtracking For example Rapuru et al [22] proposed a robusttracking algorithm by efficiently fusing tracking learningand detection with the systematic model update strategy ofkernelized correlation filter tracker
Although much efforts have been made single-sensorfeature fusion-based tracking suffer low accuracy and robust-ness due to the lack of diversity information Fusion of visibleand infrared sensors one of the typical multisensor coopera-tions provides complementarily useful features which is ableto achieve a more robust and accurate tracking result [23] Liet al [24] designed a fusion scheme containing joint sparserepresentation and colearning update model to fuse colorvisual spectrum and thermal spectrum images for objecttracking Li et al [25] proposed an adaptive fusion schemebased on collaborative sparse representation in Bayesianfiltering framework for online tracking Mangale and Kham-bete [26] developed reliable camouflaged target detection andtracking system using fusion of visible and infrared imagingYun et al [23] proposed a compressive time-space Kalmanfusion tracking with time-space adaptability for visible andinfrared images and introduced extended Kalman filter toupdate fusion coefficients optimally A visible and infraredfusion tracking algorithm based on multiview multikernelfusionmodel is presented in [27] Zhang et al [28] transferredvisible tracking data to infrared data to obtain better trackingperformances Lan et al [29] proposed joint feature learningand discriminative classifier framework for multimodalitytracking which jointly eliminate outlier samples caused bylarge variations and learn discriminability-consistent featuresfrom heterogeneous modalities Li et al [30] proposed aconvolutional neural network architecture including a two-streamConvNet and a FusionNet which proves that trackingwith visible and infrared fusion outperforms that with singlesensor in terms of accuracy and robustness
DCF-based trackers have significant low computationalload and are especially suitable for a variety of real-timechallenging applications However most of the DCF-basedtrackers suffer low accuracy and robustness due to the lack ofdiversity information extracted from a single type of spectralimage (visible spectrum) Therefore this paper proposes adiscriminative fusion correlation learning model to improveDCF-based tracking performance by combining multiplefeatures from visible and infrared imaging sensors The maincontributions of our work are summarized as follows
(i) A discriminative fusion correlation learning modelis presented to fuse visible and infrared featuressuch that valuable information from all sensors ispreserved
(ii) The proposed fusion learning filters are obtainedvia late fusion with early estimation in which the
Mathematical Problems in Engineering 3
0
Fusionlearning filter
+
Visible image
Infrared image
Multi-channel featureDiscriminativefilter selection
Response mapTracking result
Update
Update
Figure 1 Discriminative fusion correlation learning for visible and infrared tracking Blue and green boxes denote the target and backgroundsamples extracted from the tracking result respectively
performances of the filters are weighted to improvethe flexibility of fusion
(iii) The proposed discriminative filter selection modelconsiders the surrounding background informationin order to increase the discriminability of the tem-plate filters so as to improve model learning
The remainder of this paper is organized as follows InSection 2 the multichannel discriminative correlation filteris introduced In Section 3 we describe our work in detailThe experimental results are presented in Section 4 Section 5concludes with a general discussion
2 Multichannel DiscriminativeCorrelation Filter
Multichannel DCF provides superior robustness and effi-ciency in dealing with challenging tracking tasks [14] Inthe multichannel DCF-based tracking algorithm 119889 channelHistogram of Oriented Gradient (HOG) features [14] fromthe target sample 119904 are extracted tomaintain diverse informa-tion During training process the goal is to learn correlationfilter ℎ which is achieved by minimizing the error of thecorrelation response compared to the desired correlationoutput 119892 as
120576 =1003817100381710038171003817100381710038171003817100381710038171003817119892 minus119889
sum119897=1
ℎ119897 lowast 1199041198971003817100381710038171003817100381710038171003817100381710038171003817
2
+ 120582119889
sum119897=1
10038171003817100381710038171003817ℎ119897100381710038171003817100381710038172 (1)
where lowast denotes circular correlation and 120582 is the weightparameter [14] 119904119897 and ℎ119897 (119897 = 1 sdot sdot sdot 119889) are the 119897-th channelfeature and the corresponding correlation filter respectivelyThe correlation output 119892 is supposed to be a Gaussianfunction with a parametrized standard deviation [14]
The minimization of (1) can be solved by minimizing (2)in the Fourier domain as
119867119897 = 119866119878119897
sum119889119896=1 119878119896119878119896 + 120582
119897 = 1 sdot sdot sdot 119889 (2)
where 119867 119866 and 119878 are the discrete Fourier transform (DFT)of ℎ 119892 and 119904 respectively The symbol bar sdot denotes complexconjugation The multiplications and divisions in (2) areperformed pointwiseThe numerator119860119897119905 and denominator 119861119905of the filter119867119897119905 in (2) are updated as
119860119897119905 = (1 minus 120578)119860119897119905minus1 + 120578119866119878119897119905 119897 = 1 sdot sdot sdot 119889
119861119905 = (1 minus 120578)119860 119905minus1 + 120578119889
sum119896=1
119878119896119878119896(3)
where 120578 is a learning rate parameterDuring tracking process the DFT of the correlation score
119910119905 of the test sample 119911119905 is computed in the Fourier domainas
119884119905 =sum119889119897=1 119860
119897
119905minus1119885119897119905119861119905minus1 + 120582 (4)
where119884119905 and119885119897119905 are theDFTs of119910119905 and 119911119897119905 respectively119860119897119905 and119861119905 are the numerator and denominator of the filter updated inthe previous frame respectively Then the correlation scores119910119905 is obtained by taking the inverse DFT 119910119905 = Fminus1119884119905 Theestimate of the current target state is obtained by finding themaximum correlation score among the test samples
3 Proposed Discriminative FusionCorrelation Learning
In this section we introduce the tracking framework of theproposed algorithm the general scheme ofwhich is describedin Figure 1 Firstly the multichannel features are extractedrespectively from the visible and infrared images accordingto [14] Secondly the proposed discriminative filter selectionand the fusion filter learning are applied to get the fusionresponse map Finally the discriminative filters and fusionfilters are updated via the tracking result obtained by theresponse map We will discuss them specifically below
4 Mathematical Problems in Engineering
31 Discriminative Filter Selection According to DCF-basedtrackers we obtain the correlation output 119892 by
119892 =119889
sum119897=1
ℎ119897119879 lowast 119904119897119879 119897 = 1 sdot sdot sdot 119889 (5)
where 119904119897119879 and ℎ119897119879 are the target sample and target correlationfilter corresponding to the 119897-th channel feature among 119889channels respectively In this paper 119892 is selected as a 2DGaussian function where 120590 = 20 [13]
Before tracking we need to choose the optimal targetcorrelation filters in the training step via minimizing (6) as
119867119897119879 =119866119878119897119879
sum119889119896=1 119878119896
119879119878119896119879 + 120582 119897 = 1 sdot sdot sdot 119889 (6)
where 119878119897119879 119867119897119879 and 119866 are the DFTs of 119904119897119879 ℎ119897119879 and 119892respectively
Different from a single training sample of the targetappearance multiple background samples at different loca-tions around target need to be considered to maintain a sta-ble model However extracting multichannel features fromeach background sample increase computational complexsignificantly Moreover in practice single channel featuresfrom multiple background samples are enough to presentsatisfied performances Therefore in this paper we extract119872 background samples randomly in the range of an annulusaround the target location [11] and obtain the correlationoutput 119892 as
119892 =119872
sum119898=1
ℎ119861 lowast 119904119898119861 119898 = 1 sdot sdot sdot 119872 (7)
where 119904119898119861 119898 = 1 sdot sdot sdot 119872 denotes the119898-th background sam-ple
Similarly the optimal background correlation filters inthe training step are selected via minimizing (8) as
119867119898119861 = 119866119878119898119861sum119872119895=1 119878
119895
119861119878119895119861 + 120582 119898 = 1 sdot sdot sdot 119872 (8)
where 119878119898119861 and119867119898119861 are the DFTs of 119904119898119861 and ℎ119898119861 respectivelyWhile tracking DFT 119884119905119890119904 of the estimated discriminative
correlation score 119910119905119890119904 of the test sample 119911119905 is defined as
119884119905119890119904 =119884119905119890119904119879119884119905119890119904119861
=(sum119889119897=1 119860
119897
119905minus1119879119885119897119905) (119861119905minus1119879 + 120582)(sum119872119898=1 119860
119898
119905minus1119861119885119905) (119861119905minus1119861 + 120582)
=(sum119889119897=1 119860
119897
119905minus1119879119885119897119905) 119861119905minus1119861(119885119905sum119872119898=1 119860
119898
119905minus1119861) (119861119905minus1119879 + 120582)
(9)
where 119884119905119890119904119879 and 119884119905119890119904119861 are the DFTs of the target andbackground correlation scores 119910119905119890119904119879 and 119910119905119890119904119861 respectivelyand 119885119897119905 is the DFTs of 119911119897119905 119860119897119905minus1119879 and 119861119905minus1119879 denote thenumerator and denominator of the filter 119867119897119879 in (6) 119860119898119905minus1119861
and119861119905minus1119861 denote the numerator and denominator of the filter119867119898119861 in (8)Then the discriminative correlation scores 119910119905119890119904 areobtained by taking the inverse DFT 119910119905119890119904 = Fminus1119884119905119890119904 Theestimate of the current target state 1199111199050 119890119904 is obtained by findingthe maximum correlation score among the test samples as1199111199050 119890119904 997888rarr 1199050 1199050 = arg max119905119910119905119890119904
32 Fusion Learning Filter As proved by Wagner et al [31]late fusion with early estimation provides better performancethan early fusion with late estimation Based on this conclu-sion we use the discriminative correlation filters to obtainthe estimate of target location in visible and infrared imagesrespectively and then do fusion with the fusion correlationfilters Let 119911119905119890119904119894 119894 isin V119894 119894119903 denote the estimate of targetlocation of visible V119894 or infrared 119894119903 images Then we definethe region119877∘(sdot ∙) denoting theminimumbounding rectanglethat contains the regions of samples sdot and ∙ in image ∘Thus we extract the fusion test samples through 119911119905119894 isin119877119894(119911119905119890119904V119894 119911119905119890119904119894119903) 119894 isin V119894 119894119903 and define the DFT 119884119905119891119906 ofthe fusion correlation score 119910119905119891119906 of fusion sample 119911119905119894 that isdefined as
119884119905119891119906 =119884119905119879119891119906119884119905119861119891119906
= sum119894isinV119894119894119903 120572119894119884119905119890119904119879119894sum119894isinV119894119894119903 120572119894119884119905119890119904119861119894
=sum119894isinV119894119894119903 120572119894 ((sum119889119897=1 119860
119897
119905minus1119879119894119885119897119905119894) (119861119905minus1119879119894 + 120582))sum119894isinV119894119894119903 120572119894 ((sum119872119898=1 119860
119898
119905minus1119861119894119885119905119894) (119861119905minus1119861119894 + 120582))
= sum119894isinV119894119894119903
120572119894 (sum119889119897=1 119860119897
119905minus1119879119894119885119897119905119894) 119861119905minus1119861119894(119885119905119894sum119872119898=1 119860
119898
119905minus1119861119894) (119861119905minus1119879119894 + 120582)
(10)
where 119884119905119891119906119879 and 119884119905119891119906119861 are the DFTs of the target and back-ground fusion correlation scores 119910119905119891119906119879 and 119910119905119891119906119861 respec-tively 119911119897119905119894 isin 119877119894119897(119911119905119890119904V119894 119911119905119890119904119894119903) 119894 isin V119894 119894119903 in which 119894119897 is the 119897-thchannel feature of image 119894 119885119897119905119894 and 119885119905119894 are the DFTs of sam-ples 119911119897119905119894 and 119911119905119894 respectively120572119894 denotes the imageweights thatare denoted as
120572i =max119894119910119905119890119904119894
sum119894isinV119894119894119903max119894119910119905119890119904119894 (11)
where 119910119905119890119904119894 is the correlation score of the 119894-th image com-puted by (9)
After obtaining 119884119905119891119906 the fusion correlation score 119910119905119891119906is obtained by taking the inverse DFT 119910119905119891119906 = Fminus1119884119905119891119906The fusion location of the current target state is obtainedby finding the maximum correlation score among the testsamples
The whole tracking process of DFCL is summarized inAlgorithm 1
4 Experiments
The proposed DFCL algorithm was tested on several chal-lenging real-world sequences and some qualitative andquantitative analyses were performed on the tracking resultsin this section
Mathematical Problems in Engineering 5
InputThe 119905-th visible and infrared imagesFor 119905 = 1 to number of frames do
1 Crop the samples and extract the 119897-th (119897 = 1 sdot sdot sdot 119889) channel features 119911119897119905 for visible andinfrared images respectively
2 Compute the discriminative correlation scores 119910119905119890119904 using Eq (9)3 Compute the fusion correlation scores 119910119905119891119906 using Eq (10)4 Obtain the tracking result by maximizing 1199101199051198911199065 Extract the 119897-th (119897 = 1 sdot sdot sdot 119889) channel feature 119904119897119879 of the target samples and the119898-th
(119898 = 1 sdot sdot sdot119872) sample 1199041198971198616 Update the discriminative correlation filters119867119897119879 and119867119898119861 using Eq (6) and Eq (8)
respectivelyend forOutput Target result and the discriminative correlation filters119867119897119879 and119867119898119861
Algorithm 1 Discriminative fusion correlation learning for visible and infrared tracking
41 Experimental Environment and Evaluation CriteriaDFCL was implemented with C++ programming languageandNet Framework 40 in Visual Studio 2010 on an IntelDual-Core 170GHz CPU with 4 GB RAM Two metricsie location error (pixel) and overlapping rate are usedto evaluate the tracking results quantitatively The locationerror is computed as 119890119903119903119900119903 = radic(119909119866 minus 119909119879)2 + (119910119866 minus 119910119879)2where (119909119866 119910119866) and (119909119879 119910119879) are the ground truth (eitherdownloaded from a standard database or located manually)and tracking bounding box centers respectivelyThe trackingoverlapping rate is defined as 119900V119890119903119897119886119901119901119894119899119892 = 119886119903119890119886(119877119874119868119866 cap119877119874119868119879)119886119903119890119886(119877119874119868119866cup119877119874119868119879) where119877119874119868119866 and119877119874119868119879 denote theground truth and tracking bounding box respectively and119886119903119890119886(sdot) is the rectangular area function A smaller locationerror and a larger overlapping rate indicate higher accuracyand robustness
42 Experimental Results The performance of DFCL wascompared with state-of-the-art trackers Struck [32] ODFS[33] STC [34] KCF [35] ROT [36] DCF-based trackersMOSSE [13] DSST [14] fDSST [15] and visible-infraredfusion trackers TSKF [23] MVMKF [27] L1-PF [19] JSR[24] and CSR [25] Figures 2ndash6 present the experimentalresults of the test trackers in challenging visible sequencesnamed Biker [37] Campus [37] Car [38] Crossroad [39]Hotkettle [39] Inglassandmobile [39] Labman [40] Pedes-trian [41] and Runner [38] as well as their correspondinginfrared sequences Biker-ir Campus-ir Car-ir Crossroad-ir Labman-ir Hotkettle-ir Inglassandmobile-ir Pedestrian-ir and Runner-ir Single-sensor trackers were separatelytested on visible and the corresponding infrared sequenceswhile visible-infrared fusion trackers obtain the results withinformation fromboth visible and infrared sequences For theconvenience of presentation some tracking curves are notshown entirely in the Figures Next the performance of thetrackers in each sequence is described in detail
(a) SequencesBiker andBiker-irBiker presents the exam-ple of complex background clutters The target human in thevisible sequence encounters similar background disturbance(ie bikes) which causes the ODFS MOSSE fDSST TSKF
and MVMKF trackers to drift away from the target Thecorresponding infrared sequence Biker-ir provides temper-ature information that eliminates the background clutter inBiker But when the target is approaching another person ataround Frame 20 Struck ODFS STC MOSSE TSKF andMVMKF do not perform well because they are not able todistinguish target from persons with similar temperature ininfrared sequences Only KCF ROT DSST and our DFCLhave achieved precise and robust performances in thesesequences
(b) Sequences Campus and Campus-ir the target inCampus and Campus-ir undergoes background cluttersocclusion and scale variation At the beginning of CampusODFS STC KCF and ROT lose the target due to backgrounddisturbance Only TSKF and DFCL perform well whileStruck fDSST and MVMKF do not achieve accurate resultsBecause of the infrared information provided by Campus-irfewer test trackers lose tracking when background cluttershappen as shown in Figure 2 But Struck KCF and ROTmistake another person for the target As shown in Figure 2most of the trackers result in tracking failures whereas DFCLoutperforms the others in most metrics (location accuracyand success rate)
(c) SequencesCar andCar-irCar andCar-ir demonstratethe efficiency of DFCL on coping with heavy occlusions Thetarget driving car is occluded by lampposts and trees manytimes which cause tracking failures of most trackers OnlyTSKF MVMKF and DFCL are able to handle the occlusionthroughout the tracking process in this sequence As shownin Figure 2most trackers performbetter inCar-ir than inCarbecause the infrared features can overcome the difficultiesof target detection among surrounding similar backgroundSTC TSKF MVMKF and DFCL are able to handle thisproblem whereas the result of DFCL is the most accurate asshown in Figure 2
(d) Sequences Crossroad and Crossroad-ir the target inCrossroad and Crossroad-ir undergoes heavy backgroundclutters when she crosses the roadWhile the target is passingby the road lamp both ODFS and JSR lose the target Thenwhen a car passes by the target Struck TSKF and MVMKFdrift away from the target When the target goes toward
6 Mathematical Problems in Engineering
20 53 147
Biker
20 53 147
Biker-ir
8 84 156
Campus
8 84 156
Campus-ir
14 23 27
Car
14 23 27
Car-ir
136 289 767
Crossroad
136 289 767
Crossroad-ir
133 451 1068
Hotkettle
133 451 1068
Hotkettle-ir
309 428 511
Inglassandmobile
309 428 511
Inglassandmobile-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF
302 320 446
Labman
302 320 446
Labman-ir
29 228 363
Pedestrian
29 228 363
Pedestrian-ir
14 94 138
Runner
14 94 138
Runner-ir
L1-PF JSR CSR
Figure 2 Tracking performances of the test sequences
Mathematical Problems in Engineering 7
Biker
0
40
80
1 41 81 121
Biker-ir
0
50
100
150
1 41 81 121
Campus
0
50
100
150
200
1 51 101 151 201
Campus-ir
0
100
200
300
1 51 101 151 201
Car
0
100
200
1 11 21 31
Car-ir
0
100
200
300
1 11 21 31
0
50
100
150
200
250
1 201 401 6010
100
200
1 201 401 601
Crossroad Crossroad-ir
0
50
100
150
200
250
300
1 201 401 601 801 1001
Hotkettle
0
50
100
150
200
250
1 201 401 601 801 1001
Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0
50
100
150
200
1 201 401 6010
50
100
150
1 201 401 601
Labman
0
100
200
300
1 101 201 301 401
Labman-ir
0
200
400
1 101 201 301 401
Pedestrian
0
50
100
150
1 101 201 301
Pedestrian-ir
0
50
100
150
1 101 201 301
Runner
0
100
200
1 61 121 181
Runner-ir
0
100
200
1 61 121 181
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
Figure 3 Location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
0
05
1
1 41 81 1210
05
1
1 41 81 1210
05
1
1 51 101 151 2010
05
1
1 51 101 151 2010
05
1
1 11 21 310
05
1
1 11 21 31
0
05
1
1 201 401 6010
05
1
1 201 401 6010
05
1
1 201 401 601 801 10010
05
1
1 201 401 601 801 10010
05
1
1 201 401 6010
05
1
1 201 401 601
0
05
1
1 101 201 301 4010
05
1
1 101 201 301 4010
05
1
1 101 201 3010
05
1
1 101 201 3010
05
1
1 61 121 1810
05
1
1 61 121 181
Figure 4 Overlapping rate of the test sequences
the sidewalk most of the trackers are not able to handlethe problem of heavy background clutters but our trackerperforms satisfying tracking results as shown in Figures 2ndash4
(e) Sequences Hotkettle and Hotkettle-ir in thesesequences tracking is hard because of the changes of thecomplex background clutters Most trackers perform betterin Hotkettle-ir than in Hotkettle for the reason that thetemperature diverge makes the hot target more distinct inthe cold background Struck KCF DSST fDSST and DFCL
can achieve robust and accurate tracking performances asshown in Figures 2ndash4
(f) Sequences Inglassandmobile and Inglassandmobile-irSequences Inglassandmobile and Inglassandmobile-ir demon-strate the performances of the 14 trackers under the cir-cumstances of background clutters illumination changesand occlusion As shown in Figure 2 when the illuminationchanges at around Frames 300 ODFS and fDSST lose thetarget and KCF TSKF and L1-PF drift a little away from the
8 Mathematical Problems in Engineering
10048
96602
171062
9467 27351 20776 17357
3462 21838 17229
2866
241288
9947 6518
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Biker Biker-ir Campus Campus-ir
4536
114901
6739 4323 3944
187420
3574 3805 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2758 9391
287670
2255 4284
216795
1778 2248 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
6301
165743 160172
122008 116442
151244
7767 7767 3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
59762
9866
138565
63421 63817
145486
11886 16255
3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Car Car-ir
122137
109606
122092 122500 122514
106350
121940 122072
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
25018 22604 24216
122500
55885
106350
121940 122530
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Labman Labman-ir
19128
263409
304244
34664 26765
390209
27024 18464 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
7160 16449
377858
7428 38194
497072
21473 12348 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Pedestrian Pedestrian-ir Runner Runner-ir
117051 116737 120324
142077 137828 144675
121509
144107
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
9457
55577
141371 142012
43182
435192
141348 141350
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
116065
92197
30636
129168
27984
100629
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2876
117247
2341
127301
3465
297251
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
9575
300997
60040
11480 15424
287956
19187 2840
21838 17229 2866
241288
9947 4246
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
3638
79438 54283
4987
55103
313867
3023 4995 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
5983 9604
28494
4086 5710
196150
2693 4255 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
6687
66141
48306
16131
63235
116459
14736
74415
18142 10318 10791 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
4160
16714 26378
16071
3345
130137
14534
60554
18142 10318 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
10791
Figure 5 Average location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0993
0311
0986 0993 0993
0108
0993 0993 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
1000 0973
0014
1000 1000
0108
1000 1000 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0907
0005 0000 0056 0070
0009
0986 0986 1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0437
1000
0023
0158 0186
0028
0935
0758
1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0059
0176
0088 0088 0088 0059
0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0824
0382
0941
0088
0471
0059 0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0847
0026 0026
0520
0714
0022
0677 0733
0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0991
0877
0004
0994
0011 0009
0708
0959 0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0068 0054 0078 0049 0054 0044
0083 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0605
0020 0054 0054
0502
0024 0054 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0101
0189
0481
0016
0598
0044
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0937
0005
0904
0019
0943
0033
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0823
0016
0186
0808
0585 0585 0528
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0823
0016
0302
0808 0819
0016
0472
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0585
0105
0331
0561
0318
0000
0585 0558 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0571
0222
0454
0585 0578
0006
0585 0585 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976
0035
0749
0884
0587
0118
0917
0245
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976 0948
0604
0887
1000
0089
0925
0580
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Figure 6 Success rate of the test sequences
targetWhen the target is approaching a tree the backgroundclutters makes most of trackers cause tracking failures thatcan be seen from Figure 2 Our DFCL can overcome thesechallenges and perform well in these sequences
(g) Sequences Labman and Labman-ir the experimentsin Sequences Labman and Labman-ir aim to evaluate the per-formances on tracking under appearance variation rotationscale variation and background clutter In Labman whenthe target man is walking into the laboratory ODFS STCand MOSSE lose the target When the man keeps shaking
and turning around his head at around Frame 400 KCFROT and DSST cause tracking failures Also most trackersachieve better tracking performances in Labman-ir as shownin Figure 2
(h) Sequences Pedestrian and Pedestrian-ir the target inPedestrian and Pedestrian-ir undergoes heavy backgroundclutters and occlusion As shown in Figure 2 other trackersresult in tracking failures in Pedestrian whereas our trackershows satisfying performances in terms of both accuracyand robustnessThe efficient infrared features extracted from
Mathematical Problems in Engineering 9
DFCLDCL MOSSE
Average location error (pixel) Success rate
011 001 006
059
000 012
002 004 004
052
019 017
063
021
073
039 053
037
100 100
082 082
057
099 100 099 099
18742 15124
10635
2078
31387
11646
39021
10063 14467
1853 6166 8940
1786
9545 4990 5658
2692 9228
340 315 1899 652 495 723 1013 138 284
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Figure 7 Average location error (pixel) and success rate of DCL DFCL and MOSSE on the test sequences
Pedestrian-ir ensure the tracking successes of Struck STCand DFCL as can be seen from Figures 2ndash4
(i) Sequences Runner and Runner-ir Runner and Runner-ir contain examples of heavy occlusion abrupt movementand scale variation The target running man is occludedby lampposts trees stone tablet and bushes many timesresulting in tracking failures ofmost trackers Also the abruptmovement and scale variation cause many trackers to driftaway the target in both Runner and Runner-ir as shown inFigure 2 Once again ourDFCL is able to overcome the aboveproblems and achieve good performances
Figures 5 and 6 are included here to demonstrate quan-titatively the performances on average location error (pixel)and success rate The success rate is defined as the numberof times success is achieved in the whole tracking processby considering one frame as a success if the overlapping rateexceeds 05 [33] A smaller average location error and a largersuccess rate indicate increased accuracy and robustnessFigures 5 and 6 show that DFCL performs satisfying most ofthe tracking sequences
To validate the effectiveness of the discriminative filterselection model of DFCL we compare the tracker DCL(the proposed DFCL without the fusion learning model)with DFCL and the original DCF tracker MOSSE on visiblesequencesThe performances shown in Figure 7 demonstratethe efficiency of the discriminative filter selection modelespecially in the sequences with background clutters ieSequences BikerHotkettle Inglassandmobile and Pedestrian
5 Conclusion
Discriminative correlation filter- (DCF-) based trackers havethe advantage of being computationally efficient and morerobust than most of the other state-of-the-art trackers inchallenging tracking tasks thereby making them especiallysuitable for a variety of real-time challenging applications
Howevermost of theDCF-based trackers suffer low accuracydue to the lack of diversity information extracted from asingle type of spectral image (visible spectrum) Fusion ofvisible and infrared sensors one of the typical multisensorcooperation provides complementarily useful features andconsistently helps recognize the target from the backgroundefficiently in visual tracking For the above reasons this paperproposes a discriminative fusion correlation learning modelto improve DCF-based tracking performance by combiningmultiple features from visible and infrared imaging sensorsThe proposed fusion learning filters are obtained via latefusion with early estimation in which the performances ofthe filters are weighted to improve the flexibility of fusionMoreover the proposed discriminative filter selection modelconsiders the surrounding background information in orderto increase the discriminability of the template filters soas to improve model learning Numerous real-world videosequences were used to test DFCL and other state-of-the-art algorithms and here we only selected representativevideos for presentation Experimental results demonstratedthat DFCL is highly accurate and robust
Data Availability
The data used to support the findings of this study weresupplied by China University of Mining and Technologyunder license and so cannot be made freely available
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
This work is supported by the Natural Science Foun-dation of Jiangsu Province (BK20180640 BK20150204)
10 Mathematical Problems in Engineering
Research Development Programme of Jiangsu Province(BE2015040) the State Key Research Development Program(2016YFC0801403) and the National Natural Science Foun-dation of China (51504214 51504255 51734009 61771417 and61873246)
References
[1] S A Wibowo H Lee E K Kim and S Kim ldquoVisual trackingbased on complementary learners with distractor handlingrdquoMathematical Problems in Engineering vol 2017 Article ID5295601 13 pages 2017
[2] T Zhou M Zhu D Zeng and H Yang ldquoScale adaptive kernel-ized correlation filter tracker with feature fusionrdquoMathematicalProblems in Engineering vol 2017 Article ID 1605959 8 pages2017
[3] C Wang H K Galoogahi C Lin and S Lucey ldquoDeep-LK forefficient adaptive object trackingrdquo Computer Vision and PatternRecognition 2017
[4] Z Chi H Li H Lu and M-H Yang ldquoDual deep network forvisual trackingrdquo IEEE Transactions on Image Processing vol 26no 4 pp 2005ndash2015 2017
[5] Y Yang W Hu Y Xie W Zhang and T Zhang ldquoTemporalrestricted visual tracking via reverse-low-rank sparse learningrdquoIEEE Transactions on Cybernetics vol 47 no 2 pp 485ndash4982017
[6] Y Sui G Wang and L Zhang ldquoCorrelation filter learningtoward peak strength for visual trackingrdquo IEEE Transactions onCybernetics vol 48 no 4 pp 1290ndash1303 2018
[7] K Zhang X Li H Song Q Liu and W Lian ldquoVisual track-ing using spatio-temporally nonlocally regularized correlationfilterrdquo Pattern Recognition vol 83 pp 185ndash195 2018
[8] Y Qi L Qin J Zhang S Zhang Q Huang and M-H YangldquoStructure-aware local sparse coding for visual trackingrdquo IEEETransactions on Image Processing vol 27 no 8 pp 3857ndash38692018
[9] B Bai B Zhong G Ouyang et al ldquoKernel correlation filtersfor visual tracking with adaptive fusion of heterogeneous cuesrdquoNeurocomputing vol 286 pp 109ndash120 2018
[10] Y Liu F Yang C Zhong Y Tao B Dai and M Yin ldquoVisualtracking via salient feature extraction and sparse collaborativemodelrdquo AEU - International Journal of Electronics and Commu-nications vol 87 pp 134ndash143 2018
[11] X Yun Y Sun S Wang Y Shi and N Lu ldquoMulti-layerconvolutional network-based visual tracking via importantregion selectionrdquo Neurocomputing vol 315 pp 145ndash156 2018
[12] X Lan S Zhang and P C Yuen ldquoRobust joint discriminativefeature learning for visual trackingrdquo in Proceedings of the 25thInternational Joint Conference on Artificial Intelligence IJCAI2016 pp 3403ndash3410 AAAI Press New York NY USA July2016
[13] D S Bolme J R Beveridge B A Draper and Y M LuildquoVisual object tracking using adaptive correlation filtersrdquo inProceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo10) pp 2544ndash2550 IEEE SanFrancisco Calif USA June 2010
[14] M Danelljan G Hager F Shahbaz Khan and M FelsbergldquoAccurate scale estimation for robust visual trackingrdquo in Pro-ceedings of the British Machine Vision Conference pp 651-6511BMVA Press Nottingham UK 2014
[15] M Danelljan G Hager F S Khan and M Felsberg ldquoDis-criminative scale space trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 39 no 8 pp 1561ndash15752017
[16] H K Galoogahi A Fagg and S Lucey ldquoLearning background-aware correlation filters for visual trackingrdquo inProceedings of the16th IEEE International Conference on Computer Vision ICCV2017 pp 1144ndash1152 IEEE Istanbul Turkey 2018
[17] Y SongCMa LGong J Zhang RWH Lau andM-HYangldquoCrest convolutional residual learning for visual trackingrdquoin Proceedings of the 16th IEEE International Conference onComputer Vision ICCV 2017 pp 2574ndash2583 IEEEVenice ItalyOctober 2017
[18] J Johnander M Danelljan F Khan andM Felsberg ldquoDCCOtowards deformable continuous convolution operators forvisual trackingrdquo in Proceedings of the International Conferenceon Computer Analysis of Images and Patterns pp 55ndash67Springer Ystad Sweden 2017
[19] Y Wu E Blasch G Chen L Bai and H Ling ldquoMultiplesource data fusion via sparse representation for robust visualtrackingrdquo in Proceedings of the 14th International Conference onInformation Fusion Fusion 2011 IEEE Chicago IL USA July2011
[20] B Uzkent A Rangnekar and M J Hoffman ldquoAerial vehicletracking by adaptive fusion of hyperspectral likelihood mapsrdquoin Proceedings of the 30th IEEE Conference on Computer Visionand Pattern RecognitionWorkshops CVPRW 2017 pp 233ndash242IEEE Honolulu HI USA July 2017
[21] S Chan X Zhou and S Chen ldquoRobust adaptive fusion trackingbased on complex cells and keypointsrdquo IEEE Access vol 5 pp20985ndash21001 2017
[22] M K Rapuru S Kakanuru P M Venugopal D Mishra and GR Subrahmanyam ldquoCorrelation-based tracker-level fusion forrobust visual trackingrdquo IEEE Transactions on Image Processingvol 26 no 10 pp 4832ndash4842 2017
[23] X Yun Z Jing G Xiao B Jin and C Zhang ldquoA compressivetracking based on time-space Kalman fusion modelrdquo ScienceChina Information Sciences vol 59 no 1 pp 1ndash15 2016
[24] H P Liu and F C Sun ldquoFusion tracking in color and infraredimages using joint sparse representationrdquo Science China Infor-mation Sciences vol 55 no 3 pp 590ndash599 2012
[25] C Li H Cheng S Hu X Liu J Tang and L Lin ldquoLearningcollaborative sparse representation for grayscale-thermal track-ingrdquo IEEE Transactions on Image Processing vol 25 no 12 pp5743ndash5756 2016
[26] S Mangale and M Khambete ldquoCamouflaged target detectionand tracking using thermal infrared and visible spectrum imag-ingrdquo in Automatic Diagnosis of Breast Cancer using Thermo-graphic Color Analysis and SVM Classifier vol 530 of Advancesin Intelligent Systems and Computing pp 193ndash207 SpringerInternational Publishing 2016
[27] X Yun Z Jing and B Jin ldquoVisible and infrared tracking basedon multi-view multi-kernel fusion modelrdquo Optical Review vol23 no 2 pp 244ndash253 2016
[28] L Zhang A Gonzalez-Garcia J van de Weijer M Danelljanand F S Khan ldquoSynthetic data generation for end-to-end ther-mal infrared trackingrdquo IEEE Transactions on Image Processingvol 28 no 4 pp 1837ndash1850 2019
[29] X Lan M Ye S Zhang and P C Yuen ldquoRobust collaborativediscriminative learning for rgb-infrared trackingrdquo in Proceed-ings of the 32th AAAI Conference on Artificial Intelligence pp7008ndash7015 AAAI New Orleans LA USA 2018
Mathematical Problems in Engineering 11
[30] C Li X Wu N Zhao X Cao and J Tang ldquoFusing two-streamconvolutional neural networks for RGB-T object trackingrdquoNeurocomputing vol 281 pp 78ndash85 2018
[31] JWagner V FischerMHerman and S Behnke ldquoMultispectralpedestrian detection using deep fusion convolutional neuralnetworksrdquo in Proceedings of the 24th European Symposium onArtificial Neural Networks pp 1ndash6 Bruges Belgium 2016
[32] S Hare A Saffari and P H S Torr ldquoStruck structured outputtracking with kernelsrdquo in Proceedings of the IEEE InternationalConference on Computer Vision pp 263ndash270 IEEE ProvidenceRI USA 2012
[33] K Zhang L Zhang andM-H Yang ldquoReal-time object trackingvia online discriminative feature selectionrdquo IEEE Transactionson Image Processing vol 22 no 12 pp 4664ndash4677 2013
[34] K Zhang L Zhang M Yang and D Zhang ldquoFast trackingvia spatio- temporal context learningrdquo Computer Vision andPattern Recognition 2013
[35] J F Henriques R Caseiro P Martins and J Batista ldquoHigh-speed tracking with kernelized correlation filtersrdquo IEEE Trans-actions on Pattern Analysis and Machine Intelligence vol 37 no3 pp 583ndash596 2015
[36] X Dong J Shen D Yu W Wang J Liu and H HuangldquoOcclusion-aware real-time object trackingrdquo IEEE Transactionson Multimedia vol 19 no 4 pp 763ndash771 2017
[37] C O Conaire N E OrsquoConnor and A Smeaton ldquoThermo-visual feature fusion for object tracking using multiple spa-tiogram trackersrdquo Machine Vision amp Applications vol 19 no5-6 pp 483ndash494 2008
[38] INO ldquoInorsquos video analytics datasetrdquo httpswwwinocaenvideo-analytics-dataset
[39] C Li X Liang Y Lu Z Nan and T Jin ldquoRGB-T objecttracking benchmark and baselinerdquo IEEE Transactions on ImageProcessing 2018
[40] C O Conaire N E OrsquoConnor and A F Smeaton ldquoDetectoradaptation by maximising agreement between independentdata sourcesrdquo in Proceedings of the 2007 IEEE ComputerSociety Conference on Computer Vision and Pattern RecognitionCVPRrsquo07 pp 1ndash6 IEEE Minneapolis MN USA June 2007
[41] J W Davis and V Sharma ldquoBackground-subtraction usingcontour-based fusion of thermal and visible imageryrdquoComputerVision and Image Understanding vol 106 no 2-3 pp 162ndash1822007
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom
2 Mathematical Problems in Engineering
candidates in different situations to jointly capture the targetappearance variations Moreover a variety of simple yeteffective features are effectively integrated into the learningprocess of filters to further improve the discriminative powerof the filters In the salient-sparse-collaborative tracker [10]an object salient feature map is built to create a salient-sparsediscriminative model and a salient-sparse generative modelto both handle the appearance variation and reduce track-ing drifts effectively A multilayer convolutional network-based visual tracking algorithm based on important regionselection [11] is proposed to build high entropy selectionand background discrimination models and to obtain thefeature maps by weighting the template filters with clusterweights which enables the training samples to be informativein order to provide enough stable information and also bediscriminative so as to resist distractors Generally speakingdiscriminative and generative methods have complementaryadvantages in appearance modeling and the success of avisual tracking method depends not only on its represen-tation ability against appearance variations but also on thediscriminability between target and background thus leadingto the requirement of a more robust training model [12]
Recently discriminative correlation filter- (DCF-) basedvisual tracking methods [13ndash18] have shown excellent per-formances on real-time visual tracking for its advantageof robustness and computational efficiency The DCF-basedmethods work by learning an optimal correlation filter usedto locate the target in the next frame The significant gainin speed is obtained by exploiting the fast Fourier transform(FFT) at both learning and detection stages [14] Bolmeet al [13] presented an adaptive correlation filter namedMinimum Output Sum of Squared Error (MOSSE) filterwhich produces stable correlation filters by optimizing theoutput sum of squared error Based on MOSSE Danelljan etal [14 15] proposed a novel scale adaptive tracking approachby learning separate discriminative correlation filters fortranslation and scale estimation which achieves accurate androbust scale estimation in a tracking-by-detection frame-work Galoogahi et al [16] proposed a computationallyefficient Background-aware correlation filter-based on hand-crafted features that can efficiently model how both the fore-ground and background of the object varies over time Thework in [17] reformulates DCFs as a one-layer convolutionalneural network composed of integrates feature extractionresponse map generation and model update with residuallearning Johnander et al [18] proposed a unified formulationfor learning a deformable convolution filter in which thedeformable filter is represented as a linear combination ofsubfilters and both the subfilter coefficients and their relativelocations are inferred jointly in our formulation Howeverthe above trackers fail when the target undergoes severeappearance changes due to limited data supplied by singlefeatures
Multiple feature fusion contains more useful informationthan single feature thus providing higher precision certaintyand reliability for visual tracking Wu et al [19] proposed adata fusion approach via sparse representation with applica-tions to robust visual tracking Uzkent et al [20] proposedan adaptive fusion tracking method that combines likelihood
maps from multiple bands of hyperspectral imagery intoone single more distinctive representation which increasesthe margin between mean value of foreground and back-ground pixels in the fused map Chan et al [21] proposed arobust adaptive fusion tracking method which incorporatesa novel complex cell into the group of object representationto enhance the global distinctiveness Feature fusion alsoachieves superior performances on correlation filter-basedtracking For example Rapuru et al [22] proposed a robusttracking algorithm by efficiently fusing tracking learningand detection with the systematic model update strategy ofkernelized correlation filter tracker
Although much efforts have been made single-sensorfeature fusion-based tracking suffer low accuracy and robust-ness due to the lack of diversity information Fusion of visibleand infrared sensors one of the typical multisensor coopera-tions provides complementarily useful features which is ableto achieve a more robust and accurate tracking result [23] Liet al [24] designed a fusion scheme containing joint sparserepresentation and colearning update model to fuse colorvisual spectrum and thermal spectrum images for objecttracking Li et al [25] proposed an adaptive fusion schemebased on collaborative sparse representation in Bayesianfiltering framework for online tracking Mangale and Kham-bete [26] developed reliable camouflaged target detection andtracking system using fusion of visible and infrared imagingYun et al [23] proposed a compressive time-space Kalmanfusion tracking with time-space adaptability for visible andinfrared images and introduced extended Kalman filter toupdate fusion coefficients optimally A visible and infraredfusion tracking algorithm based on multiview multikernelfusionmodel is presented in [27] Zhang et al [28] transferredvisible tracking data to infrared data to obtain better trackingperformances Lan et al [29] proposed joint feature learningand discriminative classifier framework for multimodalitytracking which jointly eliminate outlier samples caused bylarge variations and learn discriminability-consistent featuresfrom heterogeneous modalities Li et al [30] proposed aconvolutional neural network architecture including a two-streamConvNet and a FusionNet which proves that trackingwith visible and infrared fusion outperforms that with singlesensor in terms of accuracy and robustness
DCF-based trackers have significant low computationalload and are especially suitable for a variety of real-timechallenging applications However most of the DCF-basedtrackers suffer low accuracy and robustness due to the lack ofdiversity information extracted from a single type of spectralimage (visible spectrum) Therefore this paper proposes adiscriminative fusion correlation learning model to improveDCF-based tracking performance by combining multiplefeatures from visible and infrared imaging sensors The maincontributions of our work are summarized as follows
(i) A discriminative fusion correlation learning modelis presented to fuse visible and infrared featuressuch that valuable information from all sensors ispreserved
(ii) The proposed fusion learning filters are obtainedvia late fusion with early estimation in which the
Mathematical Problems in Engineering 3
0
Fusionlearning filter
+
Visible image
Infrared image
Multi-channel featureDiscriminativefilter selection
Response mapTracking result
Update
Update
Figure 1 Discriminative fusion correlation learning for visible and infrared tracking Blue and green boxes denote the target and backgroundsamples extracted from the tracking result respectively
performances of the filters are weighted to improvethe flexibility of fusion
(iii) The proposed discriminative filter selection modelconsiders the surrounding background informationin order to increase the discriminability of the tem-plate filters so as to improve model learning
The remainder of this paper is organized as follows InSection 2 the multichannel discriminative correlation filteris introduced In Section 3 we describe our work in detailThe experimental results are presented in Section 4 Section 5concludes with a general discussion
2 Multichannel DiscriminativeCorrelation Filter
Multichannel DCF provides superior robustness and effi-ciency in dealing with challenging tracking tasks [14] Inthe multichannel DCF-based tracking algorithm 119889 channelHistogram of Oriented Gradient (HOG) features [14] fromthe target sample 119904 are extracted tomaintain diverse informa-tion During training process the goal is to learn correlationfilter ℎ which is achieved by minimizing the error of thecorrelation response compared to the desired correlationoutput 119892 as
120576 =1003817100381710038171003817100381710038171003817100381710038171003817119892 minus119889
sum119897=1
ℎ119897 lowast 1199041198971003817100381710038171003817100381710038171003817100381710038171003817
2
+ 120582119889
sum119897=1
10038171003817100381710038171003817ℎ119897100381710038171003817100381710038172 (1)
where lowast denotes circular correlation and 120582 is the weightparameter [14] 119904119897 and ℎ119897 (119897 = 1 sdot sdot sdot 119889) are the 119897-th channelfeature and the corresponding correlation filter respectivelyThe correlation output 119892 is supposed to be a Gaussianfunction with a parametrized standard deviation [14]
The minimization of (1) can be solved by minimizing (2)in the Fourier domain as
119867119897 = 119866119878119897
sum119889119896=1 119878119896119878119896 + 120582
119897 = 1 sdot sdot sdot 119889 (2)
where 119867 119866 and 119878 are the discrete Fourier transform (DFT)of ℎ 119892 and 119904 respectively The symbol bar sdot denotes complexconjugation The multiplications and divisions in (2) areperformed pointwiseThe numerator119860119897119905 and denominator 119861119905of the filter119867119897119905 in (2) are updated as
119860119897119905 = (1 minus 120578)119860119897119905minus1 + 120578119866119878119897119905 119897 = 1 sdot sdot sdot 119889
119861119905 = (1 minus 120578)119860 119905minus1 + 120578119889
sum119896=1
119878119896119878119896(3)
where 120578 is a learning rate parameterDuring tracking process the DFT of the correlation score
119910119905 of the test sample 119911119905 is computed in the Fourier domainas
119884119905 =sum119889119897=1 119860
119897
119905minus1119885119897119905119861119905minus1 + 120582 (4)
where119884119905 and119885119897119905 are theDFTs of119910119905 and 119911119897119905 respectively119860119897119905 and119861119905 are the numerator and denominator of the filter updated inthe previous frame respectively Then the correlation scores119910119905 is obtained by taking the inverse DFT 119910119905 = Fminus1119884119905 Theestimate of the current target state is obtained by finding themaximum correlation score among the test samples
3 Proposed Discriminative FusionCorrelation Learning
In this section we introduce the tracking framework of theproposed algorithm the general scheme ofwhich is describedin Figure 1 Firstly the multichannel features are extractedrespectively from the visible and infrared images accordingto [14] Secondly the proposed discriminative filter selectionand the fusion filter learning are applied to get the fusionresponse map Finally the discriminative filters and fusionfilters are updated via the tracking result obtained by theresponse map We will discuss them specifically below
4 Mathematical Problems in Engineering
31 Discriminative Filter Selection According to DCF-basedtrackers we obtain the correlation output 119892 by
119892 =119889
sum119897=1
ℎ119897119879 lowast 119904119897119879 119897 = 1 sdot sdot sdot 119889 (5)
where 119904119897119879 and ℎ119897119879 are the target sample and target correlationfilter corresponding to the 119897-th channel feature among 119889channels respectively In this paper 119892 is selected as a 2DGaussian function where 120590 = 20 [13]
Before tracking we need to choose the optimal targetcorrelation filters in the training step via minimizing (6) as
119867119897119879 =119866119878119897119879
sum119889119896=1 119878119896
119879119878119896119879 + 120582 119897 = 1 sdot sdot sdot 119889 (6)
where 119878119897119879 119867119897119879 and 119866 are the DFTs of 119904119897119879 ℎ119897119879 and 119892respectively
Different from a single training sample of the targetappearance multiple background samples at different loca-tions around target need to be considered to maintain a sta-ble model However extracting multichannel features fromeach background sample increase computational complexsignificantly Moreover in practice single channel featuresfrom multiple background samples are enough to presentsatisfied performances Therefore in this paper we extract119872 background samples randomly in the range of an annulusaround the target location [11] and obtain the correlationoutput 119892 as
119892 =119872
sum119898=1
ℎ119861 lowast 119904119898119861 119898 = 1 sdot sdot sdot 119872 (7)
where 119904119898119861 119898 = 1 sdot sdot sdot 119872 denotes the119898-th background sam-ple
Similarly the optimal background correlation filters inthe training step are selected via minimizing (8) as
119867119898119861 = 119866119878119898119861sum119872119895=1 119878
119895
119861119878119895119861 + 120582 119898 = 1 sdot sdot sdot 119872 (8)
where 119878119898119861 and119867119898119861 are the DFTs of 119904119898119861 and ℎ119898119861 respectivelyWhile tracking DFT 119884119905119890119904 of the estimated discriminative
correlation score 119910119905119890119904 of the test sample 119911119905 is defined as
119884119905119890119904 =119884119905119890119904119879119884119905119890119904119861
=(sum119889119897=1 119860
119897
119905minus1119879119885119897119905) (119861119905minus1119879 + 120582)(sum119872119898=1 119860
119898
119905minus1119861119885119905) (119861119905minus1119861 + 120582)
=(sum119889119897=1 119860
119897
119905minus1119879119885119897119905) 119861119905minus1119861(119885119905sum119872119898=1 119860
119898
119905minus1119861) (119861119905minus1119879 + 120582)
(9)
where 119884119905119890119904119879 and 119884119905119890119904119861 are the DFTs of the target andbackground correlation scores 119910119905119890119904119879 and 119910119905119890119904119861 respectivelyand 119885119897119905 is the DFTs of 119911119897119905 119860119897119905minus1119879 and 119861119905minus1119879 denote thenumerator and denominator of the filter 119867119897119879 in (6) 119860119898119905minus1119861
and119861119905minus1119861 denote the numerator and denominator of the filter119867119898119861 in (8)Then the discriminative correlation scores 119910119905119890119904 areobtained by taking the inverse DFT 119910119905119890119904 = Fminus1119884119905119890119904 Theestimate of the current target state 1199111199050 119890119904 is obtained by findingthe maximum correlation score among the test samples as1199111199050 119890119904 997888rarr 1199050 1199050 = arg max119905119910119905119890119904
32 Fusion Learning Filter As proved by Wagner et al [31]late fusion with early estimation provides better performancethan early fusion with late estimation Based on this conclu-sion we use the discriminative correlation filters to obtainthe estimate of target location in visible and infrared imagesrespectively and then do fusion with the fusion correlationfilters Let 119911119905119890119904119894 119894 isin V119894 119894119903 denote the estimate of targetlocation of visible V119894 or infrared 119894119903 images Then we definethe region119877∘(sdot ∙) denoting theminimumbounding rectanglethat contains the regions of samples sdot and ∙ in image ∘Thus we extract the fusion test samples through 119911119905119894 isin119877119894(119911119905119890119904V119894 119911119905119890119904119894119903) 119894 isin V119894 119894119903 and define the DFT 119884119905119891119906 ofthe fusion correlation score 119910119905119891119906 of fusion sample 119911119905119894 that isdefined as
119884119905119891119906 =119884119905119879119891119906119884119905119861119891119906
= sum119894isinV119894119894119903 120572119894119884119905119890119904119879119894sum119894isinV119894119894119903 120572119894119884119905119890119904119861119894
=sum119894isinV119894119894119903 120572119894 ((sum119889119897=1 119860
119897
119905minus1119879119894119885119897119905119894) (119861119905minus1119879119894 + 120582))sum119894isinV119894119894119903 120572119894 ((sum119872119898=1 119860
119898
119905minus1119861119894119885119905119894) (119861119905minus1119861119894 + 120582))
= sum119894isinV119894119894119903
120572119894 (sum119889119897=1 119860119897
119905minus1119879119894119885119897119905119894) 119861119905minus1119861119894(119885119905119894sum119872119898=1 119860
119898
119905minus1119861119894) (119861119905minus1119879119894 + 120582)
(10)
where 119884119905119891119906119879 and 119884119905119891119906119861 are the DFTs of the target and back-ground fusion correlation scores 119910119905119891119906119879 and 119910119905119891119906119861 respec-tively 119911119897119905119894 isin 119877119894119897(119911119905119890119904V119894 119911119905119890119904119894119903) 119894 isin V119894 119894119903 in which 119894119897 is the 119897-thchannel feature of image 119894 119885119897119905119894 and 119885119905119894 are the DFTs of sam-ples 119911119897119905119894 and 119911119905119894 respectively120572119894 denotes the imageweights thatare denoted as
120572i =max119894119910119905119890119904119894
sum119894isinV119894119894119903max119894119910119905119890119904119894 (11)
where 119910119905119890119904119894 is the correlation score of the 119894-th image com-puted by (9)
After obtaining 119884119905119891119906 the fusion correlation score 119910119905119891119906is obtained by taking the inverse DFT 119910119905119891119906 = Fminus1119884119905119891119906The fusion location of the current target state is obtainedby finding the maximum correlation score among the testsamples
The whole tracking process of DFCL is summarized inAlgorithm 1
4 Experiments
The proposed DFCL algorithm was tested on several chal-lenging real-world sequences and some qualitative andquantitative analyses were performed on the tracking resultsin this section
Mathematical Problems in Engineering 5
InputThe 119905-th visible and infrared imagesFor 119905 = 1 to number of frames do
1 Crop the samples and extract the 119897-th (119897 = 1 sdot sdot sdot 119889) channel features 119911119897119905 for visible andinfrared images respectively
2 Compute the discriminative correlation scores 119910119905119890119904 using Eq (9)3 Compute the fusion correlation scores 119910119905119891119906 using Eq (10)4 Obtain the tracking result by maximizing 1199101199051198911199065 Extract the 119897-th (119897 = 1 sdot sdot sdot 119889) channel feature 119904119897119879 of the target samples and the119898-th
(119898 = 1 sdot sdot sdot119872) sample 1199041198971198616 Update the discriminative correlation filters119867119897119879 and119867119898119861 using Eq (6) and Eq (8)
respectivelyend forOutput Target result and the discriminative correlation filters119867119897119879 and119867119898119861
Algorithm 1 Discriminative fusion correlation learning for visible and infrared tracking
41 Experimental Environment and Evaluation CriteriaDFCL was implemented with C++ programming languageandNet Framework 40 in Visual Studio 2010 on an IntelDual-Core 170GHz CPU with 4 GB RAM Two metricsie location error (pixel) and overlapping rate are usedto evaluate the tracking results quantitatively The locationerror is computed as 119890119903119903119900119903 = radic(119909119866 minus 119909119879)2 + (119910119866 minus 119910119879)2where (119909119866 119910119866) and (119909119879 119910119879) are the ground truth (eitherdownloaded from a standard database or located manually)and tracking bounding box centers respectivelyThe trackingoverlapping rate is defined as 119900V119890119903119897119886119901119901119894119899119892 = 119886119903119890119886(119877119874119868119866 cap119877119874119868119879)119886119903119890119886(119877119874119868119866cup119877119874119868119879) where119877119874119868119866 and119877119874119868119879 denote theground truth and tracking bounding box respectively and119886119903119890119886(sdot) is the rectangular area function A smaller locationerror and a larger overlapping rate indicate higher accuracyand robustness
42 Experimental Results The performance of DFCL wascompared with state-of-the-art trackers Struck [32] ODFS[33] STC [34] KCF [35] ROT [36] DCF-based trackersMOSSE [13] DSST [14] fDSST [15] and visible-infraredfusion trackers TSKF [23] MVMKF [27] L1-PF [19] JSR[24] and CSR [25] Figures 2ndash6 present the experimentalresults of the test trackers in challenging visible sequencesnamed Biker [37] Campus [37] Car [38] Crossroad [39]Hotkettle [39] Inglassandmobile [39] Labman [40] Pedes-trian [41] and Runner [38] as well as their correspondinginfrared sequences Biker-ir Campus-ir Car-ir Crossroad-ir Labman-ir Hotkettle-ir Inglassandmobile-ir Pedestrian-ir and Runner-ir Single-sensor trackers were separatelytested on visible and the corresponding infrared sequenceswhile visible-infrared fusion trackers obtain the results withinformation fromboth visible and infrared sequences For theconvenience of presentation some tracking curves are notshown entirely in the Figures Next the performance of thetrackers in each sequence is described in detail
(a) SequencesBiker andBiker-irBiker presents the exam-ple of complex background clutters The target human in thevisible sequence encounters similar background disturbance(ie bikes) which causes the ODFS MOSSE fDSST TSKF
and MVMKF trackers to drift away from the target Thecorresponding infrared sequence Biker-ir provides temper-ature information that eliminates the background clutter inBiker But when the target is approaching another person ataround Frame 20 Struck ODFS STC MOSSE TSKF andMVMKF do not perform well because they are not able todistinguish target from persons with similar temperature ininfrared sequences Only KCF ROT DSST and our DFCLhave achieved precise and robust performances in thesesequences
(b) Sequences Campus and Campus-ir the target inCampus and Campus-ir undergoes background cluttersocclusion and scale variation At the beginning of CampusODFS STC KCF and ROT lose the target due to backgrounddisturbance Only TSKF and DFCL perform well whileStruck fDSST and MVMKF do not achieve accurate resultsBecause of the infrared information provided by Campus-irfewer test trackers lose tracking when background cluttershappen as shown in Figure 2 But Struck KCF and ROTmistake another person for the target As shown in Figure 2most of the trackers result in tracking failures whereas DFCLoutperforms the others in most metrics (location accuracyand success rate)
(c) SequencesCar andCar-irCar andCar-ir demonstratethe efficiency of DFCL on coping with heavy occlusions Thetarget driving car is occluded by lampposts and trees manytimes which cause tracking failures of most trackers OnlyTSKF MVMKF and DFCL are able to handle the occlusionthroughout the tracking process in this sequence As shownin Figure 2most trackers performbetter inCar-ir than inCarbecause the infrared features can overcome the difficultiesof target detection among surrounding similar backgroundSTC TSKF MVMKF and DFCL are able to handle thisproblem whereas the result of DFCL is the most accurate asshown in Figure 2
(d) Sequences Crossroad and Crossroad-ir the target inCrossroad and Crossroad-ir undergoes heavy backgroundclutters when she crosses the roadWhile the target is passingby the road lamp both ODFS and JSR lose the target Thenwhen a car passes by the target Struck TSKF and MVMKFdrift away from the target When the target goes toward
6 Mathematical Problems in Engineering
20 53 147
Biker
20 53 147
Biker-ir
8 84 156
Campus
8 84 156
Campus-ir
14 23 27
Car
14 23 27
Car-ir
136 289 767
Crossroad
136 289 767
Crossroad-ir
133 451 1068
Hotkettle
133 451 1068
Hotkettle-ir
309 428 511
Inglassandmobile
309 428 511
Inglassandmobile-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF
302 320 446
Labman
302 320 446
Labman-ir
29 228 363
Pedestrian
29 228 363
Pedestrian-ir
14 94 138
Runner
14 94 138
Runner-ir
L1-PF JSR CSR
Figure 2 Tracking performances of the test sequences
Mathematical Problems in Engineering 7
Biker
0
40
80
1 41 81 121
Biker-ir
0
50
100
150
1 41 81 121
Campus
0
50
100
150
200
1 51 101 151 201
Campus-ir
0
100
200
300
1 51 101 151 201
Car
0
100
200
1 11 21 31
Car-ir
0
100
200
300
1 11 21 31
0
50
100
150
200
250
1 201 401 6010
100
200
1 201 401 601
Crossroad Crossroad-ir
0
50
100
150
200
250
300
1 201 401 601 801 1001
Hotkettle
0
50
100
150
200
250
1 201 401 601 801 1001
Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0
50
100
150
200
1 201 401 6010
50
100
150
1 201 401 601
Labman
0
100
200
300
1 101 201 301 401
Labman-ir
0
200
400
1 101 201 301 401
Pedestrian
0
50
100
150
1 101 201 301
Pedestrian-ir
0
50
100
150
1 101 201 301
Runner
0
100
200
1 61 121 181
Runner-ir
0
100
200
1 61 121 181
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
Figure 3 Location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
0
05
1
1 41 81 1210
05
1
1 41 81 1210
05
1
1 51 101 151 2010
05
1
1 51 101 151 2010
05
1
1 11 21 310
05
1
1 11 21 31
0
05
1
1 201 401 6010
05
1
1 201 401 6010
05
1
1 201 401 601 801 10010
05
1
1 201 401 601 801 10010
05
1
1 201 401 6010
05
1
1 201 401 601
0
05
1
1 101 201 301 4010
05
1
1 101 201 301 4010
05
1
1 101 201 3010
05
1
1 101 201 3010
05
1
1 61 121 1810
05
1
1 61 121 181
Figure 4 Overlapping rate of the test sequences
the sidewalk most of the trackers are not able to handlethe problem of heavy background clutters but our trackerperforms satisfying tracking results as shown in Figures 2ndash4
(e) Sequences Hotkettle and Hotkettle-ir in thesesequences tracking is hard because of the changes of thecomplex background clutters Most trackers perform betterin Hotkettle-ir than in Hotkettle for the reason that thetemperature diverge makes the hot target more distinct inthe cold background Struck KCF DSST fDSST and DFCL
can achieve robust and accurate tracking performances asshown in Figures 2ndash4
(f) Sequences Inglassandmobile and Inglassandmobile-irSequences Inglassandmobile and Inglassandmobile-ir demon-strate the performances of the 14 trackers under the cir-cumstances of background clutters illumination changesand occlusion As shown in Figure 2 when the illuminationchanges at around Frames 300 ODFS and fDSST lose thetarget and KCF TSKF and L1-PF drift a little away from the
8 Mathematical Problems in Engineering
10048
96602
171062
9467 27351 20776 17357
3462 21838 17229
2866
241288
9947 6518
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Biker Biker-ir Campus Campus-ir
4536
114901
6739 4323 3944
187420
3574 3805 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2758 9391
287670
2255 4284
216795
1778 2248 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
6301
165743 160172
122008 116442
151244
7767 7767 3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
59762
9866
138565
63421 63817
145486
11886 16255
3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Car Car-ir
122137
109606
122092 122500 122514
106350
121940 122072
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
25018 22604 24216
122500
55885
106350
121940 122530
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Labman Labman-ir
19128
263409
304244
34664 26765
390209
27024 18464 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
7160 16449
377858
7428 38194
497072
21473 12348 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Pedestrian Pedestrian-ir Runner Runner-ir
117051 116737 120324
142077 137828 144675
121509
144107
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
9457
55577
141371 142012
43182
435192
141348 141350
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
116065
92197
30636
129168
27984
100629
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2876
117247
2341
127301
3465
297251
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
9575
300997
60040
11480 15424
287956
19187 2840
21838 17229 2866
241288
9947 4246
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
3638
79438 54283
4987
55103
313867
3023 4995 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
5983 9604
28494
4086 5710
196150
2693 4255 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
6687
66141
48306
16131
63235
116459
14736
74415
18142 10318 10791 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
4160
16714 26378
16071
3345
130137
14534
60554
18142 10318 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
10791
Figure 5 Average location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0993
0311
0986 0993 0993
0108
0993 0993 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
1000 0973
0014
1000 1000
0108
1000 1000 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0907
0005 0000 0056 0070
0009
0986 0986 1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0437
1000
0023
0158 0186
0028
0935
0758
1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0059
0176
0088 0088 0088 0059
0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0824
0382
0941
0088
0471
0059 0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0847
0026 0026
0520
0714
0022
0677 0733
0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0991
0877
0004
0994
0011 0009
0708
0959 0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0068 0054 0078 0049 0054 0044
0083 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0605
0020 0054 0054
0502
0024 0054 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0101
0189
0481
0016
0598
0044
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0937
0005
0904
0019
0943
0033
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0823
0016
0186
0808
0585 0585 0528
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0823
0016
0302
0808 0819
0016
0472
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0585
0105
0331
0561
0318
0000
0585 0558 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0571
0222
0454
0585 0578
0006
0585 0585 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976
0035
0749
0884
0587
0118
0917
0245
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976 0948
0604
0887
1000
0089
0925
0580
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Figure 6 Success rate of the test sequences
targetWhen the target is approaching a tree the backgroundclutters makes most of trackers cause tracking failures thatcan be seen from Figure 2 Our DFCL can overcome thesechallenges and perform well in these sequences
(g) Sequences Labman and Labman-ir the experimentsin Sequences Labman and Labman-ir aim to evaluate the per-formances on tracking under appearance variation rotationscale variation and background clutter In Labman whenthe target man is walking into the laboratory ODFS STCand MOSSE lose the target When the man keeps shaking
and turning around his head at around Frame 400 KCFROT and DSST cause tracking failures Also most trackersachieve better tracking performances in Labman-ir as shownin Figure 2
(h) Sequences Pedestrian and Pedestrian-ir the target inPedestrian and Pedestrian-ir undergoes heavy backgroundclutters and occlusion As shown in Figure 2 other trackersresult in tracking failures in Pedestrian whereas our trackershows satisfying performances in terms of both accuracyand robustnessThe efficient infrared features extracted from
Mathematical Problems in Engineering 9
DFCLDCL MOSSE
Average location error (pixel) Success rate
011 001 006
059
000 012
002 004 004
052
019 017
063
021
073
039 053
037
100 100
082 082
057
099 100 099 099
18742 15124
10635
2078
31387
11646
39021
10063 14467
1853 6166 8940
1786
9545 4990 5658
2692 9228
340 315 1899 652 495 723 1013 138 284
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Figure 7 Average location error (pixel) and success rate of DCL DFCL and MOSSE on the test sequences
Pedestrian-ir ensure the tracking successes of Struck STCand DFCL as can be seen from Figures 2ndash4
(i) Sequences Runner and Runner-ir Runner and Runner-ir contain examples of heavy occlusion abrupt movementand scale variation The target running man is occludedby lampposts trees stone tablet and bushes many timesresulting in tracking failures ofmost trackers Also the abruptmovement and scale variation cause many trackers to driftaway the target in both Runner and Runner-ir as shown inFigure 2 Once again ourDFCL is able to overcome the aboveproblems and achieve good performances
Figures 5 and 6 are included here to demonstrate quan-titatively the performances on average location error (pixel)and success rate The success rate is defined as the numberof times success is achieved in the whole tracking processby considering one frame as a success if the overlapping rateexceeds 05 [33] A smaller average location error and a largersuccess rate indicate increased accuracy and robustnessFigures 5 and 6 show that DFCL performs satisfying most ofthe tracking sequences
To validate the effectiveness of the discriminative filterselection model of DFCL we compare the tracker DCL(the proposed DFCL without the fusion learning model)with DFCL and the original DCF tracker MOSSE on visiblesequencesThe performances shown in Figure 7 demonstratethe efficiency of the discriminative filter selection modelespecially in the sequences with background clutters ieSequences BikerHotkettle Inglassandmobile and Pedestrian
5 Conclusion
Discriminative correlation filter- (DCF-) based trackers havethe advantage of being computationally efficient and morerobust than most of the other state-of-the-art trackers inchallenging tracking tasks thereby making them especiallysuitable for a variety of real-time challenging applications
Howevermost of theDCF-based trackers suffer low accuracydue to the lack of diversity information extracted from asingle type of spectral image (visible spectrum) Fusion ofvisible and infrared sensors one of the typical multisensorcooperation provides complementarily useful features andconsistently helps recognize the target from the backgroundefficiently in visual tracking For the above reasons this paperproposes a discriminative fusion correlation learning modelto improve DCF-based tracking performance by combiningmultiple features from visible and infrared imaging sensorsThe proposed fusion learning filters are obtained via latefusion with early estimation in which the performances ofthe filters are weighted to improve the flexibility of fusionMoreover the proposed discriminative filter selection modelconsiders the surrounding background information in orderto increase the discriminability of the template filters soas to improve model learning Numerous real-world videosequences were used to test DFCL and other state-of-the-art algorithms and here we only selected representativevideos for presentation Experimental results demonstratedthat DFCL is highly accurate and robust
Data Availability
The data used to support the findings of this study weresupplied by China University of Mining and Technologyunder license and so cannot be made freely available
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
This work is supported by the Natural Science Foun-dation of Jiangsu Province (BK20180640 BK20150204)
10 Mathematical Problems in Engineering
Research Development Programme of Jiangsu Province(BE2015040) the State Key Research Development Program(2016YFC0801403) and the National Natural Science Foun-dation of China (51504214 51504255 51734009 61771417 and61873246)
References
[1] S A Wibowo H Lee E K Kim and S Kim ldquoVisual trackingbased on complementary learners with distractor handlingrdquoMathematical Problems in Engineering vol 2017 Article ID5295601 13 pages 2017
[2] T Zhou M Zhu D Zeng and H Yang ldquoScale adaptive kernel-ized correlation filter tracker with feature fusionrdquoMathematicalProblems in Engineering vol 2017 Article ID 1605959 8 pages2017
[3] C Wang H K Galoogahi C Lin and S Lucey ldquoDeep-LK forefficient adaptive object trackingrdquo Computer Vision and PatternRecognition 2017
[4] Z Chi H Li H Lu and M-H Yang ldquoDual deep network forvisual trackingrdquo IEEE Transactions on Image Processing vol 26no 4 pp 2005ndash2015 2017
[5] Y Yang W Hu Y Xie W Zhang and T Zhang ldquoTemporalrestricted visual tracking via reverse-low-rank sparse learningrdquoIEEE Transactions on Cybernetics vol 47 no 2 pp 485ndash4982017
[6] Y Sui G Wang and L Zhang ldquoCorrelation filter learningtoward peak strength for visual trackingrdquo IEEE Transactions onCybernetics vol 48 no 4 pp 1290ndash1303 2018
[7] K Zhang X Li H Song Q Liu and W Lian ldquoVisual track-ing using spatio-temporally nonlocally regularized correlationfilterrdquo Pattern Recognition vol 83 pp 185ndash195 2018
[8] Y Qi L Qin J Zhang S Zhang Q Huang and M-H YangldquoStructure-aware local sparse coding for visual trackingrdquo IEEETransactions on Image Processing vol 27 no 8 pp 3857ndash38692018
[9] B Bai B Zhong G Ouyang et al ldquoKernel correlation filtersfor visual tracking with adaptive fusion of heterogeneous cuesrdquoNeurocomputing vol 286 pp 109ndash120 2018
[10] Y Liu F Yang C Zhong Y Tao B Dai and M Yin ldquoVisualtracking via salient feature extraction and sparse collaborativemodelrdquo AEU - International Journal of Electronics and Commu-nications vol 87 pp 134ndash143 2018
[11] X Yun Y Sun S Wang Y Shi and N Lu ldquoMulti-layerconvolutional network-based visual tracking via importantregion selectionrdquo Neurocomputing vol 315 pp 145ndash156 2018
[12] X Lan S Zhang and P C Yuen ldquoRobust joint discriminativefeature learning for visual trackingrdquo in Proceedings of the 25thInternational Joint Conference on Artificial Intelligence IJCAI2016 pp 3403ndash3410 AAAI Press New York NY USA July2016
[13] D S Bolme J R Beveridge B A Draper and Y M LuildquoVisual object tracking using adaptive correlation filtersrdquo inProceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo10) pp 2544ndash2550 IEEE SanFrancisco Calif USA June 2010
[14] M Danelljan G Hager F Shahbaz Khan and M FelsbergldquoAccurate scale estimation for robust visual trackingrdquo in Pro-ceedings of the British Machine Vision Conference pp 651-6511BMVA Press Nottingham UK 2014
[15] M Danelljan G Hager F S Khan and M Felsberg ldquoDis-criminative scale space trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 39 no 8 pp 1561ndash15752017
[16] H K Galoogahi A Fagg and S Lucey ldquoLearning background-aware correlation filters for visual trackingrdquo inProceedings of the16th IEEE International Conference on Computer Vision ICCV2017 pp 1144ndash1152 IEEE Istanbul Turkey 2018
[17] Y SongCMa LGong J Zhang RWH Lau andM-HYangldquoCrest convolutional residual learning for visual trackingrdquoin Proceedings of the 16th IEEE International Conference onComputer Vision ICCV 2017 pp 2574ndash2583 IEEEVenice ItalyOctober 2017
[18] J Johnander M Danelljan F Khan andM Felsberg ldquoDCCOtowards deformable continuous convolution operators forvisual trackingrdquo in Proceedings of the International Conferenceon Computer Analysis of Images and Patterns pp 55ndash67Springer Ystad Sweden 2017
[19] Y Wu E Blasch G Chen L Bai and H Ling ldquoMultiplesource data fusion via sparse representation for robust visualtrackingrdquo in Proceedings of the 14th International Conference onInformation Fusion Fusion 2011 IEEE Chicago IL USA July2011
[20] B Uzkent A Rangnekar and M J Hoffman ldquoAerial vehicletracking by adaptive fusion of hyperspectral likelihood mapsrdquoin Proceedings of the 30th IEEE Conference on Computer Visionand Pattern RecognitionWorkshops CVPRW 2017 pp 233ndash242IEEE Honolulu HI USA July 2017
[21] S Chan X Zhou and S Chen ldquoRobust adaptive fusion trackingbased on complex cells and keypointsrdquo IEEE Access vol 5 pp20985ndash21001 2017
[22] M K Rapuru S Kakanuru P M Venugopal D Mishra and GR Subrahmanyam ldquoCorrelation-based tracker-level fusion forrobust visual trackingrdquo IEEE Transactions on Image Processingvol 26 no 10 pp 4832ndash4842 2017
[23] X Yun Z Jing G Xiao B Jin and C Zhang ldquoA compressivetracking based on time-space Kalman fusion modelrdquo ScienceChina Information Sciences vol 59 no 1 pp 1ndash15 2016
[24] H P Liu and F C Sun ldquoFusion tracking in color and infraredimages using joint sparse representationrdquo Science China Infor-mation Sciences vol 55 no 3 pp 590ndash599 2012
[25] C Li H Cheng S Hu X Liu J Tang and L Lin ldquoLearningcollaborative sparse representation for grayscale-thermal track-ingrdquo IEEE Transactions on Image Processing vol 25 no 12 pp5743ndash5756 2016
[26] S Mangale and M Khambete ldquoCamouflaged target detectionand tracking using thermal infrared and visible spectrum imag-ingrdquo in Automatic Diagnosis of Breast Cancer using Thermo-graphic Color Analysis and SVM Classifier vol 530 of Advancesin Intelligent Systems and Computing pp 193ndash207 SpringerInternational Publishing 2016
[27] X Yun Z Jing and B Jin ldquoVisible and infrared tracking basedon multi-view multi-kernel fusion modelrdquo Optical Review vol23 no 2 pp 244ndash253 2016
[28] L Zhang A Gonzalez-Garcia J van de Weijer M Danelljanand F S Khan ldquoSynthetic data generation for end-to-end ther-mal infrared trackingrdquo IEEE Transactions on Image Processingvol 28 no 4 pp 1837ndash1850 2019
[29] X Lan M Ye S Zhang and P C Yuen ldquoRobust collaborativediscriminative learning for rgb-infrared trackingrdquo in Proceed-ings of the 32th AAAI Conference on Artificial Intelligence pp7008ndash7015 AAAI New Orleans LA USA 2018
Mathematical Problems in Engineering 11
[30] C Li X Wu N Zhao X Cao and J Tang ldquoFusing two-streamconvolutional neural networks for RGB-T object trackingrdquoNeurocomputing vol 281 pp 78ndash85 2018
[31] JWagner V FischerMHerman and S Behnke ldquoMultispectralpedestrian detection using deep fusion convolutional neuralnetworksrdquo in Proceedings of the 24th European Symposium onArtificial Neural Networks pp 1ndash6 Bruges Belgium 2016
[32] S Hare A Saffari and P H S Torr ldquoStruck structured outputtracking with kernelsrdquo in Proceedings of the IEEE InternationalConference on Computer Vision pp 263ndash270 IEEE ProvidenceRI USA 2012
[33] K Zhang L Zhang andM-H Yang ldquoReal-time object trackingvia online discriminative feature selectionrdquo IEEE Transactionson Image Processing vol 22 no 12 pp 4664ndash4677 2013
[34] K Zhang L Zhang M Yang and D Zhang ldquoFast trackingvia spatio- temporal context learningrdquo Computer Vision andPattern Recognition 2013
[35] J F Henriques R Caseiro P Martins and J Batista ldquoHigh-speed tracking with kernelized correlation filtersrdquo IEEE Trans-actions on Pattern Analysis and Machine Intelligence vol 37 no3 pp 583ndash596 2015
[36] X Dong J Shen D Yu W Wang J Liu and H HuangldquoOcclusion-aware real-time object trackingrdquo IEEE Transactionson Multimedia vol 19 no 4 pp 763ndash771 2017
[37] C O Conaire N E OrsquoConnor and A Smeaton ldquoThermo-visual feature fusion for object tracking using multiple spa-tiogram trackersrdquo Machine Vision amp Applications vol 19 no5-6 pp 483ndash494 2008
[38] INO ldquoInorsquos video analytics datasetrdquo httpswwwinocaenvideo-analytics-dataset
[39] C Li X Liang Y Lu Z Nan and T Jin ldquoRGB-T objecttracking benchmark and baselinerdquo IEEE Transactions on ImageProcessing 2018
[40] C O Conaire N E OrsquoConnor and A F Smeaton ldquoDetectoradaptation by maximising agreement between independentdata sourcesrdquo in Proceedings of the 2007 IEEE ComputerSociety Conference on Computer Vision and Pattern RecognitionCVPRrsquo07 pp 1ndash6 IEEE Minneapolis MN USA June 2007
[41] J W Davis and V Sharma ldquoBackground-subtraction usingcontour-based fusion of thermal and visible imageryrdquoComputerVision and Image Understanding vol 106 no 2-3 pp 162ndash1822007
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom
Mathematical Problems in Engineering 3
0
Fusionlearning filter
+
Visible image
Infrared image
Multi-channel featureDiscriminativefilter selection
Response mapTracking result
Update
Update
Figure 1 Discriminative fusion correlation learning for visible and infrared tracking Blue and green boxes denote the target and backgroundsamples extracted from the tracking result respectively
performances of the filters are weighted to improvethe flexibility of fusion
(iii) The proposed discriminative filter selection modelconsiders the surrounding background informationin order to increase the discriminability of the tem-plate filters so as to improve model learning
The remainder of this paper is organized as follows InSection 2 the multichannel discriminative correlation filteris introduced In Section 3 we describe our work in detailThe experimental results are presented in Section 4 Section 5concludes with a general discussion
2 Multichannel DiscriminativeCorrelation Filter
Multichannel DCF provides superior robustness and effi-ciency in dealing with challenging tracking tasks [14] Inthe multichannel DCF-based tracking algorithm 119889 channelHistogram of Oriented Gradient (HOG) features [14] fromthe target sample 119904 are extracted tomaintain diverse informa-tion During training process the goal is to learn correlationfilter ℎ which is achieved by minimizing the error of thecorrelation response compared to the desired correlationoutput 119892 as
120576 =1003817100381710038171003817100381710038171003817100381710038171003817119892 minus119889
sum119897=1
ℎ119897 lowast 1199041198971003817100381710038171003817100381710038171003817100381710038171003817
2
+ 120582119889
sum119897=1
10038171003817100381710038171003817ℎ119897100381710038171003817100381710038172 (1)
where lowast denotes circular correlation and 120582 is the weightparameter [14] 119904119897 and ℎ119897 (119897 = 1 sdot sdot sdot 119889) are the 119897-th channelfeature and the corresponding correlation filter respectivelyThe correlation output 119892 is supposed to be a Gaussianfunction with a parametrized standard deviation [14]
The minimization of (1) can be solved by minimizing (2)in the Fourier domain as
119867119897 = 119866119878119897
sum119889119896=1 119878119896119878119896 + 120582
119897 = 1 sdot sdot sdot 119889 (2)
where 119867 119866 and 119878 are the discrete Fourier transform (DFT)of ℎ 119892 and 119904 respectively The symbol bar sdot denotes complexconjugation The multiplications and divisions in (2) areperformed pointwiseThe numerator119860119897119905 and denominator 119861119905of the filter119867119897119905 in (2) are updated as
119860119897119905 = (1 minus 120578)119860119897119905minus1 + 120578119866119878119897119905 119897 = 1 sdot sdot sdot 119889
119861119905 = (1 minus 120578)119860 119905minus1 + 120578119889
sum119896=1
119878119896119878119896(3)
where 120578 is a learning rate parameterDuring tracking process the DFT of the correlation score
119910119905 of the test sample 119911119905 is computed in the Fourier domainas
119884119905 =sum119889119897=1 119860
119897
119905minus1119885119897119905119861119905minus1 + 120582 (4)
where119884119905 and119885119897119905 are theDFTs of119910119905 and 119911119897119905 respectively119860119897119905 and119861119905 are the numerator and denominator of the filter updated inthe previous frame respectively Then the correlation scores119910119905 is obtained by taking the inverse DFT 119910119905 = Fminus1119884119905 Theestimate of the current target state is obtained by finding themaximum correlation score among the test samples
3 Proposed Discriminative FusionCorrelation Learning
In this section we introduce the tracking framework of theproposed algorithm the general scheme ofwhich is describedin Figure 1 Firstly the multichannel features are extractedrespectively from the visible and infrared images accordingto [14] Secondly the proposed discriminative filter selectionand the fusion filter learning are applied to get the fusionresponse map Finally the discriminative filters and fusionfilters are updated via the tracking result obtained by theresponse map We will discuss them specifically below
4 Mathematical Problems in Engineering
31 Discriminative Filter Selection According to DCF-basedtrackers we obtain the correlation output 119892 by
119892 =119889
sum119897=1
ℎ119897119879 lowast 119904119897119879 119897 = 1 sdot sdot sdot 119889 (5)
where 119904119897119879 and ℎ119897119879 are the target sample and target correlationfilter corresponding to the 119897-th channel feature among 119889channels respectively In this paper 119892 is selected as a 2DGaussian function where 120590 = 20 [13]
Before tracking we need to choose the optimal targetcorrelation filters in the training step via minimizing (6) as
119867119897119879 =119866119878119897119879
sum119889119896=1 119878119896
119879119878119896119879 + 120582 119897 = 1 sdot sdot sdot 119889 (6)
where 119878119897119879 119867119897119879 and 119866 are the DFTs of 119904119897119879 ℎ119897119879 and 119892respectively
Different from a single training sample of the targetappearance multiple background samples at different loca-tions around target need to be considered to maintain a sta-ble model However extracting multichannel features fromeach background sample increase computational complexsignificantly Moreover in practice single channel featuresfrom multiple background samples are enough to presentsatisfied performances Therefore in this paper we extract119872 background samples randomly in the range of an annulusaround the target location [11] and obtain the correlationoutput 119892 as
119892 =119872
sum119898=1
ℎ119861 lowast 119904119898119861 119898 = 1 sdot sdot sdot 119872 (7)
where 119904119898119861 119898 = 1 sdot sdot sdot 119872 denotes the119898-th background sam-ple
Similarly the optimal background correlation filters inthe training step are selected via minimizing (8) as
119867119898119861 = 119866119878119898119861sum119872119895=1 119878
119895
119861119878119895119861 + 120582 119898 = 1 sdot sdot sdot 119872 (8)
where 119878119898119861 and119867119898119861 are the DFTs of 119904119898119861 and ℎ119898119861 respectivelyWhile tracking DFT 119884119905119890119904 of the estimated discriminative
correlation score 119910119905119890119904 of the test sample 119911119905 is defined as
119884119905119890119904 =119884119905119890119904119879119884119905119890119904119861
=(sum119889119897=1 119860
119897
119905minus1119879119885119897119905) (119861119905minus1119879 + 120582)(sum119872119898=1 119860
119898
119905minus1119861119885119905) (119861119905minus1119861 + 120582)
=(sum119889119897=1 119860
119897
119905minus1119879119885119897119905) 119861119905minus1119861(119885119905sum119872119898=1 119860
119898
119905minus1119861) (119861119905minus1119879 + 120582)
(9)
where 119884119905119890119904119879 and 119884119905119890119904119861 are the DFTs of the target andbackground correlation scores 119910119905119890119904119879 and 119910119905119890119904119861 respectivelyand 119885119897119905 is the DFTs of 119911119897119905 119860119897119905minus1119879 and 119861119905minus1119879 denote thenumerator and denominator of the filter 119867119897119879 in (6) 119860119898119905minus1119861
and119861119905minus1119861 denote the numerator and denominator of the filter119867119898119861 in (8)Then the discriminative correlation scores 119910119905119890119904 areobtained by taking the inverse DFT 119910119905119890119904 = Fminus1119884119905119890119904 Theestimate of the current target state 1199111199050 119890119904 is obtained by findingthe maximum correlation score among the test samples as1199111199050 119890119904 997888rarr 1199050 1199050 = arg max119905119910119905119890119904
32 Fusion Learning Filter As proved by Wagner et al [31]late fusion with early estimation provides better performancethan early fusion with late estimation Based on this conclu-sion we use the discriminative correlation filters to obtainthe estimate of target location in visible and infrared imagesrespectively and then do fusion with the fusion correlationfilters Let 119911119905119890119904119894 119894 isin V119894 119894119903 denote the estimate of targetlocation of visible V119894 or infrared 119894119903 images Then we definethe region119877∘(sdot ∙) denoting theminimumbounding rectanglethat contains the regions of samples sdot and ∙ in image ∘Thus we extract the fusion test samples through 119911119905119894 isin119877119894(119911119905119890119904V119894 119911119905119890119904119894119903) 119894 isin V119894 119894119903 and define the DFT 119884119905119891119906 ofthe fusion correlation score 119910119905119891119906 of fusion sample 119911119905119894 that isdefined as
119884119905119891119906 =119884119905119879119891119906119884119905119861119891119906
= sum119894isinV119894119894119903 120572119894119884119905119890119904119879119894sum119894isinV119894119894119903 120572119894119884119905119890119904119861119894
=sum119894isinV119894119894119903 120572119894 ((sum119889119897=1 119860
119897
119905minus1119879119894119885119897119905119894) (119861119905minus1119879119894 + 120582))sum119894isinV119894119894119903 120572119894 ((sum119872119898=1 119860
119898
119905minus1119861119894119885119905119894) (119861119905minus1119861119894 + 120582))
= sum119894isinV119894119894119903
120572119894 (sum119889119897=1 119860119897
119905minus1119879119894119885119897119905119894) 119861119905minus1119861119894(119885119905119894sum119872119898=1 119860
119898
119905minus1119861119894) (119861119905minus1119879119894 + 120582)
(10)
where 119884119905119891119906119879 and 119884119905119891119906119861 are the DFTs of the target and back-ground fusion correlation scores 119910119905119891119906119879 and 119910119905119891119906119861 respec-tively 119911119897119905119894 isin 119877119894119897(119911119905119890119904V119894 119911119905119890119904119894119903) 119894 isin V119894 119894119903 in which 119894119897 is the 119897-thchannel feature of image 119894 119885119897119905119894 and 119885119905119894 are the DFTs of sam-ples 119911119897119905119894 and 119911119905119894 respectively120572119894 denotes the imageweights thatare denoted as
120572i =max119894119910119905119890119904119894
sum119894isinV119894119894119903max119894119910119905119890119904119894 (11)
where 119910119905119890119904119894 is the correlation score of the 119894-th image com-puted by (9)
After obtaining 119884119905119891119906 the fusion correlation score 119910119905119891119906is obtained by taking the inverse DFT 119910119905119891119906 = Fminus1119884119905119891119906The fusion location of the current target state is obtainedby finding the maximum correlation score among the testsamples
The whole tracking process of DFCL is summarized inAlgorithm 1
4 Experiments
The proposed DFCL algorithm was tested on several chal-lenging real-world sequences and some qualitative andquantitative analyses were performed on the tracking resultsin this section
Mathematical Problems in Engineering 5
InputThe 119905-th visible and infrared imagesFor 119905 = 1 to number of frames do
1 Crop the samples and extract the 119897-th (119897 = 1 sdot sdot sdot 119889) channel features 119911119897119905 for visible andinfrared images respectively
2 Compute the discriminative correlation scores 119910119905119890119904 using Eq (9)3 Compute the fusion correlation scores 119910119905119891119906 using Eq (10)4 Obtain the tracking result by maximizing 1199101199051198911199065 Extract the 119897-th (119897 = 1 sdot sdot sdot 119889) channel feature 119904119897119879 of the target samples and the119898-th
(119898 = 1 sdot sdot sdot119872) sample 1199041198971198616 Update the discriminative correlation filters119867119897119879 and119867119898119861 using Eq (6) and Eq (8)
respectivelyend forOutput Target result and the discriminative correlation filters119867119897119879 and119867119898119861
Algorithm 1 Discriminative fusion correlation learning for visible and infrared tracking
41 Experimental Environment and Evaluation CriteriaDFCL was implemented with C++ programming languageandNet Framework 40 in Visual Studio 2010 on an IntelDual-Core 170GHz CPU with 4 GB RAM Two metricsie location error (pixel) and overlapping rate are usedto evaluate the tracking results quantitatively The locationerror is computed as 119890119903119903119900119903 = radic(119909119866 minus 119909119879)2 + (119910119866 minus 119910119879)2where (119909119866 119910119866) and (119909119879 119910119879) are the ground truth (eitherdownloaded from a standard database or located manually)and tracking bounding box centers respectivelyThe trackingoverlapping rate is defined as 119900V119890119903119897119886119901119901119894119899119892 = 119886119903119890119886(119877119874119868119866 cap119877119874119868119879)119886119903119890119886(119877119874119868119866cup119877119874119868119879) where119877119874119868119866 and119877119874119868119879 denote theground truth and tracking bounding box respectively and119886119903119890119886(sdot) is the rectangular area function A smaller locationerror and a larger overlapping rate indicate higher accuracyand robustness
42 Experimental Results The performance of DFCL wascompared with state-of-the-art trackers Struck [32] ODFS[33] STC [34] KCF [35] ROT [36] DCF-based trackersMOSSE [13] DSST [14] fDSST [15] and visible-infraredfusion trackers TSKF [23] MVMKF [27] L1-PF [19] JSR[24] and CSR [25] Figures 2ndash6 present the experimentalresults of the test trackers in challenging visible sequencesnamed Biker [37] Campus [37] Car [38] Crossroad [39]Hotkettle [39] Inglassandmobile [39] Labman [40] Pedes-trian [41] and Runner [38] as well as their correspondinginfrared sequences Biker-ir Campus-ir Car-ir Crossroad-ir Labman-ir Hotkettle-ir Inglassandmobile-ir Pedestrian-ir and Runner-ir Single-sensor trackers were separatelytested on visible and the corresponding infrared sequenceswhile visible-infrared fusion trackers obtain the results withinformation fromboth visible and infrared sequences For theconvenience of presentation some tracking curves are notshown entirely in the Figures Next the performance of thetrackers in each sequence is described in detail
(a) SequencesBiker andBiker-irBiker presents the exam-ple of complex background clutters The target human in thevisible sequence encounters similar background disturbance(ie bikes) which causes the ODFS MOSSE fDSST TSKF
and MVMKF trackers to drift away from the target Thecorresponding infrared sequence Biker-ir provides temper-ature information that eliminates the background clutter inBiker But when the target is approaching another person ataround Frame 20 Struck ODFS STC MOSSE TSKF andMVMKF do not perform well because they are not able todistinguish target from persons with similar temperature ininfrared sequences Only KCF ROT DSST and our DFCLhave achieved precise and robust performances in thesesequences
(b) Sequences Campus and Campus-ir the target inCampus and Campus-ir undergoes background cluttersocclusion and scale variation At the beginning of CampusODFS STC KCF and ROT lose the target due to backgrounddisturbance Only TSKF and DFCL perform well whileStruck fDSST and MVMKF do not achieve accurate resultsBecause of the infrared information provided by Campus-irfewer test trackers lose tracking when background cluttershappen as shown in Figure 2 But Struck KCF and ROTmistake another person for the target As shown in Figure 2most of the trackers result in tracking failures whereas DFCLoutperforms the others in most metrics (location accuracyand success rate)
(c) SequencesCar andCar-irCar andCar-ir demonstratethe efficiency of DFCL on coping with heavy occlusions Thetarget driving car is occluded by lampposts and trees manytimes which cause tracking failures of most trackers OnlyTSKF MVMKF and DFCL are able to handle the occlusionthroughout the tracking process in this sequence As shownin Figure 2most trackers performbetter inCar-ir than inCarbecause the infrared features can overcome the difficultiesof target detection among surrounding similar backgroundSTC TSKF MVMKF and DFCL are able to handle thisproblem whereas the result of DFCL is the most accurate asshown in Figure 2
(d) Sequences Crossroad and Crossroad-ir the target inCrossroad and Crossroad-ir undergoes heavy backgroundclutters when she crosses the roadWhile the target is passingby the road lamp both ODFS and JSR lose the target Thenwhen a car passes by the target Struck TSKF and MVMKFdrift away from the target When the target goes toward
6 Mathematical Problems in Engineering
20 53 147
Biker
20 53 147
Biker-ir
8 84 156
Campus
8 84 156
Campus-ir
14 23 27
Car
14 23 27
Car-ir
136 289 767
Crossroad
136 289 767
Crossroad-ir
133 451 1068
Hotkettle
133 451 1068
Hotkettle-ir
309 428 511
Inglassandmobile
309 428 511
Inglassandmobile-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF
302 320 446
Labman
302 320 446
Labman-ir
29 228 363
Pedestrian
29 228 363
Pedestrian-ir
14 94 138
Runner
14 94 138
Runner-ir
L1-PF JSR CSR
Figure 2 Tracking performances of the test sequences
Mathematical Problems in Engineering 7
Biker
0
40
80
1 41 81 121
Biker-ir
0
50
100
150
1 41 81 121
Campus
0
50
100
150
200
1 51 101 151 201
Campus-ir
0
100
200
300
1 51 101 151 201
Car
0
100
200
1 11 21 31
Car-ir
0
100
200
300
1 11 21 31
0
50
100
150
200
250
1 201 401 6010
100
200
1 201 401 601
Crossroad Crossroad-ir
0
50
100
150
200
250
300
1 201 401 601 801 1001
Hotkettle
0
50
100
150
200
250
1 201 401 601 801 1001
Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0
50
100
150
200
1 201 401 6010
50
100
150
1 201 401 601
Labman
0
100
200
300
1 101 201 301 401
Labman-ir
0
200
400
1 101 201 301 401
Pedestrian
0
50
100
150
1 101 201 301
Pedestrian-ir
0
50
100
150
1 101 201 301
Runner
0
100
200
1 61 121 181
Runner-ir
0
100
200
1 61 121 181
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
Figure 3 Location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
0
05
1
1 41 81 1210
05
1
1 41 81 1210
05
1
1 51 101 151 2010
05
1
1 51 101 151 2010
05
1
1 11 21 310
05
1
1 11 21 31
0
05
1
1 201 401 6010
05
1
1 201 401 6010
05
1
1 201 401 601 801 10010
05
1
1 201 401 601 801 10010
05
1
1 201 401 6010
05
1
1 201 401 601
0
05
1
1 101 201 301 4010
05
1
1 101 201 301 4010
05
1
1 101 201 3010
05
1
1 101 201 3010
05
1
1 61 121 1810
05
1
1 61 121 181
Figure 4 Overlapping rate of the test sequences
the sidewalk most of the trackers are not able to handlethe problem of heavy background clutters but our trackerperforms satisfying tracking results as shown in Figures 2ndash4
(e) Sequences Hotkettle and Hotkettle-ir in thesesequences tracking is hard because of the changes of thecomplex background clutters Most trackers perform betterin Hotkettle-ir than in Hotkettle for the reason that thetemperature diverge makes the hot target more distinct inthe cold background Struck KCF DSST fDSST and DFCL
can achieve robust and accurate tracking performances asshown in Figures 2ndash4
(f) Sequences Inglassandmobile and Inglassandmobile-irSequences Inglassandmobile and Inglassandmobile-ir demon-strate the performances of the 14 trackers under the cir-cumstances of background clutters illumination changesand occlusion As shown in Figure 2 when the illuminationchanges at around Frames 300 ODFS and fDSST lose thetarget and KCF TSKF and L1-PF drift a little away from the
8 Mathematical Problems in Engineering
10048
96602
171062
9467 27351 20776 17357
3462 21838 17229
2866
241288
9947 6518
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Biker Biker-ir Campus Campus-ir
4536
114901
6739 4323 3944
187420
3574 3805 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2758 9391
287670
2255 4284
216795
1778 2248 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
6301
165743 160172
122008 116442
151244
7767 7767 3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
59762
9866
138565
63421 63817
145486
11886 16255
3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Car Car-ir
122137
109606
122092 122500 122514
106350
121940 122072
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
25018 22604 24216
122500
55885
106350
121940 122530
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Labman Labman-ir
19128
263409
304244
34664 26765
390209
27024 18464 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
7160 16449
377858
7428 38194
497072
21473 12348 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Pedestrian Pedestrian-ir Runner Runner-ir
117051 116737 120324
142077 137828 144675
121509
144107
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
9457
55577
141371 142012
43182
435192
141348 141350
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
116065
92197
30636
129168
27984
100629
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2876
117247
2341
127301
3465
297251
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
9575
300997
60040
11480 15424
287956
19187 2840
21838 17229 2866
241288
9947 4246
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
3638
79438 54283
4987
55103
313867
3023 4995 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
5983 9604
28494
4086 5710
196150
2693 4255 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
6687
66141
48306
16131
63235
116459
14736
74415
18142 10318 10791 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
4160
16714 26378
16071
3345
130137
14534
60554
18142 10318 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
10791
Figure 5 Average location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0993
0311
0986 0993 0993
0108
0993 0993 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
1000 0973
0014
1000 1000
0108
1000 1000 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0907
0005 0000 0056 0070
0009
0986 0986 1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0437
1000
0023
0158 0186
0028
0935
0758
1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0059
0176
0088 0088 0088 0059
0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0824
0382
0941
0088
0471
0059 0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0847
0026 0026
0520
0714
0022
0677 0733
0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0991
0877
0004
0994
0011 0009
0708
0959 0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0068 0054 0078 0049 0054 0044
0083 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0605
0020 0054 0054
0502
0024 0054 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0101
0189
0481
0016
0598
0044
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0937
0005
0904
0019
0943
0033
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0823
0016
0186
0808
0585 0585 0528
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0823
0016
0302
0808 0819
0016
0472
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0585
0105
0331
0561
0318
0000
0585 0558 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0571
0222
0454
0585 0578
0006
0585 0585 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976
0035
0749
0884
0587
0118
0917
0245
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976 0948
0604
0887
1000
0089
0925
0580
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Figure 6 Success rate of the test sequences
targetWhen the target is approaching a tree the backgroundclutters makes most of trackers cause tracking failures thatcan be seen from Figure 2 Our DFCL can overcome thesechallenges and perform well in these sequences
(g) Sequences Labman and Labman-ir the experimentsin Sequences Labman and Labman-ir aim to evaluate the per-formances on tracking under appearance variation rotationscale variation and background clutter In Labman whenthe target man is walking into the laboratory ODFS STCand MOSSE lose the target When the man keeps shaking
and turning around his head at around Frame 400 KCFROT and DSST cause tracking failures Also most trackersachieve better tracking performances in Labman-ir as shownin Figure 2
(h) Sequences Pedestrian and Pedestrian-ir the target inPedestrian and Pedestrian-ir undergoes heavy backgroundclutters and occlusion As shown in Figure 2 other trackersresult in tracking failures in Pedestrian whereas our trackershows satisfying performances in terms of both accuracyand robustnessThe efficient infrared features extracted from
Mathematical Problems in Engineering 9
DFCLDCL MOSSE
Average location error (pixel) Success rate
011 001 006
059
000 012
002 004 004
052
019 017
063
021
073
039 053
037
100 100
082 082
057
099 100 099 099
18742 15124
10635
2078
31387
11646
39021
10063 14467
1853 6166 8940
1786
9545 4990 5658
2692 9228
340 315 1899 652 495 723 1013 138 284
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Figure 7 Average location error (pixel) and success rate of DCL DFCL and MOSSE on the test sequences
Pedestrian-ir ensure the tracking successes of Struck STCand DFCL as can be seen from Figures 2ndash4
(i) Sequences Runner and Runner-ir Runner and Runner-ir contain examples of heavy occlusion abrupt movementand scale variation The target running man is occludedby lampposts trees stone tablet and bushes many timesresulting in tracking failures ofmost trackers Also the abruptmovement and scale variation cause many trackers to driftaway the target in both Runner and Runner-ir as shown inFigure 2 Once again ourDFCL is able to overcome the aboveproblems and achieve good performances
Figures 5 and 6 are included here to demonstrate quan-titatively the performances on average location error (pixel)and success rate The success rate is defined as the numberof times success is achieved in the whole tracking processby considering one frame as a success if the overlapping rateexceeds 05 [33] A smaller average location error and a largersuccess rate indicate increased accuracy and robustnessFigures 5 and 6 show that DFCL performs satisfying most ofthe tracking sequences
To validate the effectiveness of the discriminative filterselection model of DFCL we compare the tracker DCL(the proposed DFCL without the fusion learning model)with DFCL and the original DCF tracker MOSSE on visiblesequencesThe performances shown in Figure 7 demonstratethe efficiency of the discriminative filter selection modelespecially in the sequences with background clutters ieSequences BikerHotkettle Inglassandmobile and Pedestrian
5 Conclusion
Discriminative correlation filter- (DCF-) based trackers havethe advantage of being computationally efficient and morerobust than most of the other state-of-the-art trackers inchallenging tracking tasks thereby making them especiallysuitable for a variety of real-time challenging applications
Howevermost of theDCF-based trackers suffer low accuracydue to the lack of diversity information extracted from asingle type of spectral image (visible spectrum) Fusion ofvisible and infrared sensors one of the typical multisensorcooperation provides complementarily useful features andconsistently helps recognize the target from the backgroundefficiently in visual tracking For the above reasons this paperproposes a discriminative fusion correlation learning modelto improve DCF-based tracking performance by combiningmultiple features from visible and infrared imaging sensorsThe proposed fusion learning filters are obtained via latefusion with early estimation in which the performances ofthe filters are weighted to improve the flexibility of fusionMoreover the proposed discriminative filter selection modelconsiders the surrounding background information in orderto increase the discriminability of the template filters soas to improve model learning Numerous real-world videosequences were used to test DFCL and other state-of-the-art algorithms and here we only selected representativevideos for presentation Experimental results demonstratedthat DFCL is highly accurate and robust
Data Availability
The data used to support the findings of this study weresupplied by China University of Mining and Technologyunder license and so cannot be made freely available
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
This work is supported by the Natural Science Foun-dation of Jiangsu Province (BK20180640 BK20150204)
10 Mathematical Problems in Engineering
Research Development Programme of Jiangsu Province(BE2015040) the State Key Research Development Program(2016YFC0801403) and the National Natural Science Foun-dation of China (51504214 51504255 51734009 61771417 and61873246)
References
[1] S A Wibowo H Lee E K Kim and S Kim ldquoVisual trackingbased on complementary learners with distractor handlingrdquoMathematical Problems in Engineering vol 2017 Article ID5295601 13 pages 2017
[2] T Zhou M Zhu D Zeng and H Yang ldquoScale adaptive kernel-ized correlation filter tracker with feature fusionrdquoMathematicalProblems in Engineering vol 2017 Article ID 1605959 8 pages2017
[3] C Wang H K Galoogahi C Lin and S Lucey ldquoDeep-LK forefficient adaptive object trackingrdquo Computer Vision and PatternRecognition 2017
[4] Z Chi H Li H Lu and M-H Yang ldquoDual deep network forvisual trackingrdquo IEEE Transactions on Image Processing vol 26no 4 pp 2005ndash2015 2017
[5] Y Yang W Hu Y Xie W Zhang and T Zhang ldquoTemporalrestricted visual tracking via reverse-low-rank sparse learningrdquoIEEE Transactions on Cybernetics vol 47 no 2 pp 485ndash4982017
[6] Y Sui G Wang and L Zhang ldquoCorrelation filter learningtoward peak strength for visual trackingrdquo IEEE Transactions onCybernetics vol 48 no 4 pp 1290ndash1303 2018
[7] K Zhang X Li H Song Q Liu and W Lian ldquoVisual track-ing using spatio-temporally nonlocally regularized correlationfilterrdquo Pattern Recognition vol 83 pp 185ndash195 2018
[8] Y Qi L Qin J Zhang S Zhang Q Huang and M-H YangldquoStructure-aware local sparse coding for visual trackingrdquo IEEETransactions on Image Processing vol 27 no 8 pp 3857ndash38692018
[9] B Bai B Zhong G Ouyang et al ldquoKernel correlation filtersfor visual tracking with adaptive fusion of heterogeneous cuesrdquoNeurocomputing vol 286 pp 109ndash120 2018
[10] Y Liu F Yang C Zhong Y Tao B Dai and M Yin ldquoVisualtracking via salient feature extraction and sparse collaborativemodelrdquo AEU - International Journal of Electronics and Commu-nications vol 87 pp 134ndash143 2018
[11] X Yun Y Sun S Wang Y Shi and N Lu ldquoMulti-layerconvolutional network-based visual tracking via importantregion selectionrdquo Neurocomputing vol 315 pp 145ndash156 2018
[12] X Lan S Zhang and P C Yuen ldquoRobust joint discriminativefeature learning for visual trackingrdquo in Proceedings of the 25thInternational Joint Conference on Artificial Intelligence IJCAI2016 pp 3403ndash3410 AAAI Press New York NY USA July2016
[13] D S Bolme J R Beveridge B A Draper and Y M LuildquoVisual object tracking using adaptive correlation filtersrdquo inProceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo10) pp 2544ndash2550 IEEE SanFrancisco Calif USA June 2010
[14] M Danelljan G Hager F Shahbaz Khan and M FelsbergldquoAccurate scale estimation for robust visual trackingrdquo in Pro-ceedings of the British Machine Vision Conference pp 651-6511BMVA Press Nottingham UK 2014
[15] M Danelljan G Hager F S Khan and M Felsberg ldquoDis-criminative scale space trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 39 no 8 pp 1561ndash15752017
[16] H K Galoogahi A Fagg and S Lucey ldquoLearning background-aware correlation filters for visual trackingrdquo inProceedings of the16th IEEE International Conference on Computer Vision ICCV2017 pp 1144ndash1152 IEEE Istanbul Turkey 2018
[17] Y SongCMa LGong J Zhang RWH Lau andM-HYangldquoCrest convolutional residual learning for visual trackingrdquoin Proceedings of the 16th IEEE International Conference onComputer Vision ICCV 2017 pp 2574ndash2583 IEEEVenice ItalyOctober 2017
[18] J Johnander M Danelljan F Khan andM Felsberg ldquoDCCOtowards deformable continuous convolution operators forvisual trackingrdquo in Proceedings of the International Conferenceon Computer Analysis of Images and Patterns pp 55ndash67Springer Ystad Sweden 2017
[19] Y Wu E Blasch G Chen L Bai and H Ling ldquoMultiplesource data fusion via sparse representation for robust visualtrackingrdquo in Proceedings of the 14th International Conference onInformation Fusion Fusion 2011 IEEE Chicago IL USA July2011
[20] B Uzkent A Rangnekar and M J Hoffman ldquoAerial vehicletracking by adaptive fusion of hyperspectral likelihood mapsrdquoin Proceedings of the 30th IEEE Conference on Computer Visionand Pattern RecognitionWorkshops CVPRW 2017 pp 233ndash242IEEE Honolulu HI USA July 2017
[21] S Chan X Zhou and S Chen ldquoRobust adaptive fusion trackingbased on complex cells and keypointsrdquo IEEE Access vol 5 pp20985ndash21001 2017
[22] M K Rapuru S Kakanuru P M Venugopal D Mishra and GR Subrahmanyam ldquoCorrelation-based tracker-level fusion forrobust visual trackingrdquo IEEE Transactions on Image Processingvol 26 no 10 pp 4832ndash4842 2017
[23] X Yun Z Jing G Xiao B Jin and C Zhang ldquoA compressivetracking based on time-space Kalman fusion modelrdquo ScienceChina Information Sciences vol 59 no 1 pp 1ndash15 2016
[24] H P Liu and F C Sun ldquoFusion tracking in color and infraredimages using joint sparse representationrdquo Science China Infor-mation Sciences vol 55 no 3 pp 590ndash599 2012
[25] C Li H Cheng S Hu X Liu J Tang and L Lin ldquoLearningcollaborative sparse representation for grayscale-thermal track-ingrdquo IEEE Transactions on Image Processing vol 25 no 12 pp5743ndash5756 2016
[26] S Mangale and M Khambete ldquoCamouflaged target detectionand tracking using thermal infrared and visible spectrum imag-ingrdquo in Automatic Diagnosis of Breast Cancer using Thermo-graphic Color Analysis and SVM Classifier vol 530 of Advancesin Intelligent Systems and Computing pp 193ndash207 SpringerInternational Publishing 2016
[27] X Yun Z Jing and B Jin ldquoVisible and infrared tracking basedon multi-view multi-kernel fusion modelrdquo Optical Review vol23 no 2 pp 244ndash253 2016
[28] L Zhang A Gonzalez-Garcia J van de Weijer M Danelljanand F S Khan ldquoSynthetic data generation for end-to-end ther-mal infrared trackingrdquo IEEE Transactions on Image Processingvol 28 no 4 pp 1837ndash1850 2019
[29] X Lan M Ye S Zhang and P C Yuen ldquoRobust collaborativediscriminative learning for rgb-infrared trackingrdquo in Proceed-ings of the 32th AAAI Conference on Artificial Intelligence pp7008ndash7015 AAAI New Orleans LA USA 2018
Mathematical Problems in Engineering 11
[30] C Li X Wu N Zhao X Cao and J Tang ldquoFusing two-streamconvolutional neural networks for RGB-T object trackingrdquoNeurocomputing vol 281 pp 78ndash85 2018
[31] JWagner V FischerMHerman and S Behnke ldquoMultispectralpedestrian detection using deep fusion convolutional neuralnetworksrdquo in Proceedings of the 24th European Symposium onArtificial Neural Networks pp 1ndash6 Bruges Belgium 2016
[32] S Hare A Saffari and P H S Torr ldquoStruck structured outputtracking with kernelsrdquo in Proceedings of the IEEE InternationalConference on Computer Vision pp 263ndash270 IEEE ProvidenceRI USA 2012
[33] K Zhang L Zhang andM-H Yang ldquoReal-time object trackingvia online discriminative feature selectionrdquo IEEE Transactionson Image Processing vol 22 no 12 pp 4664ndash4677 2013
[34] K Zhang L Zhang M Yang and D Zhang ldquoFast trackingvia spatio- temporal context learningrdquo Computer Vision andPattern Recognition 2013
[35] J F Henriques R Caseiro P Martins and J Batista ldquoHigh-speed tracking with kernelized correlation filtersrdquo IEEE Trans-actions on Pattern Analysis and Machine Intelligence vol 37 no3 pp 583ndash596 2015
[36] X Dong J Shen D Yu W Wang J Liu and H HuangldquoOcclusion-aware real-time object trackingrdquo IEEE Transactionson Multimedia vol 19 no 4 pp 763ndash771 2017
[37] C O Conaire N E OrsquoConnor and A Smeaton ldquoThermo-visual feature fusion for object tracking using multiple spa-tiogram trackersrdquo Machine Vision amp Applications vol 19 no5-6 pp 483ndash494 2008
[38] INO ldquoInorsquos video analytics datasetrdquo httpswwwinocaenvideo-analytics-dataset
[39] C Li X Liang Y Lu Z Nan and T Jin ldquoRGB-T objecttracking benchmark and baselinerdquo IEEE Transactions on ImageProcessing 2018
[40] C O Conaire N E OrsquoConnor and A F Smeaton ldquoDetectoradaptation by maximising agreement between independentdata sourcesrdquo in Proceedings of the 2007 IEEE ComputerSociety Conference on Computer Vision and Pattern RecognitionCVPRrsquo07 pp 1ndash6 IEEE Minneapolis MN USA June 2007
[41] J W Davis and V Sharma ldquoBackground-subtraction usingcontour-based fusion of thermal and visible imageryrdquoComputerVision and Image Understanding vol 106 no 2-3 pp 162ndash1822007
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom
4 Mathematical Problems in Engineering
31 Discriminative Filter Selection According to DCF-basedtrackers we obtain the correlation output 119892 by
119892 =119889
sum119897=1
ℎ119897119879 lowast 119904119897119879 119897 = 1 sdot sdot sdot 119889 (5)
where 119904119897119879 and ℎ119897119879 are the target sample and target correlationfilter corresponding to the 119897-th channel feature among 119889channels respectively In this paper 119892 is selected as a 2DGaussian function where 120590 = 20 [13]
Before tracking we need to choose the optimal targetcorrelation filters in the training step via minimizing (6) as
119867119897119879 =119866119878119897119879
sum119889119896=1 119878119896
119879119878119896119879 + 120582 119897 = 1 sdot sdot sdot 119889 (6)
where 119878119897119879 119867119897119879 and 119866 are the DFTs of 119904119897119879 ℎ119897119879 and 119892respectively
Different from a single training sample of the targetappearance multiple background samples at different loca-tions around target need to be considered to maintain a sta-ble model However extracting multichannel features fromeach background sample increase computational complexsignificantly Moreover in practice single channel featuresfrom multiple background samples are enough to presentsatisfied performances Therefore in this paper we extract119872 background samples randomly in the range of an annulusaround the target location [11] and obtain the correlationoutput 119892 as
119892 =119872
sum119898=1
ℎ119861 lowast 119904119898119861 119898 = 1 sdot sdot sdot 119872 (7)
where 119904119898119861 119898 = 1 sdot sdot sdot 119872 denotes the119898-th background sam-ple
Similarly the optimal background correlation filters inthe training step are selected via minimizing (8) as
119867119898119861 = 119866119878119898119861sum119872119895=1 119878
119895
119861119878119895119861 + 120582 119898 = 1 sdot sdot sdot 119872 (8)
where 119878119898119861 and119867119898119861 are the DFTs of 119904119898119861 and ℎ119898119861 respectivelyWhile tracking DFT 119884119905119890119904 of the estimated discriminative
correlation score 119910119905119890119904 of the test sample 119911119905 is defined as
119884119905119890119904 =119884119905119890119904119879119884119905119890119904119861
=(sum119889119897=1 119860
119897
119905minus1119879119885119897119905) (119861119905minus1119879 + 120582)(sum119872119898=1 119860
119898
119905minus1119861119885119905) (119861119905minus1119861 + 120582)
=(sum119889119897=1 119860
119897
119905minus1119879119885119897119905) 119861119905minus1119861(119885119905sum119872119898=1 119860
119898
119905minus1119861) (119861119905minus1119879 + 120582)
(9)
where 119884119905119890119904119879 and 119884119905119890119904119861 are the DFTs of the target andbackground correlation scores 119910119905119890119904119879 and 119910119905119890119904119861 respectivelyand 119885119897119905 is the DFTs of 119911119897119905 119860119897119905minus1119879 and 119861119905minus1119879 denote thenumerator and denominator of the filter 119867119897119879 in (6) 119860119898119905minus1119861
and119861119905minus1119861 denote the numerator and denominator of the filter119867119898119861 in (8)Then the discriminative correlation scores 119910119905119890119904 areobtained by taking the inverse DFT 119910119905119890119904 = Fminus1119884119905119890119904 Theestimate of the current target state 1199111199050 119890119904 is obtained by findingthe maximum correlation score among the test samples as1199111199050 119890119904 997888rarr 1199050 1199050 = arg max119905119910119905119890119904
32 Fusion Learning Filter As proved by Wagner et al [31]late fusion with early estimation provides better performancethan early fusion with late estimation Based on this conclu-sion we use the discriminative correlation filters to obtainthe estimate of target location in visible and infrared imagesrespectively and then do fusion with the fusion correlationfilters Let 119911119905119890119904119894 119894 isin V119894 119894119903 denote the estimate of targetlocation of visible V119894 or infrared 119894119903 images Then we definethe region119877∘(sdot ∙) denoting theminimumbounding rectanglethat contains the regions of samples sdot and ∙ in image ∘Thus we extract the fusion test samples through 119911119905119894 isin119877119894(119911119905119890119904V119894 119911119905119890119904119894119903) 119894 isin V119894 119894119903 and define the DFT 119884119905119891119906 ofthe fusion correlation score 119910119905119891119906 of fusion sample 119911119905119894 that isdefined as
119884119905119891119906 =119884119905119879119891119906119884119905119861119891119906
= sum119894isinV119894119894119903 120572119894119884119905119890119904119879119894sum119894isinV119894119894119903 120572119894119884119905119890119904119861119894
=sum119894isinV119894119894119903 120572119894 ((sum119889119897=1 119860
119897
119905minus1119879119894119885119897119905119894) (119861119905minus1119879119894 + 120582))sum119894isinV119894119894119903 120572119894 ((sum119872119898=1 119860
119898
119905minus1119861119894119885119905119894) (119861119905minus1119861119894 + 120582))
= sum119894isinV119894119894119903
120572119894 (sum119889119897=1 119860119897
119905minus1119879119894119885119897119905119894) 119861119905minus1119861119894(119885119905119894sum119872119898=1 119860
119898
119905minus1119861119894) (119861119905minus1119879119894 + 120582)
(10)
where 119884119905119891119906119879 and 119884119905119891119906119861 are the DFTs of the target and back-ground fusion correlation scores 119910119905119891119906119879 and 119910119905119891119906119861 respec-tively 119911119897119905119894 isin 119877119894119897(119911119905119890119904V119894 119911119905119890119904119894119903) 119894 isin V119894 119894119903 in which 119894119897 is the 119897-thchannel feature of image 119894 119885119897119905119894 and 119885119905119894 are the DFTs of sam-ples 119911119897119905119894 and 119911119905119894 respectively120572119894 denotes the imageweights thatare denoted as
120572i =max119894119910119905119890119904119894
sum119894isinV119894119894119903max119894119910119905119890119904119894 (11)
where 119910119905119890119904119894 is the correlation score of the 119894-th image com-puted by (9)
After obtaining 119884119905119891119906 the fusion correlation score 119910119905119891119906is obtained by taking the inverse DFT 119910119905119891119906 = Fminus1119884119905119891119906The fusion location of the current target state is obtainedby finding the maximum correlation score among the testsamples
The whole tracking process of DFCL is summarized inAlgorithm 1
4 Experiments
The proposed DFCL algorithm was tested on several chal-lenging real-world sequences and some qualitative andquantitative analyses were performed on the tracking resultsin this section
Mathematical Problems in Engineering 5
InputThe 119905-th visible and infrared imagesFor 119905 = 1 to number of frames do
1 Crop the samples and extract the 119897-th (119897 = 1 sdot sdot sdot 119889) channel features 119911119897119905 for visible andinfrared images respectively
2 Compute the discriminative correlation scores 119910119905119890119904 using Eq (9)3 Compute the fusion correlation scores 119910119905119891119906 using Eq (10)4 Obtain the tracking result by maximizing 1199101199051198911199065 Extract the 119897-th (119897 = 1 sdot sdot sdot 119889) channel feature 119904119897119879 of the target samples and the119898-th
(119898 = 1 sdot sdot sdot119872) sample 1199041198971198616 Update the discriminative correlation filters119867119897119879 and119867119898119861 using Eq (6) and Eq (8)
respectivelyend forOutput Target result and the discriminative correlation filters119867119897119879 and119867119898119861
Algorithm 1 Discriminative fusion correlation learning for visible and infrared tracking
41 Experimental Environment and Evaluation CriteriaDFCL was implemented with C++ programming languageandNet Framework 40 in Visual Studio 2010 on an IntelDual-Core 170GHz CPU with 4 GB RAM Two metricsie location error (pixel) and overlapping rate are usedto evaluate the tracking results quantitatively The locationerror is computed as 119890119903119903119900119903 = radic(119909119866 minus 119909119879)2 + (119910119866 minus 119910119879)2where (119909119866 119910119866) and (119909119879 119910119879) are the ground truth (eitherdownloaded from a standard database or located manually)and tracking bounding box centers respectivelyThe trackingoverlapping rate is defined as 119900V119890119903119897119886119901119901119894119899119892 = 119886119903119890119886(119877119874119868119866 cap119877119874119868119879)119886119903119890119886(119877119874119868119866cup119877119874119868119879) where119877119874119868119866 and119877119874119868119879 denote theground truth and tracking bounding box respectively and119886119903119890119886(sdot) is the rectangular area function A smaller locationerror and a larger overlapping rate indicate higher accuracyand robustness
42 Experimental Results The performance of DFCL wascompared with state-of-the-art trackers Struck [32] ODFS[33] STC [34] KCF [35] ROT [36] DCF-based trackersMOSSE [13] DSST [14] fDSST [15] and visible-infraredfusion trackers TSKF [23] MVMKF [27] L1-PF [19] JSR[24] and CSR [25] Figures 2ndash6 present the experimentalresults of the test trackers in challenging visible sequencesnamed Biker [37] Campus [37] Car [38] Crossroad [39]Hotkettle [39] Inglassandmobile [39] Labman [40] Pedes-trian [41] and Runner [38] as well as their correspondinginfrared sequences Biker-ir Campus-ir Car-ir Crossroad-ir Labman-ir Hotkettle-ir Inglassandmobile-ir Pedestrian-ir and Runner-ir Single-sensor trackers were separatelytested on visible and the corresponding infrared sequenceswhile visible-infrared fusion trackers obtain the results withinformation fromboth visible and infrared sequences For theconvenience of presentation some tracking curves are notshown entirely in the Figures Next the performance of thetrackers in each sequence is described in detail
(a) SequencesBiker andBiker-irBiker presents the exam-ple of complex background clutters The target human in thevisible sequence encounters similar background disturbance(ie bikes) which causes the ODFS MOSSE fDSST TSKF
and MVMKF trackers to drift away from the target Thecorresponding infrared sequence Biker-ir provides temper-ature information that eliminates the background clutter inBiker But when the target is approaching another person ataround Frame 20 Struck ODFS STC MOSSE TSKF andMVMKF do not perform well because they are not able todistinguish target from persons with similar temperature ininfrared sequences Only KCF ROT DSST and our DFCLhave achieved precise and robust performances in thesesequences
(b) Sequences Campus and Campus-ir the target inCampus and Campus-ir undergoes background cluttersocclusion and scale variation At the beginning of CampusODFS STC KCF and ROT lose the target due to backgrounddisturbance Only TSKF and DFCL perform well whileStruck fDSST and MVMKF do not achieve accurate resultsBecause of the infrared information provided by Campus-irfewer test trackers lose tracking when background cluttershappen as shown in Figure 2 But Struck KCF and ROTmistake another person for the target As shown in Figure 2most of the trackers result in tracking failures whereas DFCLoutperforms the others in most metrics (location accuracyand success rate)
(c) SequencesCar andCar-irCar andCar-ir demonstratethe efficiency of DFCL on coping with heavy occlusions Thetarget driving car is occluded by lampposts and trees manytimes which cause tracking failures of most trackers OnlyTSKF MVMKF and DFCL are able to handle the occlusionthroughout the tracking process in this sequence As shownin Figure 2most trackers performbetter inCar-ir than inCarbecause the infrared features can overcome the difficultiesof target detection among surrounding similar backgroundSTC TSKF MVMKF and DFCL are able to handle thisproblem whereas the result of DFCL is the most accurate asshown in Figure 2
(d) Sequences Crossroad and Crossroad-ir the target inCrossroad and Crossroad-ir undergoes heavy backgroundclutters when she crosses the roadWhile the target is passingby the road lamp both ODFS and JSR lose the target Thenwhen a car passes by the target Struck TSKF and MVMKFdrift away from the target When the target goes toward
6 Mathematical Problems in Engineering
20 53 147
Biker
20 53 147
Biker-ir
8 84 156
Campus
8 84 156
Campus-ir
14 23 27
Car
14 23 27
Car-ir
136 289 767
Crossroad
136 289 767
Crossroad-ir
133 451 1068
Hotkettle
133 451 1068
Hotkettle-ir
309 428 511
Inglassandmobile
309 428 511
Inglassandmobile-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF
302 320 446
Labman
302 320 446
Labman-ir
29 228 363
Pedestrian
29 228 363
Pedestrian-ir
14 94 138
Runner
14 94 138
Runner-ir
L1-PF JSR CSR
Figure 2 Tracking performances of the test sequences
Mathematical Problems in Engineering 7
Biker
0
40
80
1 41 81 121
Biker-ir
0
50
100
150
1 41 81 121
Campus
0
50
100
150
200
1 51 101 151 201
Campus-ir
0
100
200
300
1 51 101 151 201
Car
0
100
200
1 11 21 31
Car-ir
0
100
200
300
1 11 21 31
0
50
100
150
200
250
1 201 401 6010
100
200
1 201 401 601
Crossroad Crossroad-ir
0
50
100
150
200
250
300
1 201 401 601 801 1001
Hotkettle
0
50
100
150
200
250
1 201 401 601 801 1001
Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0
50
100
150
200
1 201 401 6010
50
100
150
1 201 401 601
Labman
0
100
200
300
1 101 201 301 401
Labman-ir
0
200
400
1 101 201 301 401
Pedestrian
0
50
100
150
1 101 201 301
Pedestrian-ir
0
50
100
150
1 101 201 301
Runner
0
100
200
1 61 121 181
Runner-ir
0
100
200
1 61 121 181
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
Figure 3 Location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
0
05
1
1 41 81 1210
05
1
1 41 81 1210
05
1
1 51 101 151 2010
05
1
1 51 101 151 2010
05
1
1 11 21 310
05
1
1 11 21 31
0
05
1
1 201 401 6010
05
1
1 201 401 6010
05
1
1 201 401 601 801 10010
05
1
1 201 401 601 801 10010
05
1
1 201 401 6010
05
1
1 201 401 601
0
05
1
1 101 201 301 4010
05
1
1 101 201 301 4010
05
1
1 101 201 3010
05
1
1 101 201 3010
05
1
1 61 121 1810
05
1
1 61 121 181
Figure 4 Overlapping rate of the test sequences
the sidewalk most of the trackers are not able to handlethe problem of heavy background clutters but our trackerperforms satisfying tracking results as shown in Figures 2ndash4
(e) Sequences Hotkettle and Hotkettle-ir in thesesequences tracking is hard because of the changes of thecomplex background clutters Most trackers perform betterin Hotkettle-ir than in Hotkettle for the reason that thetemperature diverge makes the hot target more distinct inthe cold background Struck KCF DSST fDSST and DFCL
can achieve robust and accurate tracking performances asshown in Figures 2ndash4
(f) Sequences Inglassandmobile and Inglassandmobile-irSequences Inglassandmobile and Inglassandmobile-ir demon-strate the performances of the 14 trackers under the cir-cumstances of background clutters illumination changesand occlusion As shown in Figure 2 when the illuminationchanges at around Frames 300 ODFS and fDSST lose thetarget and KCF TSKF and L1-PF drift a little away from the
8 Mathematical Problems in Engineering
10048
96602
171062
9467 27351 20776 17357
3462 21838 17229
2866
241288
9947 6518
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Biker Biker-ir Campus Campus-ir
4536
114901
6739 4323 3944
187420
3574 3805 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2758 9391
287670
2255 4284
216795
1778 2248 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
6301
165743 160172
122008 116442
151244
7767 7767 3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
59762
9866
138565
63421 63817
145486
11886 16255
3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Car Car-ir
122137
109606
122092 122500 122514
106350
121940 122072
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
25018 22604 24216
122500
55885
106350
121940 122530
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Labman Labman-ir
19128
263409
304244
34664 26765
390209
27024 18464 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
7160 16449
377858
7428 38194
497072
21473 12348 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Pedestrian Pedestrian-ir Runner Runner-ir
117051 116737 120324
142077 137828 144675
121509
144107
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
9457
55577
141371 142012
43182
435192
141348 141350
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
116065
92197
30636
129168
27984
100629
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2876
117247
2341
127301
3465
297251
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
9575
300997
60040
11480 15424
287956
19187 2840
21838 17229 2866
241288
9947 4246
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
3638
79438 54283
4987
55103
313867
3023 4995 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
5983 9604
28494
4086 5710
196150
2693 4255 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
6687
66141
48306
16131
63235
116459
14736
74415
18142 10318 10791 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
4160
16714 26378
16071
3345
130137
14534
60554
18142 10318 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
10791
Figure 5 Average location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0993
0311
0986 0993 0993
0108
0993 0993 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
1000 0973
0014
1000 1000
0108
1000 1000 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0907
0005 0000 0056 0070
0009
0986 0986 1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0437
1000
0023
0158 0186
0028
0935
0758
1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0059
0176
0088 0088 0088 0059
0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0824
0382
0941
0088
0471
0059 0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0847
0026 0026
0520
0714
0022
0677 0733
0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0991
0877
0004
0994
0011 0009
0708
0959 0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0068 0054 0078 0049 0054 0044
0083 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0605
0020 0054 0054
0502
0024 0054 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0101
0189
0481
0016
0598
0044
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0937
0005
0904
0019
0943
0033
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0823
0016
0186
0808
0585 0585 0528
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0823
0016
0302
0808 0819
0016
0472
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0585
0105
0331
0561
0318
0000
0585 0558 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0571
0222
0454
0585 0578
0006
0585 0585 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976
0035
0749
0884
0587
0118
0917
0245
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976 0948
0604
0887
1000
0089
0925
0580
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Figure 6 Success rate of the test sequences
targetWhen the target is approaching a tree the backgroundclutters makes most of trackers cause tracking failures thatcan be seen from Figure 2 Our DFCL can overcome thesechallenges and perform well in these sequences
(g) Sequences Labman and Labman-ir the experimentsin Sequences Labman and Labman-ir aim to evaluate the per-formances on tracking under appearance variation rotationscale variation and background clutter In Labman whenthe target man is walking into the laboratory ODFS STCand MOSSE lose the target When the man keeps shaking
and turning around his head at around Frame 400 KCFROT and DSST cause tracking failures Also most trackersachieve better tracking performances in Labman-ir as shownin Figure 2
(h) Sequences Pedestrian and Pedestrian-ir the target inPedestrian and Pedestrian-ir undergoes heavy backgroundclutters and occlusion As shown in Figure 2 other trackersresult in tracking failures in Pedestrian whereas our trackershows satisfying performances in terms of both accuracyand robustnessThe efficient infrared features extracted from
Mathematical Problems in Engineering 9
DFCLDCL MOSSE
Average location error (pixel) Success rate
011 001 006
059
000 012
002 004 004
052
019 017
063
021
073
039 053
037
100 100
082 082
057
099 100 099 099
18742 15124
10635
2078
31387
11646
39021
10063 14467
1853 6166 8940
1786
9545 4990 5658
2692 9228
340 315 1899 652 495 723 1013 138 284
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Figure 7 Average location error (pixel) and success rate of DCL DFCL and MOSSE on the test sequences
Pedestrian-ir ensure the tracking successes of Struck STCand DFCL as can be seen from Figures 2ndash4
(i) Sequences Runner and Runner-ir Runner and Runner-ir contain examples of heavy occlusion abrupt movementand scale variation The target running man is occludedby lampposts trees stone tablet and bushes many timesresulting in tracking failures ofmost trackers Also the abruptmovement and scale variation cause many trackers to driftaway the target in both Runner and Runner-ir as shown inFigure 2 Once again ourDFCL is able to overcome the aboveproblems and achieve good performances
Figures 5 and 6 are included here to demonstrate quan-titatively the performances on average location error (pixel)and success rate The success rate is defined as the numberof times success is achieved in the whole tracking processby considering one frame as a success if the overlapping rateexceeds 05 [33] A smaller average location error and a largersuccess rate indicate increased accuracy and robustnessFigures 5 and 6 show that DFCL performs satisfying most ofthe tracking sequences
To validate the effectiveness of the discriminative filterselection model of DFCL we compare the tracker DCL(the proposed DFCL without the fusion learning model)with DFCL and the original DCF tracker MOSSE on visiblesequencesThe performances shown in Figure 7 demonstratethe efficiency of the discriminative filter selection modelespecially in the sequences with background clutters ieSequences BikerHotkettle Inglassandmobile and Pedestrian
5 Conclusion
Discriminative correlation filter- (DCF-) based trackers havethe advantage of being computationally efficient and morerobust than most of the other state-of-the-art trackers inchallenging tracking tasks thereby making them especiallysuitable for a variety of real-time challenging applications
Howevermost of theDCF-based trackers suffer low accuracydue to the lack of diversity information extracted from asingle type of spectral image (visible spectrum) Fusion ofvisible and infrared sensors one of the typical multisensorcooperation provides complementarily useful features andconsistently helps recognize the target from the backgroundefficiently in visual tracking For the above reasons this paperproposes a discriminative fusion correlation learning modelto improve DCF-based tracking performance by combiningmultiple features from visible and infrared imaging sensorsThe proposed fusion learning filters are obtained via latefusion with early estimation in which the performances ofthe filters are weighted to improve the flexibility of fusionMoreover the proposed discriminative filter selection modelconsiders the surrounding background information in orderto increase the discriminability of the template filters soas to improve model learning Numerous real-world videosequences were used to test DFCL and other state-of-the-art algorithms and here we only selected representativevideos for presentation Experimental results demonstratedthat DFCL is highly accurate and robust
Data Availability
The data used to support the findings of this study weresupplied by China University of Mining and Technologyunder license and so cannot be made freely available
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
This work is supported by the Natural Science Foun-dation of Jiangsu Province (BK20180640 BK20150204)
10 Mathematical Problems in Engineering
Research Development Programme of Jiangsu Province(BE2015040) the State Key Research Development Program(2016YFC0801403) and the National Natural Science Foun-dation of China (51504214 51504255 51734009 61771417 and61873246)
References
[1] S A Wibowo H Lee E K Kim and S Kim ldquoVisual trackingbased on complementary learners with distractor handlingrdquoMathematical Problems in Engineering vol 2017 Article ID5295601 13 pages 2017
[2] T Zhou M Zhu D Zeng and H Yang ldquoScale adaptive kernel-ized correlation filter tracker with feature fusionrdquoMathematicalProblems in Engineering vol 2017 Article ID 1605959 8 pages2017
[3] C Wang H K Galoogahi C Lin and S Lucey ldquoDeep-LK forefficient adaptive object trackingrdquo Computer Vision and PatternRecognition 2017
[4] Z Chi H Li H Lu and M-H Yang ldquoDual deep network forvisual trackingrdquo IEEE Transactions on Image Processing vol 26no 4 pp 2005ndash2015 2017
[5] Y Yang W Hu Y Xie W Zhang and T Zhang ldquoTemporalrestricted visual tracking via reverse-low-rank sparse learningrdquoIEEE Transactions on Cybernetics vol 47 no 2 pp 485ndash4982017
[6] Y Sui G Wang and L Zhang ldquoCorrelation filter learningtoward peak strength for visual trackingrdquo IEEE Transactions onCybernetics vol 48 no 4 pp 1290ndash1303 2018
[7] K Zhang X Li H Song Q Liu and W Lian ldquoVisual track-ing using spatio-temporally nonlocally regularized correlationfilterrdquo Pattern Recognition vol 83 pp 185ndash195 2018
[8] Y Qi L Qin J Zhang S Zhang Q Huang and M-H YangldquoStructure-aware local sparse coding for visual trackingrdquo IEEETransactions on Image Processing vol 27 no 8 pp 3857ndash38692018
[9] B Bai B Zhong G Ouyang et al ldquoKernel correlation filtersfor visual tracking with adaptive fusion of heterogeneous cuesrdquoNeurocomputing vol 286 pp 109ndash120 2018
[10] Y Liu F Yang C Zhong Y Tao B Dai and M Yin ldquoVisualtracking via salient feature extraction and sparse collaborativemodelrdquo AEU - International Journal of Electronics and Commu-nications vol 87 pp 134ndash143 2018
[11] X Yun Y Sun S Wang Y Shi and N Lu ldquoMulti-layerconvolutional network-based visual tracking via importantregion selectionrdquo Neurocomputing vol 315 pp 145ndash156 2018
[12] X Lan S Zhang and P C Yuen ldquoRobust joint discriminativefeature learning for visual trackingrdquo in Proceedings of the 25thInternational Joint Conference on Artificial Intelligence IJCAI2016 pp 3403ndash3410 AAAI Press New York NY USA July2016
[13] D S Bolme J R Beveridge B A Draper and Y M LuildquoVisual object tracking using adaptive correlation filtersrdquo inProceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo10) pp 2544ndash2550 IEEE SanFrancisco Calif USA June 2010
[14] M Danelljan G Hager F Shahbaz Khan and M FelsbergldquoAccurate scale estimation for robust visual trackingrdquo in Pro-ceedings of the British Machine Vision Conference pp 651-6511BMVA Press Nottingham UK 2014
[15] M Danelljan G Hager F S Khan and M Felsberg ldquoDis-criminative scale space trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 39 no 8 pp 1561ndash15752017
[16] H K Galoogahi A Fagg and S Lucey ldquoLearning background-aware correlation filters for visual trackingrdquo inProceedings of the16th IEEE International Conference on Computer Vision ICCV2017 pp 1144ndash1152 IEEE Istanbul Turkey 2018
[17] Y SongCMa LGong J Zhang RWH Lau andM-HYangldquoCrest convolutional residual learning for visual trackingrdquoin Proceedings of the 16th IEEE International Conference onComputer Vision ICCV 2017 pp 2574ndash2583 IEEEVenice ItalyOctober 2017
[18] J Johnander M Danelljan F Khan andM Felsberg ldquoDCCOtowards deformable continuous convolution operators forvisual trackingrdquo in Proceedings of the International Conferenceon Computer Analysis of Images and Patterns pp 55ndash67Springer Ystad Sweden 2017
[19] Y Wu E Blasch G Chen L Bai and H Ling ldquoMultiplesource data fusion via sparse representation for robust visualtrackingrdquo in Proceedings of the 14th International Conference onInformation Fusion Fusion 2011 IEEE Chicago IL USA July2011
[20] B Uzkent A Rangnekar and M J Hoffman ldquoAerial vehicletracking by adaptive fusion of hyperspectral likelihood mapsrdquoin Proceedings of the 30th IEEE Conference on Computer Visionand Pattern RecognitionWorkshops CVPRW 2017 pp 233ndash242IEEE Honolulu HI USA July 2017
[21] S Chan X Zhou and S Chen ldquoRobust adaptive fusion trackingbased on complex cells and keypointsrdquo IEEE Access vol 5 pp20985ndash21001 2017
[22] M K Rapuru S Kakanuru P M Venugopal D Mishra and GR Subrahmanyam ldquoCorrelation-based tracker-level fusion forrobust visual trackingrdquo IEEE Transactions on Image Processingvol 26 no 10 pp 4832ndash4842 2017
[23] X Yun Z Jing G Xiao B Jin and C Zhang ldquoA compressivetracking based on time-space Kalman fusion modelrdquo ScienceChina Information Sciences vol 59 no 1 pp 1ndash15 2016
[24] H P Liu and F C Sun ldquoFusion tracking in color and infraredimages using joint sparse representationrdquo Science China Infor-mation Sciences vol 55 no 3 pp 590ndash599 2012
[25] C Li H Cheng S Hu X Liu J Tang and L Lin ldquoLearningcollaborative sparse representation for grayscale-thermal track-ingrdquo IEEE Transactions on Image Processing vol 25 no 12 pp5743ndash5756 2016
[26] S Mangale and M Khambete ldquoCamouflaged target detectionand tracking using thermal infrared and visible spectrum imag-ingrdquo in Automatic Diagnosis of Breast Cancer using Thermo-graphic Color Analysis and SVM Classifier vol 530 of Advancesin Intelligent Systems and Computing pp 193ndash207 SpringerInternational Publishing 2016
[27] X Yun Z Jing and B Jin ldquoVisible and infrared tracking basedon multi-view multi-kernel fusion modelrdquo Optical Review vol23 no 2 pp 244ndash253 2016
[28] L Zhang A Gonzalez-Garcia J van de Weijer M Danelljanand F S Khan ldquoSynthetic data generation for end-to-end ther-mal infrared trackingrdquo IEEE Transactions on Image Processingvol 28 no 4 pp 1837ndash1850 2019
[29] X Lan M Ye S Zhang and P C Yuen ldquoRobust collaborativediscriminative learning for rgb-infrared trackingrdquo in Proceed-ings of the 32th AAAI Conference on Artificial Intelligence pp7008ndash7015 AAAI New Orleans LA USA 2018
Mathematical Problems in Engineering 11
[30] C Li X Wu N Zhao X Cao and J Tang ldquoFusing two-streamconvolutional neural networks for RGB-T object trackingrdquoNeurocomputing vol 281 pp 78ndash85 2018
[31] JWagner V FischerMHerman and S Behnke ldquoMultispectralpedestrian detection using deep fusion convolutional neuralnetworksrdquo in Proceedings of the 24th European Symposium onArtificial Neural Networks pp 1ndash6 Bruges Belgium 2016
[32] S Hare A Saffari and P H S Torr ldquoStruck structured outputtracking with kernelsrdquo in Proceedings of the IEEE InternationalConference on Computer Vision pp 263ndash270 IEEE ProvidenceRI USA 2012
[33] K Zhang L Zhang andM-H Yang ldquoReal-time object trackingvia online discriminative feature selectionrdquo IEEE Transactionson Image Processing vol 22 no 12 pp 4664ndash4677 2013
[34] K Zhang L Zhang M Yang and D Zhang ldquoFast trackingvia spatio- temporal context learningrdquo Computer Vision andPattern Recognition 2013
[35] J F Henriques R Caseiro P Martins and J Batista ldquoHigh-speed tracking with kernelized correlation filtersrdquo IEEE Trans-actions on Pattern Analysis and Machine Intelligence vol 37 no3 pp 583ndash596 2015
[36] X Dong J Shen D Yu W Wang J Liu and H HuangldquoOcclusion-aware real-time object trackingrdquo IEEE Transactionson Multimedia vol 19 no 4 pp 763ndash771 2017
[37] C O Conaire N E OrsquoConnor and A Smeaton ldquoThermo-visual feature fusion for object tracking using multiple spa-tiogram trackersrdquo Machine Vision amp Applications vol 19 no5-6 pp 483ndash494 2008
[38] INO ldquoInorsquos video analytics datasetrdquo httpswwwinocaenvideo-analytics-dataset
[39] C Li X Liang Y Lu Z Nan and T Jin ldquoRGB-T objecttracking benchmark and baselinerdquo IEEE Transactions on ImageProcessing 2018
[40] C O Conaire N E OrsquoConnor and A F Smeaton ldquoDetectoradaptation by maximising agreement between independentdata sourcesrdquo in Proceedings of the 2007 IEEE ComputerSociety Conference on Computer Vision and Pattern RecognitionCVPRrsquo07 pp 1ndash6 IEEE Minneapolis MN USA June 2007
[41] J W Davis and V Sharma ldquoBackground-subtraction usingcontour-based fusion of thermal and visible imageryrdquoComputerVision and Image Understanding vol 106 no 2-3 pp 162ndash1822007
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom
Mathematical Problems in Engineering 5
InputThe 119905-th visible and infrared imagesFor 119905 = 1 to number of frames do
1 Crop the samples and extract the 119897-th (119897 = 1 sdot sdot sdot 119889) channel features 119911119897119905 for visible andinfrared images respectively
2 Compute the discriminative correlation scores 119910119905119890119904 using Eq (9)3 Compute the fusion correlation scores 119910119905119891119906 using Eq (10)4 Obtain the tracking result by maximizing 1199101199051198911199065 Extract the 119897-th (119897 = 1 sdot sdot sdot 119889) channel feature 119904119897119879 of the target samples and the119898-th
(119898 = 1 sdot sdot sdot119872) sample 1199041198971198616 Update the discriminative correlation filters119867119897119879 and119867119898119861 using Eq (6) and Eq (8)
respectivelyend forOutput Target result and the discriminative correlation filters119867119897119879 and119867119898119861
Algorithm 1 Discriminative fusion correlation learning for visible and infrared tracking
41 Experimental Environment and Evaluation CriteriaDFCL was implemented with C++ programming languageandNet Framework 40 in Visual Studio 2010 on an IntelDual-Core 170GHz CPU with 4 GB RAM Two metricsie location error (pixel) and overlapping rate are usedto evaluate the tracking results quantitatively The locationerror is computed as 119890119903119903119900119903 = radic(119909119866 minus 119909119879)2 + (119910119866 minus 119910119879)2where (119909119866 119910119866) and (119909119879 119910119879) are the ground truth (eitherdownloaded from a standard database or located manually)and tracking bounding box centers respectivelyThe trackingoverlapping rate is defined as 119900V119890119903119897119886119901119901119894119899119892 = 119886119903119890119886(119877119874119868119866 cap119877119874119868119879)119886119903119890119886(119877119874119868119866cup119877119874119868119879) where119877119874119868119866 and119877119874119868119879 denote theground truth and tracking bounding box respectively and119886119903119890119886(sdot) is the rectangular area function A smaller locationerror and a larger overlapping rate indicate higher accuracyand robustness
42 Experimental Results The performance of DFCL wascompared with state-of-the-art trackers Struck [32] ODFS[33] STC [34] KCF [35] ROT [36] DCF-based trackersMOSSE [13] DSST [14] fDSST [15] and visible-infraredfusion trackers TSKF [23] MVMKF [27] L1-PF [19] JSR[24] and CSR [25] Figures 2ndash6 present the experimentalresults of the test trackers in challenging visible sequencesnamed Biker [37] Campus [37] Car [38] Crossroad [39]Hotkettle [39] Inglassandmobile [39] Labman [40] Pedes-trian [41] and Runner [38] as well as their correspondinginfrared sequences Biker-ir Campus-ir Car-ir Crossroad-ir Labman-ir Hotkettle-ir Inglassandmobile-ir Pedestrian-ir and Runner-ir Single-sensor trackers were separatelytested on visible and the corresponding infrared sequenceswhile visible-infrared fusion trackers obtain the results withinformation fromboth visible and infrared sequences For theconvenience of presentation some tracking curves are notshown entirely in the Figures Next the performance of thetrackers in each sequence is described in detail
(a) SequencesBiker andBiker-irBiker presents the exam-ple of complex background clutters The target human in thevisible sequence encounters similar background disturbance(ie bikes) which causes the ODFS MOSSE fDSST TSKF
and MVMKF trackers to drift away from the target Thecorresponding infrared sequence Biker-ir provides temper-ature information that eliminates the background clutter inBiker But when the target is approaching another person ataround Frame 20 Struck ODFS STC MOSSE TSKF andMVMKF do not perform well because they are not able todistinguish target from persons with similar temperature ininfrared sequences Only KCF ROT DSST and our DFCLhave achieved precise and robust performances in thesesequences
(b) Sequences Campus and Campus-ir the target inCampus and Campus-ir undergoes background cluttersocclusion and scale variation At the beginning of CampusODFS STC KCF and ROT lose the target due to backgrounddisturbance Only TSKF and DFCL perform well whileStruck fDSST and MVMKF do not achieve accurate resultsBecause of the infrared information provided by Campus-irfewer test trackers lose tracking when background cluttershappen as shown in Figure 2 But Struck KCF and ROTmistake another person for the target As shown in Figure 2most of the trackers result in tracking failures whereas DFCLoutperforms the others in most metrics (location accuracyand success rate)
(c) SequencesCar andCar-irCar andCar-ir demonstratethe efficiency of DFCL on coping with heavy occlusions Thetarget driving car is occluded by lampposts and trees manytimes which cause tracking failures of most trackers OnlyTSKF MVMKF and DFCL are able to handle the occlusionthroughout the tracking process in this sequence As shownin Figure 2most trackers performbetter inCar-ir than inCarbecause the infrared features can overcome the difficultiesof target detection among surrounding similar backgroundSTC TSKF MVMKF and DFCL are able to handle thisproblem whereas the result of DFCL is the most accurate asshown in Figure 2
(d) Sequences Crossroad and Crossroad-ir the target inCrossroad and Crossroad-ir undergoes heavy backgroundclutters when she crosses the roadWhile the target is passingby the road lamp both ODFS and JSR lose the target Thenwhen a car passes by the target Struck TSKF and MVMKFdrift away from the target When the target goes toward
6 Mathematical Problems in Engineering
20 53 147
Biker
20 53 147
Biker-ir
8 84 156
Campus
8 84 156
Campus-ir
14 23 27
Car
14 23 27
Car-ir
136 289 767
Crossroad
136 289 767
Crossroad-ir
133 451 1068
Hotkettle
133 451 1068
Hotkettle-ir
309 428 511
Inglassandmobile
309 428 511
Inglassandmobile-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF
302 320 446
Labman
302 320 446
Labman-ir
29 228 363
Pedestrian
29 228 363
Pedestrian-ir
14 94 138
Runner
14 94 138
Runner-ir
L1-PF JSR CSR
Figure 2 Tracking performances of the test sequences
Mathematical Problems in Engineering 7
Biker
0
40
80
1 41 81 121
Biker-ir
0
50
100
150
1 41 81 121
Campus
0
50
100
150
200
1 51 101 151 201
Campus-ir
0
100
200
300
1 51 101 151 201
Car
0
100
200
1 11 21 31
Car-ir
0
100
200
300
1 11 21 31
0
50
100
150
200
250
1 201 401 6010
100
200
1 201 401 601
Crossroad Crossroad-ir
0
50
100
150
200
250
300
1 201 401 601 801 1001
Hotkettle
0
50
100
150
200
250
1 201 401 601 801 1001
Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0
50
100
150
200
1 201 401 6010
50
100
150
1 201 401 601
Labman
0
100
200
300
1 101 201 301 401
Labman-ir
0
200
400
1 101 201 301 401
Pedestrian
0
50
100
150
1 101 201 301
Pedestrian-ir
0
50
100
150
1 101 201 301
Runner
0
100
200
1 61 121 181
Runner-ir
0
100
200
1 61 121 181
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
Figure 3 Location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
0
05
1
1 41 81 1210
05
1
1 41 81 1210
05
1
1 51 101 151 2010
05
1
1 51 101 151 2010
05
1
1 11 21 310
05
1
1 11 21 31
0
05
1
1 201 401 6010
05
1
1 201 401 6010
05
1
1 201 401 601 801 10010
05
1
1 201 401 601 801 10010
05
1
1 201 401 6010
05
1
1 201 401 601
0
05
1
1 101 201 301 4010
05
1
1 101 201 301 4010
05
1
1 101 201 3010
05
1
1 101 201 3010
05
1
1 61 121 1810
05
1
1 61 121 181
Figure 4 Overlapping rate of the test sequences
the sidewalk most of the trackers are not able to handlethe problem of heavy background clutters but our trackerperforms satisfying tracking results as shown in Figures 2ndash4
(e) Sequences Hotkettle and Hotkettle-ir in thesesequences tracking is hard because of the changes of thecomplex background clutters Most trackers perform betterin Hotkettle-ir than in Hotkettle for the reason that thetemperature diverge makes the hot target more distinct inthe cold background Struck KCF DSST fDSST and DFCL
can achieve robust and accurate tracking performances asshown in Figures 2ndash4
(f) Sequences Inglassandmobile and Inglassandmobile-irSequences Inglassandmobile and Inglassandmobile-ir demon-strate the performances of the 14 trackers under the cir-cumstances of background clutters illumination changesand occlusion As shown in Figure 2 when the illuminationchanges at around Frames 300 ODFS and fDSST lose thetarget and KCF TSKF and L1-PF drift a little away from the
8 Mathematical Problems in Engineering
10048
96602
171062
9467 27351 20776 17357
3462 21838 17229
2866
241288
9947 6518
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Biker Biker-ir Campus Campus-ir
4536
114901
6739 4323 3944
187420
3574 3805 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2758 9391
287670
2255 4284
216795
1778 2248 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
6301
165743 160172
122008 116442
151244
7767 7767 3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
59762
9866
138565
63421 63817
145486
11886 16255
3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Car Car-ir
122137
109606
122092 122500 122514
106350
121940 122072
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
25018 22604 24216
122500
55885
106350
121940 122530
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Labman Labman-ir
19128
263409
304244
34664 26765
390209
27024 18464 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
7160 16449
377858
7428 38194
497072
21473 12348 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Pedestrian Pedestrian-ir Runner Runner-ir
117051 116737 120324
142077 137828 144675
121509
144107
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
9457
55577
141371 142012
43182
435192
141348 141350
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
116065
92197
30636
129168
27984
100629
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2876
117247
2341
127301
3465
297251
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
9575
300997
60040
11480 15424
287956
19187 2840
21838 17229 2866
241288
9947 4246
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
3638
79438 54283
4987
55103
313867
3023 4995 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
5983 9604
28494
4086 5710
196150
2693 4255 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
6687
66141
48306
16131
63235
116459
14736
74415
18142 10318 10791 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
4160
16714 26378
16071
3345
130137
14534
60554
18142 10318 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
10791
Figure 5 Average location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0993
0311
0986 0993 0993
0108
0993 0993 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
1000 0973
0014
1000 1000
0108
1000 1000 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0907
0005 0000 0056 0070
0009
0986 0986 1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0437
1000
0023
0158 0186
0028
0935
0758
1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0059
0176
0088 0088 0088 0059
0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0824
0382
0941
0088
0471
0059 0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0847
0026 0026
0520
0714
0022
0677 0733
0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0991
0877
0004
0994
0011 0009
0708
0959 0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0068 0054 0078 0049 0054 0044
0083 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0605
0020 0054 0054
0502
0024 0054 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0101
0189
0481
0016
0598
0044
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0937
0005
0904
0019
0943
0033
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0823
0016
0186
0808
0585 0585 0528
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0823
0016
0302
0808 0819
0016
0472
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0585
0105
0331
0561
0318
0000
0585 0558 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0571
0222
0454
0585 0578
0006
0585 0585 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976
0035
0749
0884
0587
0118
0917
0245
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976 0948
0604
0887
1000
0089
0925
0580
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Figure 6 Success rate of the test sequences
targetWhen the target is approaching a tree the backgroundclutters makes most of trackers cause tracking failures thatcan be seen from Figure 2 Our DFCL can overcome thesechallenges and perform well in these sequences
(g) Sequences Labman and Labman-ir the experimentsin Sequences Labman and Labman-ir aim to evaluate the per-formances on tracking under appearance variation rotationscale variation and background clutter In Labman whenthe target man is walking into the laboratory ODFS STCand MOSSE lose the target When the man keeps shaking
and turning around his head at around Frame 400 KCFROT and DSST cause tracking failures Also most trackersachieve better tracking performances in Labman-ir as shownin Figure 2
(h) Sequences Pedestrian and Pedestrian-ir the target inPedestrian and Pedestrian-ir undergoes heavy backgroundclutters and occlusion As shown in Figure 2 other trackersresult in tracking failures in Pedestrian whereas our trackershows satisfying performances in terms of both accuracyand robustnessThe efficient infrared features extracted from
Mathematical Problems in Engineering 9
DFCLDCL MOSSE
Average location error (pixel) Success rate
011 001 006
059
000 012
002 004 004
052
019 017
063
021
073
039 053
037
100 100
082 082
057
099 100 099 099
18742 15124
10635
2078
31387
11646
39021
10063 14467
1853 6166 8940
1786
9545 4990 5658
2692 9228
340 315 1899 652 495 723 1013 138 284
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Figure 7 Average location error (pixel) and success rate of DCL DFCL and MOSSE on the test sequences
Pedestrian-ir ensure the tracking successes of Struck STCand DFCL as can be seen from Figures 2ndash4
(i) Sequences Runner and Runner-ir Runner and Runner-ir contain examples of heavy occlusion abrupt movementand scale variation The target running man is occludedby lampposts trees stone tablet and bushes many timesresulting in tracking failures ofmost trackers Also the abruptmovement and scale variation cause many trackers to driftaway the target in both Runner and Runner-ir as shown inFigure 2 Once again ourDFCL is able to overcome the aboveproblems and achieve good performances
Figures 5 and 6 are included here to demonstrate quan-titatively the performances on average location error (pixel)and success rate The success rate is defined as the numberof times success is achieved in the whole tracking processby considering one frame as a success if the overlapping rateexceeds 05 [33] A smaller average location error and a largersuccess rate indicate increased accuracy and robustnessFigures 5 and 6 show that DFCL performs satisfying most ofthe tracking sequences
To validate the effectiveness of the discriminative filterselection model of DFCL we compare the tracker DCL(the proposed DFCL without the fusion learning model)with DFCL and the original DCF tracker MOSSE on visiblesequencesThe performances shown in Figure 7 demonstratethe efficiency of the discriminative filter selection modelespecially in the sequences with background clutters ieSequences BikerHotkettle Inglassandmobile and Pedestrian
5 Conclusion
Discriminative correlation filter- (DCF-) based trackers havethe advantage of being computationally efficient and morerobust than most of the other state-of-the-art trackers inchallenging tracking tasks thereby making them especiallysuitable for a variety of real-time challenging applications
Howevermost of theDCF-based trackers suffer low accuracydue to the lack of diversity information extracted from asingle type of spectral image (visible spectrum) Fusion ofvisible and infrared sensors one of the typical multisensorcooperation provides complementarily useful features andconsistently helps recognize the target from the backgroundefficiently in visual tracking For the above reasons this paperproposes a discriminative fusion correlation learning modelto improve DCF-based tracking performance by combiningmultiple features from visible and infrared imaging sensorsThe proposed fusion learning filters are obtained via latefusion with early estimation in which the performances ofthe filters are weighted to improve the flexibility of fusionMoreover the proposed discriminative filter selection modelconsiders the surrounding background information in orderto increase the discriminability of the template filters soas to improve model learning Numerous real-world videosequences were used to test DFCL and other state-of-the-art algorithms and here we only selected representativevideos for presentation Experimental results demonstratedthat DFCL is highly accurate and robust
Data Availability
The data used to support the findings of this study weresupplied by China University of Mining and Technologyunder license and so cannot be made freely available
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
This work is supported by the Natural Science Foun-dation of Jiangsu Province (BK20180640 BK20150204)
10 Mathematical Problems in Engineering
Research Development Programme of Jiangsu Province(BE2015040) the State Key Research Development Program(2016YFC0801403) and the National Natural Science Foun-dation of China (51504214 51504255 51734009 61771417 and61873246)
References
[1] S A Wibowo H Lee E K Kim and S Kim ldquoVisual trackingbased on complementary learners with distractor handlingrdquoMathematical Problems in Engineering vol 2017 Article ID5295601 13 pages 2017
[2] T Zhou M Zhu D Zeng and H Yang ldquoScale adaptive kernel-ized correlation filter tracker with feature fusionrdquoMathematicalProblems in Engineering vol 2017 Article ID 1605959 8 pages2017
[3] C Wang H K Galoogahi C Lin and S Lucey ldquoDeep-LK forefficient adaptive object trackingrdquo Computer Vision and PatternRecognition 2017
[4] Z Chi H Li H Lu and M-H Yang ldquoDual deep network forvisual trackingrdquo IEEE Transactions on Image Processing vol 26no 4 pp 2005ndash2015 2017
[5] Y Yang W Hu Y Xie W Zhang and T Zhang ldquoTemporalrestricted visual tracking via reverse-low-rank sparse learningrdquoIEEE Transactions on Cybernetics vol 47 no 2 pp 485ndash4982017
[6] Y Sui G Wang and L Zhang ldquoCorrelation filter learningtoward peak strength for visual trackingrdquo IEEE Transactions onCybernetics vol 48 no 4 pp 1290ndash1303 2018
[7] K Zhang X Li H Song Q Liu and W Lian ldquoVisual track-ing using spatio-temporally nonlocally regularized correlationfilterrdquo Pattern Recognition vol 83 pp 185ndash195 2018
[8] Y Qi L Qin J Zhang S Zhang Q Huang and M-H YangldquoStructure-aware local sparse coding for visual trackingrdquo IEEETransactions on Image Processing vol 27 no 8 pp 3857ndash38692018
[9] B Bai B Zhong G Ouyang et al ldquoKernel correlation filtersfor visual tracking with adaptive fusion of heterogeneous cuesrdquoNeurocomputing vol 286 pp 109ndash120 2018
[10] Y Liu F Yang C Zhong Y Tao B Dai and M Yin ldquoVisualtracking via salient feature extraction and sparse collaborativemodelrdquo AEU - International Journal of Electronics and Commu-nications vol 87 pp 134ndash143 2018
[11] X Yun Y Sun S Wang Y Shi and N Lu ldquoMulti-layerconvolutional network-based visual tracking via importantregion selectionrdquo Neurocomputing vol 315 pp 145ndash156 2018
[12] X Lan S Zhang and P C Yuen ldquoRobust joint discriminativefeature learning for visual trackingrdquo in Proceedings of the 25thInternational Joint Conference on Artificial Intelligence IJCAI2016 pp 3403ndash3410 AAAI Press New York NY USA July2016
[13] D S Bolme J R Beveridge B A Draper and Y M LuildquoVisual object tracking using adaptive correlation filtersrdquo inProceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo10) pp 2544ndash2550 IEEE SanFrancisco Calif USA June 2010
[14] M Danelljan G Hager F Shahbaz Khan and M FelsbergldquoAccurate scale estimation for robust visual trackingrdquo in Pro-ceedings of the British Machine Vision Conference pp 651-6511BMVA Press Nottingham UK 2014
[15] M Danelljan G Hager F S Khan and M Felsberg ldquoDis-criminative scale space trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 39 no 8 pp 1561ndash15752017
[16] H K Galoogahi A Fagg and S Lucey ldquoLearning background-aware correlation filters for visual trackingrdquo inProceedings of the16th IEEE International Conference on Computer Vision ICCV2017 pp 1144ndash1152 IEEE Istanbul Turkey 2018
[17] Y SongCMa LGong J Zhang RWH Lau andM-HYangldquoCrest convolutional residual learning for visual trackingrdquoin Proceedings of the 16th IEEE International Conference onComputer Vision ICCV 2017 pp 2574ndash2583 IEEEVenice ItalyOctober 2017
[18] J Johnander M Danelljan F Khan andM Felsberg ldquoDCCOtowards deformable continuous convolution operators forvisual trackingrdquo in Proceedings of the International Conferenceon Computer Analysis of Images and Patterns pp 55ndash67Springer Ystad Sweden 2017
[19] Y Wu E Blasch G Chen L Bai and H Ling ldquoMultiplesource data fusion via sparse representation for robust visualtrackingrdquo in Proceedings of the 14th International Conference onInformation Fusion Fusion 2011 IEEE Chicago IL USA July2011
[20] B Uzkent A Rangnekar and M J Hoffman ldquoAerial vehicletracking by adaptive fusion of hyperspectral likelihood mapsrdquoin Proceedings of the 30th IEEE Conference on Computer Visionand Pattern RecognitionWorkshops CVPRW 2017 pp 233ndash242IEEE Honolulu HI USA July 2017
[21] S Chan X Zhou and S Chen ldquoRobust adaptive fusion trackingbased on complex cells and keypointsrdquo IEEE Access vol 5 pp20985ndash21001 2017
[22] M K Rapuru S Kakanuru P M Venugopal D Mishra and GR Subrahmanyam ldquoCorrelation-based tracker-level fusion forrobust visual trackingrdquo IEEE Transactions on Image Processingvol 26 no 10 pp 4832ndash4842 2017
[23] X Yun Z Jing G Xiao B Jin and C Zhang ldquoA compressivetracking based on time-space Kalman fusion modelrdquo ScienceChina Information Sciences vol 59 no 1 pp 1ndash15 2016
[24] H P Liu and F C Sun ldquoFusion tracking in color and infraredimages using joint sparse representationrdquo Science China Infor-mation Sciences vol 55 no 3 pp 590ndash599 2012
[25] C Li H Cheng S Hu X Liu J Tang and L Lin ldquoLearningcollaborative sparse representation for grayscale-thermal track-ingrdquo IEEE Transactions on Image Processing vol 25 no 12 pp5743ndash5756 2016
[26] S Mangale and M Khambete ldquoCamouflaged target detectionand tracking using thermal infrared and visible spectrum imag-ingrdquo in Automatic Diagnosis of Breast Cancer using Thermo-graphic Color Analysis and SVM Classifier vol 530 of Advancesin Intelligent Systems and Computing pp 193ndash207 SpringerInternational Publishing 2016
[27] X Yun Z Jing and B Jin ldquoVisible and infrared tracking basedon multi-view multi-kernel fusion modelrdquo Optical Review vol23 no 2 pp 244ndash253 2016
[28] L Zhang A Gonzalez-Garcia J van de Weijer M Danelljanand F S Khan ldquoSynthetic data generation for end-to-end ther-mal infrared trackingrdquo IEEE Transactions on Image Processingvol 28 no 4 pp 1837ndash1850 2019
[29] X Lan M Ye S Zhang and P C Yuen ldquoRobust collaborativediscriminative learning for rgb-infrared trackingrdquo in Proceed-ings of the 32th AAAI Conference on Artificial Intelligence pp7008ndash7015 AAAI New Orleans LA USA 2018
Mathematical Problems in Engineering 11
[30] C Li X Wu N Zhao X Cao and J Tang ldquoFusing two-streamconvolutional neural networks for RGB-T object trackingrdquoNeurocomputing vol 281 pp 78ndash85 2018
[31] JWagner V FischerMHerman and S Behnke ldquoMultispectralpedestrian detection using deep fusion convolutional neuralnetworksrdquo in Proceedings of the 24th European Symposium onArtificial Neural Networks pp 1ndash6 Bruges Belgium 2016
[32] S Hare A Saffari and P H S Torr ldquoStruck structured outputtracking with kernelsrdquo in Proceedings of the IEEE InternationalConference on Computer Vision pp 263ndash270 IEEE ProvidenceRI USA 2012
[33] K Zhang L Zhang andM-H Yang ldquoReal-time object trackingvia online discriminative feature selectionrdquo IEEE Transactionson Image Processing vol 22 no 12 pp 4664ndash4677 2013
[34] K Zhang L Zhang M Yang and D Zhang ldquoFast trackingvia spatio- temporal context learningrdquo Computer Vision andPattern Recognition 2013
[35] J F Henriques R Caseiro P Martins and J Batista ldquoHigh-speed tracking with kernelized correlation filtersrdquo IEEE Trans-actions on Pattern Analysis and Machine Intelligence vol 37 no3 pp 583ndash596 2015
[36] X Dong J Shen D Yu W Wang J Liu and H HuangldquoOcclusion-aware real-time object trackingrdquo IEEE Transactionson Multimedia vol 19 no 4 pp 763ndash771 2017
[37] C O Conaire N E OrsquoConnor and A Smeaton ldquoThermo-visual feature fusion for object tracking using multiple spa-tiogram trackersrdquo Machine Vision amp Applications vol 19 no5-6 pp 483ndash494 2008
[38] INO ldquoInorsquos video analytics datasetrdquo httpswwwinocaenvideo-analytics-dataset
[39] C Li X Liang Y Lu Z Nan and T Jin ldquoRGB-T objecttracking benchmark and baselinerdquo IEEE Transactions on ImageProcessing 2018
[40] C O Conaire N E OrsquoConnor and A F Smeaton ldquoDetectoradaptation by maximising agreement between independentdata sourcesrdquo in Proceedings of the 2007 IEEE ComputerSociety Conference on Computer Vision and Pattern RecognitionCVPRrsquo07 pp 1ndash6 IEEE Minneapolis MN USA June 2007
[41] J W Davis and V Sharma ldquoBackground-subtraction usingcontour-based fusion of thermal and visible imageryrdquoComputerVision and Image Understanding vol 106 no 2-3 pp 162ndash1822007
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom
6 Mathematical Problems in Engineering
20 53 147
Biker
20 53 147
Biker-ir
8 84 156
Campus
8 84 156
Campus-ir
14 23 27
Car
14 23 27
Car-ir
136 289 767
Crossroad
136 289 767
Crossroad-ir
133 451 1068
Hotkettle
133 451 1068
Hotkettle-ir
309 428 511
Inglassandmobile
309 428 511
Inglassandmobile-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF
302 320 446
Labman
302 320 446
Labman-ir
29 228 363
Pedestrian
29 228 363
Pedestrian-ir
14 94 138
Runner
14 94 138
Runner-ir
L1-PF JSR CSR
Figure 2 Tracking performances of the test sequences
Mathematical Problems in Engineering 7
Biker
0
40
80
1 41 81 121
Biker-ir
0
50
100
150
1 41 81 121
Campus
0
50
100
150
200
1 51 101 151 201
Campus-ir
0
100
200
300
1 51 101 151 201
Car
0
100
200
1 11 21 31
Car-ir
0
100
200
300
1 11 21 31
0
50
100
150
200
250
1 201 401 6010
100
200
1 201 401 601
Crossroad Crossroad-ir
0
50
100
150
200
250
300
1 201 401 601 801 1001
Hotkettle
0
50
100
150
200
250
1 201 401 601 801 1001
Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0
50
100
150
200
1 201 401 6010
50
100
150
1 201 401 601
Labman
0
100
200
300
1 101 201 301 401
Labman-ir
0
200
400
1 101 201 301 401
Pedestrian
0
50
100
150
1 101 201 301
Pedestrian-ir
0
50
100
150
1 101 201 301
Runner
0
100
200
1 61 121 181
Runner-ir
0
100
200
1 61 121 181
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
Figure 3 Location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
0
05
1
1 41 81 1210
05
1
1 41 81 1210
05
1
1 51 101 151 2010
05
1
1 51 101 151 2010
05
1
1 11 21 310
05
1
1 11 21 31
0
05
1
1 201 401 6010
05
1
1 201 401 6010
05
1
1 201 401 601 801 10010
05
1
1 201 401 601 801 10010
05
1
1 201 401 6010
05
1
1 201 401 601
0
05
1
1 101 201 301 4010
05
1
1 101 201 301 4010
05
1
1 101 201 3010
05
1
1 101 201 3010
05
1
1 61 121 1810
05
1
1 61 121 181
Figure 4 Overlapping rate of the test sequences
the sidewalk most of the trackers are not able to handlethe problem of heavy background clutters but our trackerperforms satisfying tracking results as shown in Figures 2ndash4
(e) Sequences Hotkettle and Hotkettle-ir in thesesequences tracking is hard because of the changes of thecomplex background clutters Most trackers perform betterin Hotkettle-ir than in Hotkettle for the reason that thetemperature diverge makes the hot target more distinct inthe cold background Struck KCF DSST fDSST and DFCL
can achieve robust and accurate tracking performances asshown in Figures 2ndash4
(f) Sequences Inglassandmobile and Inglassandmobile-irSequences Inglassandmobile and Inglassandmobile-ir demon-strate the performances of the 14 trackers under the cir-cumstances of background clutters illumination changesand occlusion As shown in Figure 2 when the illuminationchanges at around Frames 300 ODFS and fDSST lose thetarget and KCF TSKF and L1-PF drift a little away from the
8 Mathematical Problems in Engineering
10048
96602
171062
9467 27351 20776 17357
3462 21838 17229
2866
241288
9947 6518
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Biker Biker-ir Campus Campus-ir
4536
114901
6739 4323 3944
187420
3574 3805 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2758 9391
287670
2255 4284
216795
1778 2248 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
6301
165743 160172
122008 116442
151244
7767 7767 3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
59762
9866
138565
63421 63817
145486
11886 16255
3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Car Car-ir
122137
109606
122092 122500 122514
106350
121940 122072
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
25018 22604 24216
122500
55885
106350
121940 122530
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Labman Labman-ir
19128
263409
304244
34664 26765
390209
27024 18464 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
7160 16449
377858
7428 38194
497072
21473 12348 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Pedestrian Pedestrian-ir Runner Runner-ir
117051 116737 120324
142077 137828 144675
121509
144107
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
9457
55577
141371 142012
43182
435192
141348 141350
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
116065
92197
30636
129168
27984
100629
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2876
117247
2341
127301
3465
297251
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
9575
300997
60040
11480 15424
287956
19187 2840
21838 17229 2866
241288
9947 4246
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
3638
79438 54283
4987
55103
313867
3023 4995 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
5983 9604
28494
4086 5710
196150
2693 4255 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
6687
66141
48306
16131
63235
116459
14736
74415
18142 10318 10791 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
4160
16714 26378
16071
3345
130137
14534
60554
18142 10318 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
10791
Figure 5 Average location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0993
0311
0986 0993 0993
0108
0993 0993 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
1000 0973
0014
1000 1000
0108
1000 1000 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0907
0005 0000 0056 0070
0009
0986 0986 1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0437
1000
0023
0158 0186
0028
0935
0758
1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0059
0176
0088 0088 0088 0059
0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0824
0382
0941
0088
0471
0059 0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0847
0026 0026
0520
0714
0022
0677 0733
0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0991
0877
0004
0994
0011 0009
0708
0959 0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0068 0054 0078 0049 0054 0044
0083 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0605
0020 0054 0054
0502
0024 0054 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0101
0189
0481
0016
0598
0044
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0937
0005
0904
0019
0943
0033
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0823
0016
0186
0808
0585 0585 0528
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0823
0016
0302
0808 0819
0016
0472
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0585
0105
0331
0561
0318
0000
0585 0558 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0571
0222
0454
0585 0578
0006
0585 0585 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976
0035
0749
0884
0587
0118
0917
0245
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976 0948
0604
0887
1000
0089
0925
0580
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Figure 6 Success rate of the test sequences
targetWhen the target is approaching a tree the backgroundclutters makes most of trackers cause tracking failures thatcan be seen from Figure 2 Our DFCL can overcome thesechallenges and perform well in these sequences
(g) Sequences Labman and Labman-ir the experimentsin Sequences Labman and Labman-ir aim to evaluate the per-formances on tracking under appearance variation rotationscale variation and background clutter In Labman whenthe target man is walking into the laboratory ODFS STCand MOSSE lose the target When the man keeps shaking
and turning around his head at around Frame 400 KCFROT and DSST cause tracking failures Also most trackersachieve better tracking performances in Labman-ir as shownin Figure 2
(h) Sequences Pedestrian and Pedestrian-ir the target inPedestrian and Pedestrian-ir undergoes heavy backgroundclutters and occlusion As shown in Figure 2 other trackersresult in tracking failures in Pedestrian whereas our trackershows satisfying performances in terms of both accuracyand robustnessThe efficient infrared features extracted from
Mathematical Problems in Engineering 9
DFCLDCL MOSSE
Average location error (pixel) Success rate
011 001 006
059
000 012
002 004 004
052
019 017
063
021
073
039 053
037
100 100
082 082
057
099 100 099 099
18742 15124
10635
2078
31387
11646
39021
10063 14467
1853 6166 8940
1786
9545 4990 5658
2692 9228
340 315 1899 652 495 723 1013 138 284
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Figure 7 Average location error (pixel) and success rate of DCL DFCL and MOSSE on the test sequences
Pedestrian-ir ensure the tracking successes of Struck STCand DFCL as can be seen from Figures 2ndash4
(i) Sequences Runner and Runner-ir Runner and Runner-ir contain examples of heavy occlusion abrupt movementand scale variation The target running man is occludedby lampposts trees stone tablet and bushes many timesresulting in tracking failures ofmost trackers Also the abruptmovement and scale variation cause many trackers to driftaway the target in both Runner and Runner-ir as shown inFigure 2 Once again ourDFCL is able to overcome the aboveproblems and achieve good performances
Figures 5 and 6 are included here to demonstrate quan-titatively the performances on average location error (pixel)and success rate The success rate is defined as the numberof times success is achieved in the whole tracking processby considering one frame as a success if the overlapping rateexceeds 05 [33] A smaller average location error and a largersuccess rate indicate increased accuracy and robustnessFigures 5 and 6 show that DFCL performs satisfying most ofthe tracking sequences
To validate the effectiveness of the discriminative filterselection model of DFCL we compare the tracker DCL(the proposed DFCL without the fusion learning model)with DFCL and the original DCF tracker MOSSE on visiblesequencesThe performances shown in Figure 7 demonstratethe efficiency of the discriminative filter selection modelespecially in the sequences with background clutters ieSequences BikerHotkettle Inglassandmobile and Pedestrian
5 Conclusion
Discriminative correlation filter- (DCF-) based trackers havethe advantage of being computationally efficient and morerobust than most of the other state-of-the-art trackers inchallenging tracking tasks thereby making them especiallysuitable for a variety of real-time challenging applications
Howevermost of theDCF-based trackers suffer low accuracydue to the lack of diversity information extracted from asingle type of spectral image (visible spectrum) Fusion ofvisible and infrared sensors one of the typical multisensorcooperation provides complementarily useful features andconsistently helps recognize the target from the backgroundefficiently in visual tracking For the above reasons this paperproposes a discriminative fusion correlation learning modelto improve DCF-based tracking performance by combiningmultiple features from visible and infrared imaging sensorsThe proposed fusion learning filters are obtained via latefusion with early estimation in which the performances ofthe filters are weighted to improve the flexibility of fusionMoreover the proposed discriminative filter selection modelconsiders the surrounding background information in orderto increase the discriminability of the template filters soas to improve model learning Numerous real-world videosequences were used to test DFCL and other state-of-the-art algorithms and here we only selected representativevideos for presentation Experimental results demonstratedthat DFCL is highly accurate and robust
Data Availability
The data used to support the findings of this study weresupplied by China University of Mining and Technologyunder license and so cannot be made freely available
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
This work is supported by the Natural Science Foun-dation of Jiangsu Province (BK20180640 BK20150204)
10 Mathematical Problems in Engineering
Research Development Programme of Jiangsu Province(BE2015040) the State Key Research Development Program(2016YFC0801403) and the National Natural Science Foun-dation of China (51504214 51504255 51734009 61771417 and61873246)
References
[1] S A Wibowo H Lee E K Kim and S Kim ldquoVisual trackingbased on complementary learners with distractor handlingrdquoMathematical Problems in Engineering vol 2017 Article ID5295601 13 pages 2017
[2] T Zhou M Zhu D Zeng and H Yang ldquoScale adaptive kernel-ized correlation filter tracker with feature fusionrdquoMathematicalProblems in Engineering vol 2017 Article ID 1605959 8 pages2017
[3] C Wang H K Galoogahi C Lin and S Lucey ldquoDeep-LK forefficient adaptive object trackingrdquo Computer Vision and PatternRecognition 2017
[4] Z Chi H Li H Lu and M-H Yang ldquoDual deep network forvisual trackingrdquo IEEE Transactions on Image Processing vol 26no 4 pp 2005ndash2015 2017
[5] Y Yang W Hu Y Xie W Zhang and T Zhang ldquoTemporalrestricted visual tracking via reverse-low-rank sparse learningrdquoIEEE Transactions on Cybernetics vol 47 no 2 pp 485ndash4982017
[6] Y Sui G Wang and L Zhang ldquoCorrelation filter learningtoward peak strength for visual trackingrdquo IEEE Transactions onCybernetics vol 48 no 4 pp 1290ndash1303 2018
[7] K Zhang X Li H Song Q Liu and W Lian ldquoVisual track-ing using spatio-temporally nonlocally regularized correlationfilterrdquo Pattern Recognition vol 83 pp 185ndash195 2018
[8] Y Qi L Qin J Zhang S Zhang Q Huang and M-H YangldquoStructure-aware local sparse coding for visual trackingrdquo IEEETransactions on Image Processing vol 27 no 8 pp 3857ndash38692018
[9] B Bai B Zhong G Ouyang et al ldquoKernel correlation filtersfor visual tracking with adaptive fusion of heterogeneous cuesrdquoNeurocomputing vol 286 pp 109ndash120 2018
[10] Y Liu F Yang C Zhong Y Tao B Dai and M Yin ldquoVisualtracking via salient feature extraction and sparse collaborativemodelrdquo AEU - International Journal of Electronics and Commu-nications vol 87 pp 134ndash143 2018
[11] X Yun Y Sun S Wang Y Shi and N Lu ldquoMulti-layerconvolutional network-based visual tracking via importantregion selectionrdquo Neurocomputing vol 315 pp 145ndash156 2018
[12] X Lan S Zhang and P C Yuen ldquoRobust joint discriminativefeature learning for visual trackingrdquo in Proceedings of the 25thInternational Joint Conference on Artificial Intelligence IJCAI2016 pp 3403ndash3410 AAAI Press New York NY USA July2016
[13] D S Bolme J R Beveridge B A Draper and Y M LuildquoVisual object tracking using adaptive correlation filtersrdquo inProceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo10) pp 2544ndash2550 IEEE SanFrancisco Calif USA June 2010
[14] M Danelljan G Hager F Shahbaz Khan and M FelsbergldquoAccurate scale estimation for robust visual trackingrdquo in Pro-ceedings of the British Machine Vision Conference pp 651-6511BMVA Press Nottingham UK 2014
[15] M Danelljan G Hager F S Khan and M Felsberg ldquoDis-criminative scale space trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 39 no 8 pp 1561ndash15752017
[16] H K Galoogahi A Fagg and S Lucey ldquoLearning background-aware correlation filters for visual trackingrdquo inProceedings of the16th IEEE International Conference on Computer Vision ICCV2017 pp 1144ndash1152 IEEE Istanbul Turkey 2018
[17] Y SongCMa LGong J Zhang RWH Lau andM-HYangldquoCrest convolutional residual learning for visual trackingrdquoin Proceedings of the 16th IEEE International Conference onComputer Vision ICCV 2017 pp 2574ndash2583 IEEEVenice ItalyOctober 2017
[18] J Johnander M Danelljan F Khan andM Felsberg ldquoDCCOtowards deformable continuous convolution operators forvisual trackingrdquo in Proceedings of the International Conferenceon Computer Analysis of Images and Patterns pp 55ndash67Springer Ystad Sweden 2017
[19] Y Wu E Blasch G Chen L Bai and H Ling ldquoMultiplesource data fusion via sparse representation for robust visualtrackingrdquo in Proceedings of the 14th International Conference onInformation Fusion Fusion 2011 IEEE Chicago IL USA July2011
[20] B Uzkent A Rangnekar and M J Hoffman ldquoAerial vehicletracking by adaptive fusion of hyperspectral likelihood mapsrdquoin Proceedings of the 30th IEEE Conference on Computer Visionand Pattern RecognitionWorkshops CVPRW 2017 pp 233ndash242IEEE Honolulu HI USA July 2017
[21] S Chan X Zhou and S Chen ldquoRobust adaptive fusion trackingbased on complex cells and keypointsrdquo IEEE Access vol 5 pp20985ndash21001 2017
[22] M K Rapuru S Kakanuru P M Venugopal D Mishra and GR Subrahmanyam ldquoCorrelation-based tracker-level fusion forrobust visual trackingrdquo IEEE Transactions on Image Processingvol 26 no 10 pp 4832ndash4842 2017
[23] X Yun Z Jing G Xiao B Jin and C Zhang ldquoA compressivetracking based on time-space Kalman fusion modelrdquo ScienceChina Information Sciences vol 59 no 1 pp 1ndash15 2016
[24] H P Liu and F C Sun ldquoFusion tracking in color and infraredimages using joint sparse representationrdquo Science China Infor-mation Sciences vol 55 no 3 pp 590ndash599 2012
[25] C Li H Cheng S Hu X Liu J Tang and L Lin ldquoLearningcollaborative sparse representation for grayscale-thermal track-ingrdquo IEEE Transactions on Image Processing vol 25 no 12 pp5743ndash5756 2016
[26] S Mangale and M Khambete ldquoCamouflaged target detectionand tracking using thermal infrared and visible spectrum imag-ingrdquo in Automatic Diagnosis of Breast Cancer using Thermo-graphic Color Analysis and SVM Classifier vol 530 of Advancesin Intelligent Systems and Computing pp 193ndash207 SpringerInternational Publishing 2016
[27] X Yun Z Jing and B Jin ldquoVisible and infrared tracking basedon multi-view multi-kernel fusion modelrdquo Optical Review vol23 no 2 pp 244ndash253 2016
[28] L Zhang A Gonzalez-Garcia J van de Weijer M Danelljanand F S Khan ldquoSynthetic data generation for end-to-end ther-mal infrared trackingrdquo IEEE Transactions on Image Processingvol 28 no 4 pp 1837ndash1850 2019
[29] X Lan M Ye S Zhang and P C Yuen ldquoRobust collaborativediscriminative learning for rgb-infrared trackingrdquo in Proceed-ings of the 32th AAAI Conference on Artificial Intelligence pp7008ndash7015 AAAI New Orleans LA USA 2018
Mathematical Problems in Engineering 11
[30] C Li X Wu N Zhao X Cao and J Tang ldquoFusing two-streamconvolutional neural networks for RGB-T object trackingrdquoNeurocomputing vol 281 pp 78ndash85 2018
[31] JWagner V FischerMHerman and S Behnke ldquoMultispectralpedestrian detection using deep fusion convolutional neuralnetworksrdquo in Proceedings of the 24th European Symposium onArtificial Neural Networks pp 1ndash6 Bruges Belgium 2016
[32] S Hare A Saffari and P H S Torr ldquoStruck structured outputtracking with kernelsrdquo in Proceedings of the IEEE InternationalConference on Computer Vision pp 263ndash270 IEEE ProvidenceRI USA 2012
[33] K Zhang L Zhang andM-H Yang ldquoReal-time object trackingvia online discriminative feature selectionrdquo IEEE Transactionson Image Processing vol 22 no 12 pp 4664ndash4677 2013
[34] K Zhang L Zhang M Yang and D Zhang ldquoFast trackingvia spatio- temporal context learningrdquo Computer Vision andPattern Recognition 2013
[35] J F Henriques R Caseiro P Martins and J Batista ldquoHigh-speed tracking with kernelized correlation filtersrdquo IEEE Trans-actions on Pattern Analysis and Machine Intelligence vol 37 no3 pp 583ndash596 2015
[36] X Dong J Shen D Yu W Wang J Liu and H HuangldquoOcclusion-aware real-time object trackingrdquo IEEE Transactionson Multimedia vol 19 no 4 pp 763ndash771 2017
[37] C O Conaire N E OrsquoConnor and A Smeaton ldquoThermo-visual feature fusion for object tracking using multiple spa-tiogram trackersrdquo Machine Vision amp Applications vol 19 no5-6 pp 483ndash494 2008
[38] INO ldquoInorsquos video analytics datasetrdquo httpswwwinocaenvideo-analytics-dataset
[39] C Li X Liang Y Lu Z Nan and T Jin ldquoRGB-T objecttracking benchmark and baselinerdquo IEEE Transactions on ImageProcessing 2018
[40] C O Conaire N E OrsquoConnor and A F Smeaton ldquoDetectoradaptation by maximising agreement between independentdata sourcesrdquo in Proceedings of the 2007 IEEE ComputerSociety Conference on Computer Vision and Pattern RecognitionCVPRrsquo07 pp 1ndash6 IEEE Minneapolis MN USA June 2007
[41] J W Davis and V Sharma ldquoBackground-subtraction usingcontour-based fusion of thermal and visible imageryrdquoComputerVision and Image Understanding vol 106 no 2-3 pp 162ndash1822007
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom
Mathematical Problems in Engineering 7
Biker
0
40
80
1 41 81 121
Biker-ir
0
50
100
150
1 41 81 121
Campus
0
50
100
150
200
1 51 101 151 201
Campus-ir
0
100
200
300
1 51 101 151 201
Car
0
100
200
1 11 21 31
Car-ir
0
100
200
300
1 11 21 31
0
50
100
150
200
250
1 201 401 6010
100
200
1 201 401 601
Crossroad Crossroad-ir
0
50
100
150
200
250
300
1 201 401 601 801 1001
Hotkettle
0
50
100
150
200
250
1 201 401 601 801 1001
Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0
50
100
150
200
1 201 401 6010
50
100
150
1 201 401 601
Labman
0
100
200
300
1 101 201 301 401
Labman-ir
0
200
400
1 101 201 301 401
Pedestrian
0
50
100
150
1 101 201 301
Pedestrian-ir
0
50
100
150
1 101 201 301
Runner
0
100
200
1 61 121 181
Runner-ir
0
100
200
1 61 121 181
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
Figure 3 Location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
MOSSEODFS KCFSTCStruck ROT DSST fDSST DFCLTSKF MVMKF L1-PF JSR CSR
0
05
1
1 41 81 1210
05
1
1 41 81 1210
05
1
1 51 101 151 2010
05
1
1 51 101 151 2010
05
1
1 11 21 310
05
1
1 11 21 31
0
05
1
1 201 401 6010
05
1
1 201 401 6010
05
1
1 201 401 601 801 10010
05
1
1 201 401 601 801 10010
05
1
1 201 401 6010
05
1
1 201 401 601
0
05
1
1 101 201 301 4010
05
1
1 101 201 301 4010
05
1
1 101 201 3010
05
1
1 101 201 3010
05
1
1 61 121 1810
05
1
1 61 121 181
Figure 4 Overlapping rate of the test sequences
the sidewalk most of the trackers are not able to handlethe problem of heavy background clutters but our trackerperforms satisfying tracking results as shown in Figures 2ndash4
(e) Sequences Hotkettle and Hotkettle-ir in thesesequences tracking is hard because of the changes of thecomplex background clutters Most trackers perform betterin Hotkettle-ir than in Hotkettle for the reason that thetemperature diverge makes the hot target more distinct inthe cold background Struck KCF DSST fDSST and DFCL
can achieve robust and accurate tracking performances asshown in Figures 2ndash4
(f) Sequences Inglassandmobile and Inglassandmobile-irSequences Inglassandmobile and Inglassandmobile-ir demon-strate the performances of the 14 trackers under the cir-cumstances of background clutters illumination changesand occlusion As shown in Figure 2 when the illuminationchanges at around Frames 300 ODFS and fDSST lose thetarget and KCF TSKF and L1-PF drift a little away from the
8 Mathematical Problems in Engineering
10048
96602
171062
9467 27351 20776 17357
3462 21838 17229
2866
241288
9947 6518
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Biker Biker-ir Campus Campus-ir
4536
114901
6739 4323 3944
187420
3574 3805 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2758 9391
287670
2255 4284
216795
1778 2248 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
6301
165743 160172
122008 116442
151244
7767 7767 3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
59762
9866
138565
63421 63817
145486
11886 16255
3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Car Car-ir
122137
109606
122092 122500 122514
106350
121940 122072
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
25018 22604 24216
122500
55885
106350
121940 122530
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Labman Labman-ir
19128
263409
304244
34664 26765
390209
27024 18464 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
7160 16449
377858
7428 38194
497072
21473 12348 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Pedestrian Pedestrian-ir Runner Runner-ir
117051 116737 120324
142077 137828 144675
121509
144107
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
9457
55577
141371 142012
43182
435192
141348 141350
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
116065
92197
30636
129168
27984
100629
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2876
117247
2341
127301
3465
297251
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
9575
300997
60040
11480 15424
287956
19187 2840
21838 17229 2866
241288
9947 4246
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
3638
79438 54283
4987
55103
313867
3023 4995 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
5983 9604
28494
4086 5710
196150
2693 4255 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
6687
66141
48306
16131
63235
116459
14736
74415
18142 10318 10791 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
4160
16714 26378
16071
3345
130137
14534
60554
18142 10318 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
10791
Figure 5 Average location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0993
0311
0986 0993 0993
0108
0993 0993 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
1000 0973
0014
1000 1000
0108
1000 1000 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0907
0005 0000 0056 0070
0009
0986 0986 1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0437
1000
0023
0158 0186
0028
0935
0758
1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0059
0176
0088 0088 0088 0059
0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0824
0382
0941
0088
0471
0059 0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0847
0026 0026
0520
0714
0022
0677 0733
0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0991
0877
0004
0994
0011 0009
0708
0959 0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0068 0054 0078 0049 0054 0044
0083 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0605
0020 0054 0054
0502
0024 0054 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0101
0189
0481
0016
0598
0044
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0937
0005
0904
0019
0943
0033
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0823
0016
0186
0808
0585 0585 0528
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0823
0016
0302
0808 0819
0016
0472
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0585
0105
0331
0561
0318
0000
0585 0558 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0571
0222
0454
0585 0578
0006
0585 0585 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976
0035
0749
0884
0587
0118
0917
0245
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976 0948
0604
0887
1000
0089
0925
0580
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Figure 6 Success rate of the test sequences
targetWhen the target is approaching a tree the backgroundclutters makes most of trackers cause tracking failures thatcan be seen from Figure 2 Our DFCL can overcome thesechallenges and perform well in these sequences
(g) Sequences Labman and Labman-ir the experimentsin Sequences Labman and Labman-ir aim to evaluate the per-formances on tracking under appearance variation rotationscale variation and background clutter In Labman whenthe target man is walking into the laboratory ODFS STCand MOSSE lose the target When the man keeps shaking
and turning around his head at around Frame 400 KCFROT and DSST cause tracking failures Also most trackersachieve better tracking performances in Labman-ir as shownin Figure 2
(h) Sequences Pedestrian and Pedestrian-ir the target inPedestrian and Pedestrian-ir undergoes heavy backgroundclutters and occlusion As shown in Figure 2 other trackersresult in tracking failures in Pedestrian whereas our trackershows satisfying performances in terms of both accuracyand robustnessThe efficient infrared features extracted from
Mathematical Problems in Engineering 9
DFCLDCL MOSSE
Average location error (pixel) Success rate
011 001 006
059
000 012
002 004 004
052
019 017
063
021
073
039 053
037
100 100
082 082
057
099 100 099 099
18742 15124
10635
2078
31387
11646
39021
10063 14467
1853 6166 8940
1786
9545 4990 5658
2692 9228
340 315 1899 652 495 723 1013 138 284
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Figure 7 Average location error (pixel) and success rate of DCL DFCL and MOSSE on the test sequences
Pedestrian-ir ensure the tracking successes of Struck STCand DFCL as can be seen from Figures 2ndash4
(i) Sequences Runner and Runner-ir Runner and Runner-ir contain examples of heavy occlusion abrupt movementand scale variation The target running man is occludedby lampposts trees stone tablet and bushes many timesresulting in tracking failures ofmost trackers Also the abruptmovement and scale variation cause many trackers to driftaway the target in both Runner and Runner-ir as shown inFigure 2 Once again ourDFCL is able to overcome the aboveproblems and achieve good performances
Figures 5 and 6 are included here to demonstrate quan-titatively the performances on average location error (pixel)and success rate The success rate is defined as the numberof times success is achieved in the whole tracking processby considering one frame as a success if the overlapping rateexceeds 05 [33] A smaller average location error and a largersuccess rate indicate increased accuracy and robustnessFigures 5 and 6 show that DFCL performs satisfying most ofthe tracking sequences
To validate the effectiveness of the discriminative filterselection model of DFCL we compare the tracker DCL(the proposed DFCL without the fusion learning model)with DFCL and the original DCF tracker MOSSE on visiblesequencesThe performances shown in Figure 7 demonstratethe efficiency of the discriminative filter selection modelespecially in the sequences with background clutters ieSequences BikerHotkettle Inglassandmobile and Pedestrian
5 Conclusion
Discriminative correlation filter- (DCF-) based trackers havethe advantage of being computationally efficient and morerobust than most of the other state-of-the-art trackers inchallenging tracking tasks thereby making them especiallysuitable for a variety of real-time challenging applications
Howevermost of theDCF-based trackers suffer low accuracydue to the lack of diversity information extracted from asingle type of spectral image (visible spectrum) Fusion ofvisible and infrared sensors one of the typical multisensorcooperation provides complementarily useful features andconsistently helps recognize the target from the backgroundefficiently in visual tracking For the above reasons this paperproposes a discriminative fusion correlation learning modelto improve DCF-based tracking performance by combiningmultiple features from visible and infrared imaging sensorsThe proposed fusion learning filters are obtained via latefusion with early estimation in which the performances ofthe filters are weighted to improve the flexibility of fusionMoreover the proposed discriminative filter selection modelconsiders the surrounding background information in orderto increase the discriminability of the template filters soas to improve model learning Numerous real-world videosequences were used to test DFCL and other state-of-the-art algorithms and here we only selected representativevideos for presentation Experimental results demonstratedthat DFCL is highly accurate and robust
Data Availability
The data used to support the findings of this study weresupplied by China University of Mining and Technologyunder license and so cannot be made freely available
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
This work is supported by the Natural Science Foun-dation of Jiangsu Province (BK20180640 BK20150204)
10 Mathematical Problems in Engineering
Research Development Programme of Jiangsu Province(BE2015040) the State Key Research Development Program(2016YFC0801403) and the National Natural Science Foun-dation of China (51504214 51504255 51734009 61771417 and61873246)
References
[1] S A Wibowo H Lee E K Kim and S Kim ldquoVisual trackingbased on complementary learners with distractor handlingrdquoMathematical Problems in Engineering vol 2017 Article ID5295601 13 pages 2017
[2] T Zhou M Zhu D Zeng and H Yang ldquoScale adaptive kernel-ized correlation filter tracker with feature fusionrdquoMathematicalProblems in Engineering vol 2017 Article ID 1605959 8 pages2017
[3] C Wang H K Galoogahi C Lin and S Lucey ldquoDeep-LK forefficient adaptive object trackingrdquo Computer Vision and PatternRecognition 2017
[4] Z Chi H Li H Lu and M-H Yang ldquoDual deep network forvisual trackingrdquo IEEE Transactions on Image Processing vol 26no 4 pp 2005ndash2015 2017
[5] Y Yang W Hu Y Xie W Zhang and T Zhang ldquoTemporalrestricted visual tracking via reverse-low-rank sparse learningrdquoIEEE Transactions on Cybernetics vol 47 no 2 pp 485ndash4982017
[6] Y Sui G Wang and L Zhang ldquoCorrelation filter learningtoward peak strength for visual trackingrdquo IEEE Transactions onCybernetics vol 48 no 4 pp 1290ndash1303 2018
[7] K Zhang X Li H Song Q Liu and W Lian ldquoVisual track-ing using spatio-temporally nonlocally regularized correlationfilterrdquo Pattern Recognition vol 83 pp 185ndash195 2018
[8] Y Qi L Qin J Zhang S Zhang Q Huang and M-H YangldquoStructure-aware local sparse coding for visual trackingrdquo IEEETransactions on Image Processing vol 27 no 8 pp 3857ndash38692018
[9] B Bai B Zhong G Ouyang et al ldquoKernel correlation filtersfor visual tracking with adaptive fusion of heterogeneous cuesrdquoNeurocomputing vol 286 pp 109ndash120 2018
[10] Y Liu F Yang C Zhong Y Tao B Dai and M Yin ldquoVisualtracking via salient feature extraction and sparse collaborativemodelrdquo AEU - International Journal of Electronics and Commu-nications vol 87 pp 134ndash143 2018
[11] X Yun Y Sun S Wang Y Shi and N Lu ldquoMulti-layerconvolutional network-based visual tracking via importantregion selectionrdquo Neurocomputing vol 315 pp 145ndash156 2018
[12] X Lan S Zhang and P C Yuen ldquoRobust joint discriminativefeature learning for visual trackingrdquo in Proceedings of the 25thInternational Joint Conference on Artificial Intelligence IJCAI2016 pp 3403ndash3410 AAAI Press New York NY USA July2016
[13] D S Bolme J R Beveridge B A Draper and Y M LuildquoVisual object tracking using adaptive correlation filtersrdquo inProceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo10) pp 2544ndash2550 IEEE SanFrancisco Calif USA June 2010
[14] M Danelljan G Hager F Shahbaz Khan and M FelsbergldquoAccurate scale estimation for robust visual trackingrdquo in Pro-ceedings of the British Machine Vision Conference pp 651-6511BMVA Press Nottingham UK 2014
[15] M Danelljan G Hager F S Khan and M Felsberg ldquoDis-criminative scale space trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 39 no 8 pp 1561ndash15752017
[16] H K Galoogahi A Fagg and S Lucey ldquoLearning background-aware correlation filters for visual trackingrdquo inProceedings of the16th IEEE International Conference on Computer Vision ICCV2017 pp 1144ndash1152 IEEE Istanbul Turkey 2018
[17] Y SongCMa LGong J Zhang RWH Lau andM-HYangldquoCrest convolutional residual learning for visual trackingrdquoin Proceedings of the 16th IEEE International Conference onComputer Vision ICCV 2017 pp 2574ndash2583 IEEEVenice ItalyOctober 2017
[18] J Johnander M Danelljan F Khan andM Felsberg ldquoDCCOtowards deformable continuous convolution operators forvisual trackingrdquo in Proceedings of the International Conferenceon Computer Analysis of Images and Patterns pp 55ndash67Springer Ystad Sweden 2017
[19] Y Wu E Blasch G Chen L Bai and H Ling ldquoMultiplesource data fusion via sparse representation for robust visualtrackingrdquo in Proceedings of the 14th International Conference onInformation Fusion Fusion 2011 IEEE Chicago IL USA July2011
[20] B Uzkent A Rangnekar and M J Hoffman ldquoAerial vehicletracking by adaptive fusion of hyperspectral likelihood mapsrdquoin Proceedings of the 30th IEEE Conference on Computer Visionand Pattern RecognitionWorkshops CVPRW 2017 pp 233ndash242IEEE Honolulu HI USA July 2017
[21] S Chan X Zhou and S Chen ldquoRobust adaptive fusion trackingbased on complex cells and keypointsrdquo IEEE Access vol 5 pp20985ndash21001 2017
[22] M K Rapuru S Kakanuru P M Venugopal D Mishra and GR Subrahmanyam ldquoCorrelation-based tracker-level fusion forrobust visual trackingrdquo IEEE Transactions on Image Processingvol 26 no 10 pp 4832ndash4842 2017
[23] X Yun Z Jing G Xiao B Jin and C Zhang ldquoA compressivetracking based on time-space Kalman fusion modelrdquo ScienceChina Information Sciences vol 59 no 1 pp 1ndash15 2016
[24] H P Liu and F C Sun ldquoFusion tracking in color and infraredimages using joint sparse representationrdquo Science China Infor-mation Sciences vol 55 no 3 pp 590ndash599 2012
[25] C Li H Cheng S Hu X Liu J Tang and L Lin ldquoLearningcollaborative sparse representation for grayscale-thermal track-ingrdquo IEEE Transactions on Image Processing vol 25 no 12 pp5743ndash5756 2016
[26] S Mangale and M Khambete ldquoCamouflaged target detectionand tracking using thermal infrared and visible spectrum imag-ingrdquo in Automatic Diagnosis of Breast Cancer using Thermo-graphic Color Analysis and SVM Classifier vol 530 of Advancesin Intelligent Systems and Computing pp 193ndash207 SpringerInternational Publishing 2016
[27] X Yun Z Jing and B Jin ldquoVisible and infrared tracking basedon multi-view multi-kernel fusion modelrdquo Optical Review vol23 no 2 pp 244ndash253 2016
[28] L Zhang A Gonzalez-Garcia J van de Weijer M Danelljanand F S Khan ldquoSynthetic data generation for end-to-end ther-mal infrared trackingrdquo IEEE Transactions on Image Processingvol 28 no 4 pp 1837ndash1850 2019
[29] X Lan M Ye S Zhang and P C Yuen ldquoRobust collaborativediscriminative learning for rgb-infrared trackingrdquo in Proceed-ings of the 32th AAAI Conference on Artificial Intelligence pp7008ndash7015 AAAI New Orleans LA USA 2018
Mathematical Problems in Engineering 11
[30] C Li X Wu N Zhao X Cao and J Tang ldquoFusing two-streamconvolutional neural networks for RGB-T object trackingrdquoNeurocomputing vol 281 pp 78ndash85 2018
[31] JWagner V FischerMHerman and S Behnke ldquoMultispectralpedestrian detection using deep fusion convolutional neuralnetworksrdquo in Proceedings of the 24th European Symposium onArtificial Neural Networks pp 1ndash6 Bruges Belgium 2016
[32] S Hare A Saffari and P H S Torr ldquoStruck structured outputtracking with kernelsrdquo in Proceedings of the IEEE InternationalConference on Computer Vision pp 263ndash270 IEEE ProvidenceRI USA 2012
[33] K Zhang L Zhang andM-H Yang ldquoReal-time object trackingvia online discriminative feature selectionrdquo IEEE Transactionson Image Processing vol 22 no 12 pp 4664ndash4677 2013
[34] K Zhang L Zhang M Yang and D Zhang ldquoFast trackingvia spatio- temporal context learningrdquo Computer Vision andPattern Recognition 2013
[35] J F Henriques R Caseiro P Martins and J Batista ldquoHigh-speed tracking with kernelized correlation filtersrdquo IEEE Trans-actions on Pattern Analysis and Machine Intelligence vol 37 no3 pp 583ndash596 2015
[36] X Dong J Shen D Yu W Wang J Liu and H HuangldquoOcclusion-aware real-time object trackingrdquo IEEE Transactionson Multimedia vol 19 no 4 pp 763ndash771 2017
[37] C O Conaire N E OrsquoConnor and A Smeaton ldquoThermo-visual feature fusion for object tracking using multiple spa-tiogram trackersrdquo Machine Vision amp Applications vol 19 no5-6 pp 483ndash494 2008
[38] INO ldquoInorsquos video analytics datasetrdquo httpswwwinocaenvideo-analytics-dataset
[39] C Li X Liang Y Lu Z Nan and T Jin ldquoRGB-T objecttracking benchmark and baselinerdquo IEEE Transactions on ImageProcessing 2018
[40] C O Conaire N E OrsquoConnor and A F Smeaton ldquoDetectoradaptation by maximising agreement between independentdata sourcesrdquo in Proceedings of the 2007 IEEE ComputerSociety Conference on Computer Vision and Pattern RecognitionCVPRrsquo07 pp 1ndash6 IEEE Minneapolis MN USA June 2007
[41] J W Davis and V Sharma ldquoBackground-subtraction usingcontour-based fusion of thermal and visible imageryrdquoComputerVision and Image Understanding vol 106 no 2-3 pp 162ndash1822007
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom
8 Mathematical Problems in Engineering
10048
96602
171062
9467 27351 20776 17357
3462 21838 17229
2866
241288
9947 6518
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Biker Biker-ir Campus Campus-ir
4536
114901
6739 4323 3944
187420
3574 3805 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2758 9391
287670
2255 4284
216795
1778 2248 8484 6004 3398
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
6301
165743 160172
122008 116442
151244
7767 7767 3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
59762
9866
138565
63421 63817
145486
11886 16255
3823 11922
3151
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Car Car-ir
122137
109606
122092 122500 122514
106350
121940 122072
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
25018 22604 24216
122500
55885
106350
121940 122530
13930 15595 18990
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Labman Labman-ir
19128
263409
304244
34664 26765
390209
27024 18464 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
7160 16449
377858
7428 38194
497072
21473 12348 12792 16786 10133
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Pedestrian Pedestrian-ir Runner Runner-ir
117051 116737 120324
142077 137828 144675
121509
144107
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
9457
55577
141371 142012
43182
435192
141348 141350
3720 3928 2842
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
116065
92197
30636
129168
27984
100629
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
2876
117247
2341
127301
3465
297251
28568
61259
90064
139723
1375
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
9575
300997
60040
11480 15424
287956
19187 2840
21838 17229 2866
241288
9947 4246
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
3638
79438 54283
4987
55103
313867
3023 4995 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
5983 9604
28494
4086 5710
196150
2693 4255 6247 5758
90022
4470
91001
4947
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
6687
66141
48306
16131
63235
116459
14736
74415
18142 10318 10791 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
4160
16714 26378
16071
3345
130137
14534
60554
18142 10318 12384
39102
7233
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
10791
Figure 5 Average location error (pixel) of the test sequences
Biker Biker-ir Campus Campus-ir Car Car-ir
Labman Labman-ir Pedestrian Pedestrian-ir Runner Runner-ir
Crossroad Crossroad-ir Hotkettle Hotkettle-ir Inglassandmobile Inglassandmobile-ir
0993
0311
0986 0993 0993
0108
0993 0993 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
1000 0973
0014
1000 1000
0108
1000 1000 0966 0986 1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0907
0005 0000 0056 0070
0009
0986 0986 1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0437
1000
0023
0158 0186
0028
0935
0758
1000
0874
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0059
0176
0088 0088 0088 0059
0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0824
0382
0941
0088
0471
0059 0088 0088
0971
0824 0824
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0847
0026 0026
0520
0714
0022
0677 0733
0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0991
0877
0004
0994
0011 0009
0708
0959 0998
0817
1000
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0068 0054 0078 0049 0054 0044
0083 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0605
0020 0054 0054
0502
0024 0054 0054
0946 0922 0985
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0101
0189
0481
0016
0598
0044
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0937
0005
0904
0019
0943
0033
0598
0232
0142
0003
0995
Struck
OD
FS
STC
KCF
ROT
MO
SSE
DSST
fDSST
TSKF
MVM
KF
DFCL
0823
0016
0186
0808
0585 0585 0528
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0823
0016
0302
0808 0819
0016
0472
0821
0571
0476
0823
0113
0793 0823
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0585
0105
0331
0561
0318
0000
0585 0558 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0571
0222
0454
0585 0578
0006
0585 0585 0544 0559
0128
0538
0158
0566
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976
0035
0749
0884
0587
0118
0917
0245
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
0976 0948
0604
0887
1000
0089
0925
0580
0955 0981
0825 0782
0649
0987
StruckO
DFS
STCKCFRO
TM
OSSE
DSST
fDSST
TSKFM
VMKF
L1-PFJSRCSRD
FCL
Figure 6 Success rate of the test sequences
targetWhen the target is approaching a tree the backgroundclutters makes most of trackers cause tracking failures thatcan be seen from Figure 2 Our DFCL can overcome thesechallenges and perform well in these sequences
(g) Sequences Labman and Labman-ir the experimentsin Sequences Labman and Labman-ir aim to evaluate the per-formances on tracking under appearance variation rotationscale variation and background clutter In Labman whenthe target man is walking into the laboratory ODFS STCand MOSSE lose the target When the man keeps shaking
and turning around his head at around Frame 400 KCFROT and DSST cause tracking failures Also most trackersachieve better tracking performances in Labman-ir as shownin Figure 2
(h) Sequences Pedestrian and Pedestrian-ir the target inPedestrian and Pedestrian-ir undergoes heavy backgroundclutters and occlusion As shown in Figure 2 other trackersresult in tracking failures in Pedestrian whereas our trackershows satisfying performances in terms of both accuracyand robustnessThe efficient infrared features extracted from
Mathematical Problems in Engineering 9
DFCLDCL MOSSE
Average location error (pixel) Success rate
011 001 006
059
000 012
002 004 004
052
019 017
063
021
073
039 053
037
100 100
082 082
057
099 100 099 099
18742 15124
10635
2078
31387
11646
39021
10063 14467
1853 6166 8940
1786
9545 4990 5658
2692 9228
340 315 1899 652 495 723 1013 138 284
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Figure 7 Average location error (pixel) and success rate of DCL DFCL and MOSSE on the test sequences
Pedestrian-ir ensure the tracking successes of Struck STCand DFCL as can be seen from Figures 2ndash4
(i) Sequences Runner and Runner-ir Runner and Runner-ir contain examples of heavy occlusion abrupt movementand scale variation The target running man is occludedby lampposts trees stone tablet and bushes many timesresulting in tracking failures ofmost trackers Also the abruptmovement and scale variation cause many trackers to driftaway the target in both Runner and Runner-ir as shown inFigure 2 Once again ourDFCL is able to overcome the aboveproblems and achieve good performances
Figures 5 and 6 are included here to demonstrate quan-titatively the performances on average location error (pixel)and success rate The success rate is defined as the numberof times success is achieved in the whole tracking processby considering one frame as a success if the overlapping rateexceeds 05 [33] A smaller average location error and a largersuccess rate indicate increased accuracy and robustnessFigures 5 and 6 show that DFCL performs satisfying most ofthe tracking sequences
To validate the effectiveness of the discriminative filterselection model of DFCL we compare the tracker DCL(the proposed DFCL without the fusion learning model)with DFCL and the original DCF tracker MOSSE on visiblesequencesThe performances shown in Figure 7 demonstratethe efficiency of the discriminative filter selection modelespecially in the sequences with background clutters ieSequences BikerHotkettle Inglassandmobile and Pedestrian
5 Conclusion
Discriminative correlation filter- (DCF-) based trackers havethe advantage of being computationally efficient and morerobust than most of the other state-of-the-art trackers inchallenging tracking tasks thereby making them especiallysuitable for a variety of real-time challenging applications
Howevermost of theDCF-based trackers suffer low accuracydue to the lack of diversity information extracted from asingle type of spectral image (visible spectrum) Fusion ofvisible and infrared sensors one of the typical multisensorcooperation provides complementarily useful features andconsistently helps recognize the target from the backgroundefficiently in visual tracking For the above reasons this paperproposes a discriminative fusion correlation learning modelto improve DCF-based tracking performance by combiningmultiple features from visible and infrared imaging sensorsThe proposed fusion learning filters are obtained via latefusion with early estimation in which the performances ofthe filters are weighted to improve the flexibility of fusionMoreover the proposed discriminative filter selection modelconsiders the surrounding background information in orderto increase the discriminability of the template filters soas to improve model learning Numerous real-world videosequences were used to test DFCL and other state-of-the-art algorithms and here we only selected representativevideos for presentation Experimental results demonstratedthat DFCL is highly accurate and robust
Data Availability
The data used to support the findings of this study weresupplied by China University of Mining and Technologyunder license and so cannot be made freely available
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
This work is supported by the Natural Science Foun-dation of Jiangsu Province (BK20180640 BK20150204)
10 Mathematical Problems in Engineering
Research Development Programme of Jiangsu Province(BE2015040) the State Key Research Development Program(2016YFC0801403) and the National Natural Science Foun-dation of China (51504214 51504255 51734009 61771417 and61873246)
References
[1] S A Wibowo H Lee E K Kim and S Kim ldquoVisual trackingbased on complementary learners with distractor handlingrdquoMathematical Problems in Engineering vol 2017 Article ID5295601 13 pages 2017
[2] T Zhou M Zhu D Zeng and H Yang ldquoScale adaptive kernel-ized correlation filter tracker with feature fusionrdquoMathematicalProblems in Engineering vol 2017 Article ID 1605959 8 pages2017
[3] C Wang H K Galoogahi C Lin and S Lucey ldquoDeep-LK forefficient adaptive object trackingrdquo Computer Vision and PatternRecognition 2017
[4] Z Chi H Li H Lu and M-H Yang ldquoDual deep network forvisual trackingrdquo IEEE Transactions on Image Processing vol 26no 4 pp 2005ndash2015 2017
[5] Y Yang W Hu Y Xie W Zhang and T Zhang ldquoTemporalrestricted visual tracking via reverse-low-rank sparse learningrdquoIEEE Transactions on Cybernetics vol 47 no 2 pp 485ndash4982017
[6] Y Sui G Wang and L Zhang ldquoCorrelation filter learningtoward peak strength for visual trackingrdquo IEEE Transactions onCybernetics vol 48 no 4 pp 1290ndash1303 2018
[7] K Zhang X Li H Song Q Liu and W Lian ldquoVisual track-ing using spatio-temporally nonlocally regularized correlationfilterrdquo Pattern Recognition vol 83 pp 185ndash195 2018
[8] Y Qi L Qin J Zhang S Zhang Q Huang and M-H YangldquoStructure-aware local sparse coding for visual trackingrdquo IEEETransactions on Image Processing vol 27 no 8 pp 3857ndash38692018
[9] B Bai B Zhong G Ouyang et al ldquoKernel correlation filtersfor visual tracking with adaptive fusion of heterogeneous cuesrdquoNeurocomputing vol 286 pp 109ndash120 2018
[10] Y Liu F Yang C Zhong Y Tao B Dai and M Yin ldquoVisualtracking via salient feature extraction and sparse collaborativemodelrdquo AEU - International Journal of Electronics and Commu-nications vol 87 pp 134ndash143 2018
[11] X Yun Y Sun S Wang Y Shi and N Lu ldquoMulti-layerconvolutional network-based visual tracking via importantregion selectionrdquo Neurocomputing vol 315 pp 145ndash156 2018
[12] X Lan S Zhang and P C Yuen ldquoRobust joint discriminativefeature learning for visual trackingrdquo in Proceedings of the 25thInternational Joint Conference on Artificial Intelligence IJCAI2016 pp 3403ndash3410 AAAI Press New York NY USA July2016
[13] D S Bolme J R Beveridge B A Draper and Y M LuildquoVisual object tracking using adaptive correlation filtersrdquo inProceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo10) pp 2544ndash2550 IEEE SanFrancisco Calif USA June 2010
[14] M Danelljan G Hager F Shahbaz Khan and M FelsbergldquoAccurate scale estimation for robust visual trackingrdquo in Pro-ceedings of the British Machine Vision Conference pp 651-6511BMVA Press Nottingham UK 2014
[15] M Danelljan G Hager F S Khan and M Felsberg ldquoDis-criminative scale space trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 39 no 8 pp 1561ndash15752017
[16] H K Galoogahi A Fagg and S Lucey ldquoLearning background-aware correlation filters for visual trackingrdquo inProceedings of the16th IEEE International Conference on Computer Vision ICCV2017 pp 1144ndash1152 IEEE Istanbul Turkey 2018
[17] Y SongCMa LGong J Zhang RWH Lau andM-HYangldquoCrest convolutional residual learning for visual trackingrdquoin Proceedings of the 16th IEEE International Conference onComputer Vision ICCV 2017 pp 2574ndash2583 IEEEVenice ItalyOctober 2017
[18] J Johnander M Danelljan F Khan andM Felsberg ldquoDCCOtowards deformable continuous convolution operators forvisual trackingrdquo in Proceedings of the International Conferenceon Computer Analysis of Images and Patterns pp 55ndash67Springer Ystad Sweden 2017
[19] Y Wu E Blasch G Chen L Bai and H Ling ldquoMultiplesource data fusion via sparse representation for robust visualtrackingrdquo in Proceedings of the 14th International Conference onInformation Fusion Fusion 2011 IEEE Chicago IL USA July2011
[20] B Uzkent A Rangnekar and M J Hoffman ldquoAerial vehicletracking by adaptive fusion of hyperspectral likelihood mapsrdquoin Proceedings of the 30th IEEE Conference on Computer Visionand Pattern RecognitionWorkshops CVPRW 2017 pp 233ndash242IEEE Honolulu HI USA July 2017
[21] S Chan X Zhou and S Chen ldquoRobust adaptive fusion trackingbased on complex cells and keypointsrdquo IEEE Access vol 5 pp20985ndash21001 2017
[22] M K Rapuru S Kakanuru P M Venugopal D Mishra and GR Subrahmanyam ldquoCorrelation-based tracker-level fusion forrobust visual trackingrdquo IEEE Transactions on Image Processingvol 26 no 10 pp 4832ndash4842 2017
[23] X Yun Z Jing G Xiao B Jin and C Zhang ldquoA compressivetracking based on time-space Kalman fusion modelrdquo ScienceChina Information Sciences vol 59 no 1 pp 1ndash15 2016
[24] H P Liu and F C Sun ldquoFusion tracking in color and infraredimages using joint sparse representationrdquo Science China Infor-mation Sciences vol 55 no 3 pp 590ndash599 2012
[25] C Li H Cheng S Hu X Liu J Tang and L Lin ldquoLearningcollaborative sparse representation for grayscale-thermal track-ingrdquo IEEE Transactions on Image Processing vol 25 no 12 pp5743ndash5756 2016
[26] S Mangale and M Khambete ldquoCamouflaged target detectionand tracking using thermal infrared and visible spectrum imag-ingrdquo in Automatic Diagnosis of Breast Cancer using Thermo-graphic Color Analysis and SVM Classifier vol 530 of Advancesin Intelligent Systems and Computing pp 193ndash207 SpringerInternational Publishing 2016
[27] X Yun Z Jing and B Jin ldquoVisible and infrared tracking basedon multi-view multi-kernel fusion modelrdquo Optical Review vol23 no 2 pp 244ndash253 2016
[28] L Zhang A Gonzalez-Garcia J van de Weijer M Danelljanand F S Khan ldquoSynthetic data generation for end-to-end ther-mal infrared trackingrdquo IEEE Transactions on Image Processingvol 28 no 4 pp 1837ndash1850 2019
[29] X Lan M Ye S Zhang and P C Yuen ldquoRobust collaborativediscriminative learning for rgb-infrared trackingrdquo in Proceed-ings of the 32th AAAI Conference on Artificial Intelligence pp7008ndash7015 AAAI New Orleans LA USA 2018
Mathematical Problems in Engineering 11
[30] C Li X Wu N Zhao X Cao and J Tang ldquoFusing two-streamconvolutional neural networks for RGB-T object trackingrdquoNeurocomputing vol 281 pp 78ndash85 2018
[31] JWagner V FischerMHerman and S Behnke ldquoMultispectralpedestrian detection using deep fusion convolutional neuralnetworksrdquo in Proceedings of the 24th European Symposium onArtificial Neural Networks pp 1ndash6 Bruges Belgium 2016
[32] S Hare A Saffari and P H S Torr ldquoStruck structured outputtracking with kernelsrdquo in Proceedings of the IEEE InternationalConference on Computer Vision pp 263ndash270 IEEE ProvidenceRI USA 2012
[33] K Zhang L Zhang andM-H Yang ldquoReal-time object trackingvia online discriminative feature selectionrdquo IEEE Transactionson Image Processing vol 22 no 12 pp 4664ndash4677 2013
[34] K Zhang L Zhang M Yang and D Zhang ldquoFast trackingvia spatio- temporal context learningrdquo Computer Vision andPattern Recognition 2013
[35] J F Henriques R Caseiro P Martins and J Batista ldquoHigh-speed tracking with kernelized correlation filtersrdquo IEEE Trans-actions on Pattern Analysis and Machine Intelligence vol 37 no3 pp 583ndash596 2015
[36] X Dong J Shen D Yu W Wang J Liu and H HuangldquoOcclusion-aware real-time object trackingrdquo IEEE Transactionson Multimedia vol 19 no 4 pp 763ndash771 2017
[37] C O Conaire N E OrsquoConnor and A Smeaton ldquoThermo-visual feature fusion for object tracking using multiple spa-tiogram trackersrdquo Machine Vision amp Applications vol 19 no5-6 pp 483ndash494 2008
[38] INO ldquoInorsquos video analytics datasetrdquo httpswwwinocaenvideo-analytics-dataset
[39] C Li X Liang Y Lu Z Nan and T Jin ldquoRGB-T objecttracking benchmark and baselinerdquo IEEE Transactions on ImageProcessing 2018
[40] C O Conaire N E OrsquoConnor and A F Smeaton ldquoDetectoradaptation by maximising agreement between independentdata sourcesrdquo in Proceedings of the 2007 IEEE ComputerSociety Conference on Computer Vision and Pattern RecognitionCVPRrsquo07 pp 1ndash6 IEEE Minneapolis MN USA June 2007
[41] J W Davis and V Sharma ldquoBackground-subtraction usingcontour-based fusion of thermal and visible imageryrdquoComputerVision and Image Understanding vol 106 no 2-3 pp 162ndash1822007
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom
Mathematical Problems in Engineering 9
DFCLDCL MOSSE
Average location error (pixel) Success rate
011 001 006
059
000 012
002 004 004
052
019 017
063
021
073
039 053
037
100 100
082 082
057
099 100 099 099
18742 15124
10635
2078
31387
11646
39021
10063 14467
1853 6166 8940
1786
9545 4990 5658
2692 9228
340 315 1899 652 495 723 1013 138 284
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Bike
r
Cam
pus
Car
Cros
sroa
d
Hot
kettl
e
Ingl
assa
ndm
obile
Labm
an
Pede
stria
n
Runn
er
Figure 7 Average location error (pixel) and success rate of DCL DFCL and MOSSE on the test sequences
Pedestrian-ir ensure the tracking successes of Struck STCand DFCL as can be seen from Figures 2ndash4
(i) Sequences Runner and Runner-ir Runner and Runner-ir contain examples of heavy occlusion abrupt movementand scale variation The target running man is occludedby lampposts trees stone tablet and bushes many timesresulting in tracking failures ofmost trackers Also the abruptmovement and scale variation cause many trackers to driftaway the target in both Runner and Runner-ir as shown inFigure 2 Once again ourDFCL is able to overcome the aboveproblems and achieve good performances
Figures 5 and 6 are included here to demonstrate quan-titatively the performances on average location error (pixel)and success rate The success rate is defined as the numberof times success is achieved in the whole tracking processby considering one frame as a success if the overlapping rateexceeds 05 [33] A smaller average location error and a largersuccess rate indicate increased accuracy and robustnessFigures 5 and 6 show that DFCL performs satisfying most ofthe tracking sequences
To validate the effectiveness of the discriminative filterselection model of DFCL we compare the tracker DCL(the proposed DFCL without the fusion learning model)with DFCL and the original DCF tracker MOSSE on visiblesequencesThe performances shown in Figure 7 demonstratethe efficiency of the discriminative filter selection modelespecially in the sequences with background clutters ieSequences BikerHotkettle Inglassandmobile and Pedestrian
5 Conclusion
Discriminative correlation filter- (DCF-) based trackers havethe advantage of being computationally efficient and morerobust than most of the other state-of-the-art trackers inchallenging tracking tasks thereby making them especiallysuitable for a variety of real-time challenging applications
Howevermost of theDCF-based trackers suffer low accuracydue to the lack of diversity information extracted from asingle type of spectral image (visible spectrum) Fusion ofvisible and infrared sensors one of the typical multisensorcooperation provides complementarily useful features andconsistently helps recognize the target from the backgroundefficiently in visual tracking For the above reasons this paperproposes a discriminative fusion correlation learning modelto improve DCF-based tracking performance by combiningmultiple features from visible and infrared imaging sensorsThe proposed fusion learning filters are obtained via latefusion with early estimation in which the performances ofthe filters are weighted to improve the flexibility of fusionMoreover the proposed discriminative filter selection modelconsiders the surrounding background information in orderto increase the discriminability of the template filters soas to improve model learning Numerous real-world videosequences were used to test DFCL and other state-of-the-art algorithms and here we only selected representativevideos for presentation Experimental results demonstratedthat DFCL is highly accurate and robust
Data Availability
The data used to support the findings of this study weresupplied by China University of Mining and Technologyunder license and so cannot be made freely available
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
This work is supported by the Natural Science Foun-dation of Jiangsu Province (BK20180640 BK20150204)
10 Mathematical Problems in Engineering
Research Development Programme of Jiangsu Province(BE2015040) the State Key Research Development Program(2016YFC0801403) and the National Natural Science Foun-dation of China (51504214 51504255 51734009 61771417 and61873246)
References
[1] S A Wibowo H Lee E K Kim and S Kim ldquoVisual trackingbased on complementary learners with distractor handlingrdquoMathematical Problems in Engineering vol 2017 Article ID5295601 13 pages 2017
[2] T Zhou M Zhu D Zeng and H Yang ldquoScale adaptive kernel-ized correlation filter tracker with feature fusionrdquoMathematicalProblems in Engineering vol 2017 Article ID 1605959 8 pages2017
[3] C Wang H K Galoogahi C Lin and S Lucey ldquoDeep-LK forefficient adaptive object trackingrdquo Computer Vision and PatternRecognition 2017
[4] Z Chi H Li H Lu and M-H Yang ldquoDual deep network forvisual trackingrdquo IEEE Transactions on Image Processing vol 26no 4 pp 2005ndash2015 2017
[5] Y Yang W Hu Y Xie W Zhang and T Zhang ldquoTemporalrestricted visual tracking via reverse-low-rank sparse learningrdquoIEEE Transactions on Cybernetics vol 47 no 2 pp 485ndash4982017
[6] Y Sui G Wang and L Zhang ldquoCorrelation filter learningtoward peak strength for visual trackingrdquo IEEE Transactions onCybernetics vol 48 no 4 pp 1290ndash1303 2018
[7] K Zhang X Li H Song Q Liu and W Lian ldquoVisual track-ing using spatio-temporally nonlocally regularized correlationfilterrdquo Pattern Recognition vol 83 pp 185ndash195 2018
[8] Y Qi L Qin J Zhang S Zhang Q Huang and M-H YangldquoStructure-aware local sparse coding for visual trackingrdquo IEEETransactions on Image Processing vol 27 no 8 pp 3857ndash38692018
[9] B Bai B Zhong G Ouyang et al ldquoKernel correlation filtersfor visual tracking with adaptive fusion of heterogeneous cuesrdquoNeurocomputing vol 286 pp 109ndash120 2018
[10] Y Liu F Yang C Zhong Y Tao B Dai and M Yin ldquoVisualtracking via salient feature extraction and sparse collaborativemodelrdquo AEU - International Journal of Electronics and Commu-nications vol 87 pp 134ndash143 2018
[11] X Yun Y Sun S Wang Y Shi and N Lu ldquoMulti-layerconvolutional network-based visual tracking via importantregion selectionrdquo Neurocomputing vol 315 pp 145ndash156 2018
[12] X Lan S Zhang and P C Yuen ldquoRobust joint discriminativefeature learning for visual trackingrdquo in Proceedings of the 25thInternational Joint Conference on Artificial Intelligence IJCAI2016 pp 3403ndash3410 AAAI Press New York NY USA July2016
[13] D S Bolme J R Beveridge B A Draper and Y M LuildquoVisual object tracking using adaptive correlation filtersrdquo inProceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo10) pp 2544ndash2550 IEEE SanFrancisco Calif USA June 2010
[14] M Danelljan G Hager F Shahbaz Khan and M FelsbergldquoAccurate scale estimation for robust visual trackingrdquo in Pro-ceedings of the British Machine Vision Conference pp 651-6511BMVA Press Nottingham UK 2014
[15] M Danelljan G Hager F S Khan and M Felsberg ldquoDis-criminative scale space trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 39 no 8 pp 1561ndash15752017
[16] H K Galoogahi A Fagg and S Lucey ldquoLearning background-aware correlation filters for visual trackingrdquo inProceedings of the16th IEEE International Conference on Computer Vision ICCV2017 pp 1144ndash1152 IEEE Istanbul Turkey 2018
[17] Y SongCMa LGong J Zhang RWH Lau andM-HYangldquoCrest convolutional residual learning for visual trackingrdquoin Proceedings of the 16th IEEE International Conference onComputer Vision ICCV 2017 pp 2574ndash2583 IEEEVenice ItalyOctober 2017
[18] J Johnander M Danelljan F Khan andM Felsberg ldquoDCCOtowards deformable continuous convolution operators forvisual trackingrdquo in Proceedings of the International Conferenceon Computer Analysis of Images and Patterns pp 55ndash67Springer Ystad Sweden 2017
[19] Y Wu E Blasch G Chen L Bai and H Ling ldquoMultiplesource data fusion via sparse representation for robust visualtrackingrdquo in Proceedings of the 14th International Conference onInformation Fusion Fusion 2011 IEEE Chicago IL USA July2011
[20] B Uzkent A Rangnekar and M J Hoffman ldquoAerial vehicletracking by adaptive fusion of hyperspectral likelihood mapsrdquoin Proceedings of the 30th IEEE Conference on Computer Visionand Pattern RecognitionWorkshops CVPRW 2017 pp 233ndash242IEEE Honolulu HI USA July 2017
[21] S Chan X Zhou and S Chen ldquoRobust adaptive fusion trackingbased on complex cells and keypointsrdquo IEEE Access vol 5 pp20985ndash21001 2017
[22] M K Rapuru S Kakanuru P M Venugopal D Mishra and GR Subrahmanyam ldquoCorrelation-based tracker-level fusion forrobust visual trackingrdquo IEEE Transactions on Image Processingvol 26 no 10 pp 4832ndash4842 2017
[23] X Yun Z Jing G Xiao B Jin and C Zhang ldquoA compressivetracking based on time-space Kalman fusion modelrdquo ScienceChina Information Sciences vol 59 no 1 pp 1ndash15 2016
[24] H P Liu and F C Sun ldquoFusion tracking in color and infraredimages using joint sparse representationrdquo Science China Infor-mation Sciences vol 55 no 3 pp 590ndash599 2012
[25] C Li H Cheng S Hu X Liu J Tang and L Lin ldquoLearningcollaborative sparse representation for grayscale-thermal track-ingrdquo IEEE Transactions on Image Processing vol 25 no 12 pp5743ndash5756 2016
[26] S Mangale and M Khambete ldquoCamouflaged target detectionand tracking using thermal infrared and visible spectrum imag-ingrdquo in Automatic Diagnosis of Breast Cancer using Thermo-graphic Color Analysis and SVM Classifier vol 530 of Advancesin Intelligent Systems and Computing pp 193ndash207 SpringerInternational Publishing 2016
[27] X Yun Z Jing and B Jin ldquoVisible and infrared tracking basedon multi-view multi-kernel fusion modelrdquo Optical Review vol23 no 2 pp 244ndash253 2016
[28] L Zhang A Gonzalez-Garcia J van de Weijer M Danelljanand F S Khan ldquoSynthetic data generation for end-to-end ther-mal infrared trackingrdquo IEEE Transactions on Image Processingvol 28 no 4 pp 1837ndash1850 2019
[29] X Lan M Ye S Zhang and P C Yuen ldquoRobust collaborativediscriminative learning for rgb-infrared trackingrdquo in Proceed-ings of the 32th AAAI Conference on Artificial Intelligence pp7008ndash7015 AAAI New Orleans LA USA 2018
Mathematical Problems in Engineering 11
[30] C Li X Wu N Zhao X Cao and J Tang ldquoFusing two-streamconvolutional neural networks for RGB-T object trackingrdquoNeurocomputing vol 281 pp 78ndash85 2018
[31] JWagner V FischerMHerman and S Behnke ldquoMultispectralpedestrian detection using deep fusion convolutional neuralnetworksrdquo in Proceedings of the 24th European Symposium onArtificial Neural Networks pp 1ndash6 Bruges Belgium 2016
[32] S Hare A Saffari and P H S Torr ldquoStruck structured outputtracking with kernelsrdquo in Proceedings of the IEEE InternationalConference on Computer Vision pp 263ndash270 IEEE ProvidenceRI USA 2012
[33] K Zhang L Zhang andM-H Yang ldquoReal-time object trackingvia online discriminative feature selectionrdquo IEEE Transactionson Image Processing vol 22 no 12 pp 4664ndash4677 2013
[34] K Zhang L Zhang M Yang and D Zhang ldquoFast trackingvia spatio- temporal context learningrdquo Computer Vision andPattern Recognition 2013
[35] J F Henriques R Caseiro P Martins and J Batista ldquoHigh-speed tracking with kernelized correlation filtersrdquo IEEE Trans-actions on Pattern Analysis and Machine Intelligence vol 37 no3 pp 583ndash596 2015
[36] X Dong J Shen D Yu W Wang J Liu and H HuangldquoOcclusion-aware real-time object trackingrdquo IEEE Transactionson Multimedia vol 19 no 4 pp 763ndash771 2017
[37] C O Conaire N E OrsquoConnor and A Smeaton ldquoThermo-visual feature fusion for object tracking using multiple spa-tiogram trackersrdquo Machine Vision amp Applications vol 19 no5-6 pp 483ndash494 2008
[38] INO ldquoInorsquos video analytics datasetrdquo httpswwwinocaenvideo-analytics-dataset
[39] C Li X Liang Y Lu Z Nan and T Jin ldquoRGB-T objecttracking benchmark and baselinerdquo IEEE Transactions on ImageProcessing 2018
[40] C O Conaire N E OrsquoConnor and A F Smeaton ldquoDetectoradaptation by maximising agreement between independentdata sourcesrdquo in Proceedings of the 2007 IEEE ComputerSociety Conference on Computer Vision and Pattern RecognitionCVPRrsquo07 pp 1ndash6 IEEE Minneapolis MN USA June 2007
[41] J W Davis and V Sharma ldquoBackground-subtraction usingcontour-based fusion of thermal and visible imageryrdquoComputerVision and Image Understanding vol 106 no 2-3 pp 162ndash1822007
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom
10 Mathematical Problems in Engineering
Research Development Programme of Jiangsu Province(BE2015040) the State Key Research Development Program(2016YFC0801403) and the National Natural Science Foun-dation of China (51504214 51504255 51734009 61771417 and61873246)
References
[1] S A Wibowo H Lee E K Kim and S Kim ldquoVisual trackingbased on complementary learners with distractor handlingrdquoMathematical Problems in Engineering vol 2017 Article ID5295601 13 pages 2017
[2] T Zhou M Zhu D Zeng and H Yang ldquoScale adaptive kernel-ized correlation filter tracker with feature fusionrdquoMathematicalProblems in Engineering vol 2017 Article ID 1605959 8 pages2017
[3] C Wang H K Galoogahi C Lin and S Lucey ldquoDeep-LK forefficient adaptive object trackingrdquo Computer Vision and PatternRecognition 2017
[4] Z Chi H Li H Lu and M-H Yang ldquoDual deep network forvisual trackingrdquo IEEE Transactions on Image Processing vol 26no 4 pp 2005ndash2015 2017
[5] Y Yang W Hu Y Xie W Zhang and T Zhang ldquoTemporalrestricted visual tracking via reverse-low-rank sparse learningrdquoIEEE Transactions on Cybernetics vol 47 no 2 pp 485ndash4982017
[6] Y Sui G Wang and L Zhang ldquoCorrelation filter learningtoward peak strength for visual trackingrdquo IEEE Transactions onCybernetics vol 48 no 4 pp 1290ndash1303 2018
[7] K Zhang X Li H Song Q Liu and W Lian ldquoVisual track-ing using spatio-temporally nonlocally regularized correlationfilterrdquo Pattern Recognition vol 83 pp 185ndash195 2018
[8] Y Qi L Qin J Zhang S Zhang Q Huang and M-H YangldquoStructure-aware local sparse coding for visual trackingrdquo IEEETransactions on Image Processing vol 27 no 8 pp 3857ndash38692018
[9] B Bai B Zhong G Ouyang et al ldquoKernel correlation filtersfor visual tracking with adaptive fusion of heterogeneous cuesrdquoNeurocomputing vol 286 pp 109ndash120 2018
[10] Y Liu F Yang C Zhong Y Tao B Dai and M Yin ldquoVisualtracking via salient feature extraction and sparse collaborativemodelrdquo AEU - International Journal of Electronics and Commu-nications vol 87 pp 134ndash143 2018
[11] X Yun Y Sun S Wang Y Shi and N Lu ldquoMulti-layerconvolutional network-based visual tracking via importantregion selectionrdquo Neurocomputing vol 315 pp 145ndash156 2018
[12] X Lan S Zhang and P C Yuen ldquoRobust joint discriminativefeature learning for visual trackingrdquo in Proceedings of the 25thInternational Joint Conference on Artificial Intelligence IJCAI2016 pp 3403ndash3410 AAAI Press New York NY USA July2016
[13] D S Bolme J R Beveridge B A Draper and Y M LuildquoVisual object tracking using adaptive correlation filtersrdquo inProceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo10) pp 2544ndash2550 IEEE SanFrancisco Calif USA June 2010
[14] M Danelljan G Hager F Shahbaz Khan and M FelsbergldquoAccurate scale estimation for robust visual trackingrdquo in Pro-ceedings of the British Machine Vision Conference pp 651-6511BMVA Press Nottingham UK 2014
[15] M Danelljan G Hager F S Khan and M Felsberg ldquoDis-criminative scale space trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 39 no 8 pp 1561ndash15752017
[16] H K Galoogahi A Fagg and S Lucey ldquoLearning background-aware correlation filters for visual trackingrdquo inProceedings of the16th IEEE International Conference on Computer Vision ICCV2017 pp 1144ndash1152 IEEE Istanbul Turkey 2018
[17] Y SongCMa LGong J Zhang RWH Lau andM-HYangldquoCrest convolutional residual learning for visual trackingrdquoin Proceedings of the 16th IEEE International Conference onComputer Vision ICCV 2017 pp 2574ndash2583 IEEEVenice ItalyOctober 2017
[18] J Johnander M Danelljan F Khan andM Felsberg ldquoDCCOtowards deformable continuous convolution operators forvisual trackingrdquo in Proceedings of the International Conferenceon Computer Analysis of Images and Patterns pp 55ndash67Springer Ystad Sweden 2017
[19] Y Wu E Blasch G Chen L Bai and H Ling ldquoMultiplesource data fusion via sparse representation for robust visualtrackingrdquo in Proceedings of the 14th International Conference onInformation Fusion Fusion 2011 IEEE Chicago IL USA July2011
[20] B Uzkent A Rangnekar and M J Hoffman ldquoAerial vehicletracking by adaptive fusion of hyperspectral likelihood mapsrdquoin Proceedings of the 30th IEEE Conference on Computer Visionand Pattern RecognitionWorkshops CVPRW 2017 pp 233ndash242IEEE Honolulu HI USA July 2017
[21] S Chan X Zhou and S Chen ldquoRobust adaptive fusion trackingbased on complex cells and keypointsrdquo IEEE Access vol 5 pp20985ndash21001 2017
[22] M K Rapuru S Kakanuru P M Venugopal D Mishra and GR Subrahmanyam ldquoCorrelation-based tracker-level fusion forrobust visual trackingrdquo IEEE Transactions on Image Processingvol 26 no 10 pp 4832ndash4842 2017
[23] X Yun Z Jing G Xiao B Jin and C Zhang ldquoA compressivetracking based on time-space Kalman fusion modelrdquo ScienceChina Information Sciences vol 59 no 1 pp 1ndash15 2016
[24] H P Liu and F C Sun ldquoFusion tracking in color and infraredimages using joint sparse representationrdquo Science China Infor-mation Sciences vol 55 no 3 pp 590ndash599 2012
[25] C Li H Cheng S Hu X Liu J Tang and L Lin ldquoLearningcollaborative sparse representation for grayscale-thermal track-ingrdquo IEEE Transactions on Image Processing vol 25 no 12 pp5743ndash5756 2016
[26] S Mangale and M Khambete ldquoCamouflaged target detectionand tracking using thermal infrared and visible spectrum imag-ingrdquo in Automatic Diagnosis of Breast Cancer using Thermo-graphic Color Analysis and SVM Classifier vol 530 of Advancesin Intelligent Systems and Computing pp 193ndash207 SpringerInternational Publishing 2016
[27] X Yun Z Jing and B Jin ldquoVisible and infrared tracking basedon multi-view multi-kernel fusion modelrdquo Optical Review vol23 no 2 pp 244ndash253 2016
[28] L Zhang A Gonzalez-Garcia J van de Weijer M Danelljanand F S Khan ldquoSynthetic data generation for end-to-end ther-mal infrared trackingrdquo IEEE Transactions on Image Processingvol 28 no 4 pp 1837ndash1850 2019
[29] X Lan M Ye S Zhang and P C Yuen ldquoRobust collaborativediscriminative learning for rgb-infrared trackingrdquo in Proceed-ings of the 32th AAAI Conference on Artificial Intelligence pp7008ndash7015 AAAI New Orleans LA USA 2018
Mathematical Problems in Engineering 11
[30] C Li X Wu N Zhao X Cao and J Tang ldquoFusing two-streamconvolutional neural networks for RGB-T object trackingrdquoNeurocomputing vol 281 pp 78ndash85 2018
[31] JWagner V FischerMHerman and S Behnke ldquoMultispectralpedestrian detection using deep fusion convolutional neuralnetworksrdquo in Proceedings of the 24th European Symposium onArtificial Neural Networks pp 1ndash6 Bruges Belgium 2016
[32] S Hare A Saffari and P H S Torr ldquoStruck structured outputtracking with kernelsrdquo in Proceedings of the IEEE InternationalConference on Computer Vision pp 263ndash270 IEEE ProvidenceRI USA 2012
[33] K Zhang L Zhang andM-H Yang ldquoReal-time object trackingvia online discriminative feature selectionrdquo IEEE Transactionson Image Processing vol 22 no 12 pp 4664ndash4677 2013
[34] K Zhang L Zhang M Yang and D Zhang ldquoFast trackingvia spatio- temporal context learningrdquo Computer Vision andPattern Recognition 2013
[35] J F Henriques R Caseiro P Martins and J Batista ldquoHigh-speed tracking with kernelized correlation filtersrdquo IEEE Trans-actions on Pattern Analysis and Machine Intelligence vol 37 no3 pp 583ndash596 2015
[36] X Dong J Shen D Yu W Wang J Liu and H HuangldquoOcclusion-aware real-time object trackingrdquo IEEE Transactionson Multimedia vol 19 no 4 pp 763ndash771 2017
[37] C O Conaire N E OrsquoConnor and A Smeaton ldquoThermo-visual feature fusion for object tracking using multiple spa-tiogram trackersrdquo Machine Vision amp Applications vol 19 no5-6 pp 483ndash494 2008
[38] INO ldquoInorsquos video analytics datasetrdquo httpswwwinocaenvideo-analytics-dataset
[39] C Li X Liang Y Lu Z Nan and T Jin ldquoRGB-T objecttracking benchmark and baselinerdquo IEEE Transactions on ImageProcessing 2018
[40] C O Conaire N E OrsquoConnor and A F Smeaton ldquoDetectoradaptation by maximising agreement between independentdata sourcesrdquo in Proceedings of the 2007 IEEE ComputerSociety Conference on Computer Vision and Pattern RecognitionCVPRrsquo07 pp 1ndash6 IEEE Minneapolis MN USA June 2007
[41] J W Davis and V Sharma ldquoBackground-subtraction usingcontour-based fusion of thermal and visible imageryrdquoComputerVision and Image Understanding vol 106 no 2-3 pp 162ndash1822007
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom
Mathematical Problems in Engineering 11
[30] C Li X Wu N Zhao X Cao and J Tang ldquoFusing two-streamconvolutional neural networks for RGB-T object trackingrdquoNeurocomputing vol 281 pp 78ndash85 2018
[31] JWagner V FischerMHerman and S Behnke ldquoMultispectralpedestrian detection using deep fusion convolutional neuralnetworksrdquo in Proceedings of the 24th European Symposium onArtificial Neural Networks pp 1ndash6 Bruges Belgium 2016
[32] S Hare A Saffari and P H S Torr ldquoStruck structured outputtracking with kernelsrdquo in Proceedings of the IEEE InternationalConference on Computer Vision pp 263ndash270 IEEE ProvidenceRI USA 2012
[33] K Zhang L Zhang andM-H Yang ldquoReal-time object trackingvia online discriminative feature selectionrdquo IEEE Transactionson Image Processing vol 22 no 12 pp 4664ndash4677 2013
[34] K Zhang L Zhang M Yang and D Zhang ldquoFast trackingvia spatio- temporal context learningrdquo Computer Vision andPattern Recognition 2013
[35] J F Henriques R Caseiro P Martins and J Batista ldquoHigh-speed tracking with kernelized correlation filtersrdquo IEEE Trans-actions on Pattern Analysis and Machine Intelligence vol 37 no3 pp 583ndash596 2015
[36] X Dong J Shen D Yu W Wang J Liu and H HuangldquoOcclusion-aware real-time object trackingrdquo IEEE Transactionson Multimedia vol 19 no 4 pp 763ndash771 2017
[37] C O Conaire N E OrsquoConnor and A Smeaton ldquoThermo-visual feature fusion for object tracking using multiple spa-tiogram trackersrdquo Machine Vision amp Applications vol 19 no5-6 pp 483ndash494 2008
[38] INO ldquoInorsquos video analytics datasetrdquo httpswwwinocaenvideo-analytics-dataset
[39] C Li X Liang Y Lu Z Nan and T Jin ldquoRGB-T objecttracking benchmark and baselinerdquo IEEE Transactions on ImageProcessing 2018
[40] C O Conaire N E OrsquoConnor and A F Smeaton ldquoDetectoradaptation by maximising agreement between independentdata sourcesrdquo in Proceedings of the 2007 IEEE ComputerSociety Conference on Computer Vision and Pattern RecognitionCVPRrsquo07 pp 1ndash6 IEEE Minneapolis MN USA June 2007
[41] J W Davis and V Sharma ldquoBackground-subtraction usingcontour-based fusion of thermal and visible imageryrdquoComputerVision and Image Understanding vol 106 no 2-3 pp 162ndash1822007
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom
Hindawiwwwhindawicom Volume 2018
MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Mathematical Problems in Engineering
Applied MathematicsJournal of
Hindawiwwwhindawicom Volume 2018
Probability and StatisticsHindawiwwwhindawicom Volume 2018
Journal of
Hindawiwwwhindawicom Volume 2018
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawiwwwhindawicom Volume 2018
OptimizationJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Engineering Mathematics
International Journal of
Hindawiwwwhindawicom Volume 2018
Operations ResearchAdvances in
Journal of
Hindawiwwwhindawicom Volume 2018
Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018
International Journal of Mathematics and Mathematical Sciences
Hindawiwwwhindawicom Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Hindawiwwwhindawicom Volume 2018Volume 2018
Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in
Nature and SocietyHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Dierential EquationsInternational Journal of
Volume 2018
Hindawiwwwhindawicom Volume 2018
Decision SciencesAdvances in
Hindawiwwwhindawicom Volume 2018
AnalysisInternational Journal of
Hindawiwwwhindawicom Volume 2018
Stochastic AnalysisInternational Journal of
Submit your manuscripts atwwwhindawicom