research article object detection and tracking-based...

Research ArticleObject Detection and Tracking-Based Camera Calibration forNormalized Human Height Estimation

Jaehoon Jung1 Inhye Yoon12 Sangkeun Lee1 and Joonki Paik1

1Department of Image Chung-Ang University Seoul 156-756 Republic of Korea2ADAS Camera Team LG Electronics 322 Gyeongmyeong-daero Seo-gu Incheon 22744 Republic of Korea

Correspondence should be addressed to Joonki Paik paikjcauackr

Received 6 July 2016 Revised 10 October 2016 Accepted 25 October 2016

Academic Editor Jesus Corres

Copyright copy 2016 Jaehoon Jung et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

This paper presents a normalized human height estimation algorithm using an uncalibrated camera To estimate the normalizedhuman height the proposed algorithm detects a moving object and performs tracking-based automatic camera calibration Theproposed method consists of three steps (i) moving human detection and tracking (ii) automatic camera calibration and (iii)human height estimation and error correctionThe proposedmethod automatically calibrates camera by detectingmoving humansand estimates the human height using error correction The proposed method can be applied to object-based video surveillancesystems and digital forensic

1 Introduction

A large-scale video analysis usingmultiple cameras is gainingattractions in visual surveillance applications In particularas the use of the object-based video analysis increases thedemand for extraction of object information is growing upSince an object information is changed depending on thecamera parameters such as a location of the installed cameraviewing angle and focal length for this reason variousnormalized object feature extractionmethodswere proposed

Lao et al proposed a human motion analysis methodfor consumer surveillance system [1] This method estimateshuman moving trajectories by tracking and recognizing thehuman motion Del-Blanco et al proposed a multiple objectdetection and tracking framework for the automatic countingof object numbers in a video surveillance application [2]Lee et al detected the object region and estimated the depthinformation using multiple color-filtered apertures (MCA)[3] Chantara et al proposed fast object tracking methodusing adaptive templatematching [4] Chu and Yang detecteda moving object using a background model and estimatedthe object velocity using the object with a previously knownlength [5] Maik et al train the typical poses in both the2D image and 3D space and represent the located poses as

a silhouette for the human pose estimation [6] Kang et alproposed the human gesture detection and tracking methodby using the real-time stereo matching [7] However thismethod uses two or more cameras for the depth estimationIn order to estimate the 3D information using single camerathe camera calibration methods [8ndash11] are proposed ArfaouiandThibault used a diffractive virtual grid to estimate cameraparameters for a fish-eye lens camera [12] Neves et alcorrected the fish-eye distortion using parallel lines and thencalibrated a static and pan-tilt-zoom (PTZ) cameras using anobject height [13] Zhang et al tracked homography based onthemodel plane and then estimated camera parameters usingmaximum likelihood (ML) approach [14] Bell et al used dig-ital display to generate feature points for out-of-focused cam-era calibration [15] Kual-Zheng proposed an object heightestimation method that extracts feature points and estimatesvanishing points using a special pattern such as a cubicbox [16] Gallagher et al proposed a method to analyze ahuman age and gender by calibrating a camera and analyzingdistances among eyes nose and mouth in a human face [17]Since this method uses a special pattern board for the cameracalibration successful analysis is difficult when the size of aface is small Shao et al calibrated a camera using an opticalflow method and normalized the object height to estimate

Hindawi Publishing CorporationJournal of SensorsVolume 2016 Article ID 8347841 9 pageshttpdxdoiorg10115520168347841

2 Journal of Sensors

a moving object at the cost of increased computationalcomplexity [18] Zhao and Hu used a pure translation tocalibrate a camera [19] and Li et al reduced control pointsgiven the intrinsic camera parameters to calibrate a pan-tilt camera [20] Andalo et al estimated vanishing points byclustering lines in an image and then calculated the objectheight [21] However this method cannot accurately estimatevanishing points when the background does not includesufficient pairs of parallel lines User input of the humanheight is another burden of this method

To solve the abovementioned problems the proposedmethod calibrates a camera by detecting and tracking theobject region In addition a projective matrix which is aresult of the camera calibration is applied to the proposedhuman height estimationmethod and then estimated humanheights are accumulated and corrected using the RandomSample Consensus (RANSAC) algorithm As a result theproposedmethod can estimate the normalized human heightusing an uncalibrated camera for a visual surveillance systemThis paper is organized as follows Section 2 describes thecamera projectivemodel and Section 3 presents the proposedcamera calibration and human height estimation algorithmsExperimental results are shown in Section 4 and Section 5concludes the paper

2 Camera Projection Model-BasedCalibration A Review

An object is projected onto a two-dimensional image withdifferent sizes depending on the distance between the objectand a camera In order to estimate the human height usinga single camera the projective relationship between the 3Dspace information and the 2D image plane is neededThepin-hole camera projective model [22] is given as

119904119909 = 119860 [119877 | 119905]119883 (1)

where 119883 represents the coordinate in the 3D space matrix119860 contains intrinsic camera parameters 119877 represents thecamera rotation matrix 119905 represents the camera translationvector 119909 represents the coordinate in the 2D image plane and119904 represents the scale factorThe camera intrinsic parameter isdetermined by focal length (119891119909 119891119910) principal point (119901119909 119901119910)skewness skew and aspect ratio 119886 as

119860 = [[[119891119909 skew 1199011199090 119891119910 1199011199100 0 119886

]]] (2)

To simplify the camera calibration process the proposedmethod assuming that 119891119909 = 119891119910 the principal point is thecenter of the image skew = 0 and 119886 = 1 In the samemanner the camera rotation with regard to 119911-axis is zero andthe translations with regard to 119909-axis and 119910-axis are also zero

Using the vanishing points and lines Liu et al [23]compute the camera parameters as

119891 = radic(11988631198862 minus 119901119910) (V119910 minus 119901119910)120588 = 119886 tan(minusV119909

V119910)

120579 = 119886 tan(minusradicV2119909 + V2119910119891 ) ℎ119888 = ℎ119900(1 minus 119889 (119900ℎ V119897) 10038171003817100381710038171003817119900119891 minus V

10038171003817100381710038171003817 119889 (119900119891 V119897) 1003817100381710038171003817119900ℎ minus V1003817100381710038171003817)

(3)

where 119891 represents the focal length 120588 the rolling angle indegree 120579 the tilt angle in degree ℎ119888 the camera height V119897the horizontal vanishing line 1198861119909 + 1198862119910 + 1198863 = 0 V =[V119909 V119910]119879 the vertical vanishing point ℎ119900 the object height intheworld coordinate 119900119891 the object foot position 119900ℎ the objecthead position and 119889(119860 119861) the distancemeasure between twopoints 119860 and 119861

In order to estimate the physical size of an object in the3D space using the object size in the 2D image the proposedmethod detects the moving human to estimate the 3D spaceinformation To estimate the human height the proposedmethod assumes the foot position on the flat ground planeAs a result the foot position in the 2D image plane is inverselyprojected into the 3D space to obtain the human heightinformation

3 Normalized Human Height Estimation

The proposed human height estimation algorithm is anextended version of Jung et alrsquos work [24] and consists of threesteps (i) moving human detection and tracking (ii) auto-matic camera calibration and (iii) reference object-basedhuman height estimation with error correction Figure 1shows the block diagram of the proposed human heightestimation method where 119868119896 represents the 119896th input frame119877 the moving human region 119874 the human tracking region119875 the projective matrix 119867119896 the 119896th height estimation resultof the human and 119867 the error corrected height estimationresult

31 Moving Human Detection and Tracking The proposedmethod first detects a moving human to estimate its heightIf the detected human region includes the background regionor if the region loses some part of the human body anaccurate estimation of the human height is difficult For thisreason the proposed method generates a background usingtheGaussianmixturemodel (GMM) [25 26] and then detectsand labels the foreground imageThe regions that do not haveenough pixels in the foreground image are removed to reducethe noise

The detected foreground regions include not only a singlehuman region but also a group of human region possibly with

Journal of Sensors 3

Movingobject

detectionObject

trackingAuto

cameracalibration

Heightestimation

Errorcorrection

Inputimage

sequence

Outputdetection

result

Human detection

⋱

Ik

R O P

Hk H

Figure 1 Block diagram of the proposed human height estimation method

(a) (b)

(c) (d)

Figure 2 Moving human region detection and classification results (a) an input frame (b) the detected and labeled foreground regions (c)the pedestrian detection results and (d) classified regions

nonhuman objects which make human tracking difficultand as a result human height estimation error increasesFor that reason the proposed method classifies each regionaccording to whether it is a human region or not The pro-posed classification method uses the combined histogram oforiented gradients and local binary pattern (HOG-LBP) anda support vector machine- (SVM-) based human detectionmethod [27] Using the detected human information eachforeground region is classified into two regions The firstregion is a single human region that has only one humanobject The second region is a single nonhuman region thathas either none or multiple humans Figure 2 shows the mov-ing human region detection and classification results Figures2(b) and 2(c) respectively show the foreground image andhuman detection result of a video shown in Figure 2(a)Figure 2(d) shows the region classification result where single

human and single nonhuman regions are respectively repre-sented by red and black boxes

The proposed method tracks the human and estimatesthe height in a video using the detected single human regionAlthough theKalmanfilter tracker [28] is a popular stochastictrackingmethod it cannot track a nonlinearlymoving objectTo solve this problem the proposed method uses a particlefilter tracker [29] In a surveillance input video humaninformation such as size and shape changeswhile the humanis walking For this reason the model-based tracking [30]method models the target human using a color histogram todeal with the dynamic characteristics of the moving humanIn the proposed method the HSV color histogram is used torepresent the human region to reduce the sensitivity to theilluminance


(a) (b)

(c) (d)

Figure 3 Results of human tracking using the proposed method (a) the 2522th frame (b) the 2557th frame (c) the 2743th frame and (d)the 2819th frame

The particle filter tracking results may include a prob-abilistic error and cannot detect the entire human regionMoreover if the number of particles increases to reduce thetracking error the time complexity also increases To solvethese problems the proposed method detects the trackedhuman region by matching the detected human region withthe tracked human regions as

119874119894 = argmax119877

( 1119899119895 119868 (119877119895 119879119894)) (4)

where119874119894 represents the 119894th human region 119877119895 the 119895th movinghuman region that is detected using the background model119899119895 the number of pixels in the moving human region 119877119895 and119879119894 the tracking region about the 119894th human After matchingthe proposedmethod uses additional trackers for unmatchedsingle human regions

Figure 3 shows the human tracking results using theproposed method In Figure 3 the red box represents theparticle filter tracking result about the moving human andthe white box represents the optimal rectangular region thatencloses the detected human region

32 Vanishing Point and Line Estimation Using Human Infor-mation The normalized human height can be estimated inmeters in the 3D space by estimating camera parameters Forthe automatic camera calibration the vanishing points and

line should be estimated using the parallel lines in the imageLi et al estimated the vanishing line and point by extractingthe lines from the background structure [31] Howeverthis method cannot calibrate the camera if the backgroundstructure does not have a sufficient number of parallel linesFor automatic calibration without using parallel lines theproposed calibration method uses the moving human infor-mation [23] More specifically the proposed method detectsboth foot and head positions of the human in the 2D image asℎ2D = argmin

119900(119910119874119894 119896)

1198912D = 1119899119894 119904119899119894 119904sum119896=1

119874119904119894 (119896) (5)

where ℎ2D represents the human head position in the 2Dimage 119910119874119894 119896 the 119910-axis coordinate of the 119896th pixel in the 119894thhuman region 119874119894 1198912D the foot position in the 2D image119874119904119894 (119896) the sample foot region that consists of 10 pixels ofthe 119894th human region and 119899119894 119904 the number of pixels in thesample region

Both vanishing points and line can be estimated usingthe detected foot and head positions The vertical vanishingpoint can be estimated using the intersection between thefoot-to-head lines that include both the foot and head pointsfrom the corresponding human region The horizontal lineis estimated using two or more horizontal vanishing points


Horizontal vanishing line

Vertical vanishing point

Head point 1

Head point 2

Head point 3

Foot point 1

Foot point 2

Foot point 3

pointvanishing

Horizontal

pointvanishing

Horizontal

Figure 4 Vanishing line estimation process using the humaninformation

and the horizontal vanishing point is using the intersectionbetween the foot-to-foot and head-to-head lines The foot-to-foot and head-to-head lines respectively include foot andhead points Both vanishing points and line are estimatedusing the RANSAC algorithm to reduce the estimation errorFigure 4 illustrates the human-based vanishing point and lineestimation process

33 Human Height Estimation and Error Correction Theproposed method computes the foot point in the 3D spacefor the normalized human height estimation using multiplevideos acquired by different cameras To calculate the footpoint in the 3D space the foot point in the 2D image isinversely projected into 3D space The 3D point is on theline that connects the human foot point in the 3D space withthe corresponding image sensor Since the camera height isestimated based on the ground plane that includes the humanfoot points the 3D foot point can be obtained by normalizingthe inversely projected point with respect to the 119911-axis As aresult the foot point in 3D can be calculated as

1198913D = 1119885 (119875119879119875)minus1 1198751198791198912D (6)

where 1198913D represents the foot point in the 3D space 1198912D thefoot point in the 2D image 119875 the projective matrix and119885 the119911-axis coordinate of the point that is inversely projected fromthe foot point in 2D image

The reference head point in the 3D space can be estimatedby translating the foot point to the vertical direction of theground plane Using the reference head point in the 3D spacethe corresponding head point in the 2D image is given as

ℎ2Dref = 119875ℎ3Dref (7)

where ℎ2Dref represents the reference head point in the 2Dimage and ℎ3Dref the corresponding head point in the 3D space

Using the reference head point the human height can beestimated as

119867119900 =1003816100381610038161003816100381610038161003816100381610038161003816100381610038161003816(1199102Dref ℎ minus 1199102D119891 )(1199102Dℎ

minus 1199102D119891)1003816100381610038161003816100381610038161003816100381610038161003816100381610038161003816 119867ref (8)

where 119867119900 represents the estimated human height 119867ref thereference height 1199102Dref ℎ the 119910-axis coordinate of the referencehead point in the 2D image 1199102Dℎ the 119910-axis coordinate of thehuman head point in the 2D image and 1199102D119891 the 119910-axis coor-dinate of the human foot point in the 2D image In this workthe reference height of 18 meters was used Figure 5 showsthe human height estimation model using reference height

The accuracy of human height estimation depends onthe detected human region To reduce the human heightestimation error the proposed method accumulates theestimated human heights in each frame and corrects theerrors using the RANSAC algorithm In the first step ofthe RANSAC algorithm 119899119904 sample heights are randomlyextracted Next step computes the sum of squared differences(SSD) between the average height and each estimated humanheightThe first and second steps repeat 119899119894 times to obtain theerror corrected height

4 Experimental Results

The proposed human height estimation results are shown inthis section The test video was acquired using an uncali-brated camera viewing down the ground plane at the heightin between 22 and 72 meters Each video sequence has thesize 1280 times 720 and includesmoving humans In addition theperformance evaluation of tracking and surveillance (PETS)2009 dataset [32] was used to test the proposed algorithm

Figure 6 shows the result of human height estimationusing the proposed method Although the size of the humanin a 2D image looks different by scenes the resultingnormalized height is correctly estimated from all differentscenes using a prespecified height of the reference object forthe camera calibration

Figure 7 shows the results of error correction of the esti-mated height Figure 7(a) shows the human height estimationerror caused by the human pose change The human heightestimation error is corrected using the proposed method asshown in Figure 7(b)

Figure 8(a) shows the human height estimation errorcaused by an occlusion The height estimation error of theoccluded human is reduced by the proposed method asshown in Figure 8(b)

Figure 9 shows estimation results of multiple humanheights As shown in Figure 9(a) the height of separatedhuman is estimated In Figures 9(b) 9(c) and 9(d) somehumans adjoined and made multiple human regions Asshown in Figures 9(b) 9(c) and 9(d) the proposed methodestimates the human height by classifying each region

Figure 10 shows the estimated human height in videoframes where the ground truth of the object height is 175


Reference head point

Referenceheight Human

height


Human foot point

Human head pointHorizontal vanishing line

Horizontalvanishing

point 1

Horizontalvanishing

point 2

Figure 5 Human height estimation model using reference height

Height 180 (m) Height 180 (m) Height 182 (m)

(a)


(b)


(c)

Figure 6 Height estimation results of the same person in three different scenes (a) scene 1 (b) scene 2 and (c) scene 3

meters shown as the solid curve The dotted and daggeredcurves respectively show the estimated human height with-out and with error correction As shown in Figure 10 thehuman height estimation error is reduced by 0027 to 0012meters using the proposed error correction method

5 Conclusions

An automatic calibration method is presented using objectdetection and tracking by multiple uncalibrated camerasAs a result the camera parameters including the camera



(a)


(b)

Figure 7 Height estimation error correction results of the walking human (a) human height estimation error caused by the human posechange while the human is walking and (b) error corrected human height estimation results of (a)


(a)


(b)

Figure 8 Height estimation error correction results of the occluded human (a) human height estimation error caused by an occlusion and(b) error corrected human height estimation results of (a)


(a) (b)

(c) (d)

Figure 9 Multiple humans height estimation results (a) 430th frame (b) 461st frame (c) 484th frame and (d) 560th frame

1 11 21 31 41 51 61165

17175

18185

19195

Ground truthHeight estimation results without error correctionHeight estimation results with error correction

Figure 10 Comparison between the estimated height and groundtruth

height are estimated Moreover the proposed algorithm canbe applied to estimate the normalized human height Asa result the normalized human height is estimated usingmultiple uncalibrated cameras The proposed method can beapplied to object tracking and recognition in a very-large areavideo surveillance system

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

Thisworkwas supported by Institute for InformationampCom-munications Technology Promotion (IITP) grant funded bythe Korea government (MSIP) (B0101-15-0525 Developmentof Global Multitarget Tracking and Event Prediction Tech-niques Based on Real-Time Large-Scale Video Analysis) bythe MSIP (Ministry of Science ICT and Future Planning)Korea under the ITRC (Information Technology ResearchCenter) support program (IITP-2016-H8501-16-1018) super-vised by the IITP (Institute for Information amp Commu-nications Technology Promotion) and by the TechnologyInnovation Program (Development of Smart VideoAudioSurveillance SoC amp Core Component for Onsite DecisionSecurity System) under Grant 10047788

References

[1] W Lao J Han and P H N de With ldquoAutomatic video-basedhuman motion analyzer for consumer surveillance systemrdquoIEEE Transactions on Consumer Electronics vol 55 no 2 pp591ndash598 2009

[2] C R Del-Blanco F Jaureguizar and N Garcia ldquoAn efficientmultiple object detection and tracking framework for automaticcounting and video surveillance applicationsrdquo IEEE Transac-tions on Consumer Electronics vol 58 no 3 pp 857ndash862 2012

[3] S Lee J Lee M H Hayes and J Paik ldquoAdaptive backgroundgeneration for automatic detection of initial object region in


multiple color-filter aperture camera-based surveillance sys-temrdquo IEEE Transactions on Consumer Electronics vol 58 no1 pp 104ndash110 2012

[4] W Chantara J Mun D Shin and Y Ho ldquoObject trackingusing adaptive template matchingrdquo IEIE Transactions on SmartProcessing and Computing vol 4 no 1 pp 1ndash9 2015

[5] H-C Chu and H Yang ldquoA simple image-based object velocityestimation approachrdquo in Proceedings of the 11th IEEE Interna-tional Conference on Networking Sensing and Control (ICNSCrsquo14) pp 102ndash107 Miami Fla USA April 2014

[6] V Maik J Park D Kim and J Paik ldquoModel-based humanpose estimation and its analysis using hausdorff matchingrdquoTECHART Journal of Arts and Imaging Science vol 2 no 2p 51 2015

[7] S Kang A Roh C Eem and H Hong ldquoUsing real-timematching for human gesture detection and trackingrdquo TechartJournal of Arts and Imaging Science vol 1 no 1 pp 60ndash66 2014

[8] S Lee S Jeong H Yu et al ldquoEfficient image transformation andcamera registration for the multi-projector image calibrationrdquoTECHART Journal of Arts and Imaging Science vol 3 no 1 p38 2016

[9] H Kim D Kim and J Paik ldquoAutomatic estimation of spatiallyvarying focal length for correcting distortion in fisheye lensimagesrdquo IEEK Transactions on Smart Processing amp Computingvol 2 no 6 pp 339ndash344 2013

[10] A Poulin-Girard S Thibault and D Laurendeau ldquoInfluenceof camera calibration conditions on the accuracy of 3D recon-structionrdquo Optics Express vol 24 no 3 p 2678 2016

[11] C-H Teng Y-S Chen and W-H Hsu ldquoCamera self-calibration method suitable for variant camera constraintsrdquoApplied Optics vol 45 no 4 pp 688ndash696 2006

[12] A Arfaoui and S Thibault ldquoFisheye lens calibration usingvirtual gridrdquo Applied Optics vol 52 no 12 pp 2577ndash2583 2013

[13] J C Neves J C Moreno and H Proenca ldquoA master-slavecalibration algorithm with fish-eye correctionrdquo MathematicalProblems in Engineering vol 2015 Article ID 427270 8 pages2015

[14] Y Zhang L Zhou H Liu and Y Shang ldquoA flexible onlinecamera calibration using line segmentsrdquo Journal of Sensors vol2016 Article ID 2802343 16 pages 2016

[15] T Bell J Xu and S Zhang ldquoMethod for out-of-focus cameracalibrationrdquo Applied Optics vol 55 no 9 pp 2346ndash2352 2016

[16] L Kual-Zheng ldquoA simple calibration approach to single viewheight estimationrdquo inProceedings of the 9thComputer andRobotVision (CRV rsquo12) pp 28ndash30 May 2012

[17] A C Gallagher A C Blose and T Chen ldquoJointly estimatingdemographics and height with a calibrated camerardquo in Pro-ceedings of the IEEE 12th International Conference on ComputerVision (ICCV rsquo09) pp 1187ndash1194 Kyoto Japan September 2009

[18] J Shao S K Zhou and R Chellappa ldquoRobust height estimationof moving objects from uncalibrated videosrdquo IEEE Transactionson Image Processing vol 19 no 8 pp 2221ndash2232 2010

[19] B Zhao and ZHu ldquoCamera self-calibration from translation byreferring to a known camerardquoApplied Optics vol 54 no 25 pp7789ndash7798 2015

[20] Y Li J Zhang and J Tian ldquoMethod for pan-tilt cameracalibration using single control pointrdquo Journal of the OpticalSociety of America A vol 22 no 1 pp 156ndash163 2014

[21] F A Andalo G Taubin and S Goldenstein ldquoEfficient heightmeasurements in single images based on the detection ofvanishing pointsrdquo Computer Vision and Image Understandingvol 138 pp 51ndash60 2015

[22] Y Li J Zhang and J Tian ldquoCamera calibration from vanishingpoints in image of architectural scenesrdquo in Proceedings of theBritish Machine Vision Conference pp 382ndash391 NottinghamUK September 1999

[23] J Liu R Collins and Y Liu ldquoSurveillance camera autocalibra-tion based on pedestrian height distributionsrdquo in Proceedings ofthe British Machine Vision Conference pp 1ndash11 January 2011

[24] J Jung H Kim I Yoon and J Paik ldquoHuman height analysisusing multiple uncalibrated camerasrdquo in Proceedings of the 2016IEEE International Conference on Consumer Electronics (ICCErsquo16) pp 213ndash214 Las Vegas Nev USA January 2016

[25] Z Zivkovic ldquoImproved adaptive Gaussian mixture model forbackground subtractionrdquo inProceedings of the 17th InternationalConference on Pattern Recognition (ICPR rsquo04) vol 2 pp 28ndash31Cambridge UK August 2004

[26] Y Kim S Jeong J Oh and S Lee ldquoFast MOG (Mixture ofGaussian) algorithm based on predicting model parametersrdquoTECHART Journal of Arts and Imaging Science vol 2 no 1 pp41ndash45 2015

[27] W-J Park D-H Kim Suryanto C-G Lyuh T M Roh and S-J Ko ldquoFast human detection using selective block-based HOG-LBPrdquo in Proceedings of the 19th IEEE International Conferenceon Image Processing (ICIP rsquo12) pp 601ndash604 Orlando Fla USASeptember 2012

[28] S S Pathan A Al-Hamadi and B Michaelis ldquoIntelligentfeature-guided multi-object tracking using Kalman filterrdquo inProceedings of the 2nd International Conference on ComputerControl and Communication pp 1ndash6 February 2009

[29] T Zhang S Fei X Li and H Li ldquoAn improved particle filter fortracking color objectrdquo in Proceedings of the International Con-ference on Intelligent Computation Technology and Automation(ICICTA rsquo08) vol 2 pp 109ndash113 Hunan China October 2008

[30] E J Rhee J Park B Seo and J Park ldquoSubjective evaluationon perceptual tracking errors from modeling errors in model-based trackingrdquo IEIE Transactions on Smart Processing andComputing vol 4 no 6 pp 407ndash412 2015

[31] B Li K Peng X Ying and H Zha ldquoVanishing point detectionusing cascaded 1D Hough transform from single imagesrdquoPattern Recognition Letters vol 33 no 1 pp 1ndash8 2012

[32] Performance evaluation of tracking and surveillance 2009httpwwwcvgreadingacukPETS2009ahtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Active and Passive Electronic Components

Control Scienceand Engineering

Journal of



RotatingMachinery


Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design



Shock and Vibration


Civil EngineeringAdvances in

Acoustics and VibrationAdvances in



Electrical and Computer Engineering

Journal of

Advances inOptoElectronics


Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Chemical EngineeringInternational Journal of Antennas and

Propagation




Navigation and Observation



DistributedSensor Networks



a moving object at the cost of increased computationalcomplexity [18] Zhao and Hu used a pure translation tocalibrate a camera [19] and Li et al reduced control pointsgiven the intrinsic camera parameters to calibrate a pan-tilt camera [20] Andalo et al estimated vanishing points byclustering lines in an image and then calculated the objectheight [21] However this method cannot accurately estimatevanishing points when the background does not includesufficient pairs of parallel lines User input of the humanheight is another burden of this method

To solve the abovementioned problems the proposedmethod calibrates a camera by detecting and tracking theobject region In addition a projective matrix which is aresult of the camera calibration is applied to the proposedhuman height estimationmethod and then estimated humanheights are accumulated and corrected using the RandomSample Consensus (RANSAC) algorithm As a result theproposedmethod can estimate the normalized human heightusing an uncalibrated camera for a visual surveillance systemThis paper is organized as follows Section 2 describes thecamera projectivemodel and Section 3 presents the proposedcamera calibration and human height estimation algorithmsExperimental results are shown in Section 4 and Section 5concludes the paper

2 Camera Projection Model-BasedCalibration A Review

An object is projected onto a two-dimensional image withdifferent sizes depending on the distance between the objectand a camera In order to estimate the human height usinga single camera the projective relationship between the 3Dspace information and the 2D image plane is neededThepin-hole camera projective model [22] is given as

119904119909 = 119860 [119877 | 119905]119883 (1)

where 119883 represents the coordinate in the 3D space matrix119860 contains intrinsic camera parameters 119877 represents thecamera rotation matrix 119905 represents the camera translationvector 119909 represents the coordinate in the 2D image plane and119904 represents the scale factorThe camera intrinsic parameter isdetermined by focal length (119891119909 119891119910) principal point (119901119909 119901119910)skewness skew and aspect ratio 119886 as

119860 = [[[119891119909 skew 1199011199090 119891119910 1199011199100 0 119886

]]] (2)

To simplify the camera calibration process the proposedmethod assuming that 119891119909 = 119891119910 the principal point is thecenter of the image skew = 0 and 119886 = 1 In the samemanner the camera rotation with regard to 119911-axis is zero andthe translations with regard to 119909-axis and 119910-axis are also zero

Using the vanishing points and lines Liu et al [23]compute the camera parameters as

119891 = radic(11988631198862 minus 119901119910) (V119910 minus 119901119910)120588 = 119886 tan(minusV119909

V119910)

120579 = 119886 tan(minusradicV2119909 + V2119910119891 ) ℎ119888 = ℎ119900(1 minus 119889 (119900ℎ V119897) 10038171003817100381710038171003817119900119891 minus V

10038171003817100381710038171003817 119889 (119900119891 V119897) 1003817100381710038171003817119900ℎ minus V1003817100381710038171003817)

(3)

where 119891 represents the focal length 120588 the rolling angle indegree 120579 the tilt angle in degree ℎ119888 the camera height V119897the horizontal vanishing line 1198861119909 + 1198862119910 + 1198863 = 0 V =[V119909 V119910]119879 the vertical vanishing point ℎ119900 the object height intheworld coordinate 119900119891 the object foot position 119900ℎ the objecthead position and 119889(119860 119861) the distancemeasure between twopoints 119860 and 119861

In order to estimate the physical size of an object in the3D space using the object size in the 2D image the proposedmethod detects the moving human to estimate the 3D spaceinformation To estimate the human height the proposedmethod assumes the foot position on the flat ground planeAs a result the foot position in the 2D image plane is inverselyprojected into the 3D space to obtain the human heightinformation

3 Normalized Human Height Estimation

The proposed human height estimation algorithm is anextended version of Jung et alrsquos work [24] and consists of threesteps (i) moving human detection and tracking (ii) auto-matic camera calibration and (iii) reference object-basedhuman height estimation with error correction Figure 1shows the block diagram of the proposed human heightestimation method where 119868119896 represents the 119896th input frame119877 the moving human region 119874 the human tracking region119875 the projective matrix 119867119896 the 119896th height estimation resultof the human and 119867 the error corrected height estimationresult

31 Moving Human Detection and Tracking The proposedmethod first detects a moving human to estimate its heightIf the detected human region includes the background regionor if the region loses some part of the human body anaccurate estimation of the human height is difficult For thisreason the proposed method generates a background usingtheGaussianmixturemodel (GMM) [25 26] and then detectsand labels the foreground imageThe regions that do not haveenough pixels in the foreground image are removed to reducethe noise

The detected foreground regions include not only a singlehuman region but also a group of human region possibly with


Movingobject

detectionObject

trackingAuto

cameracalibration

Heightestimation

Errorcorrection

Inputimage

sequence

Outputdetection

result

Human detection

⋱

Ik

R O P

Hk H


(a) (b)

(c) (d)






(a) (b)

(c) (d)



119874119894 = argmax119877

( 1119899119895 119868 (119877119895 119879119894)) (4)





119900(119910119874119894 119896)

1198912D = 1119899119894 119904119899119894 119904sum119896=1

119874119904119894 (119896) (5)






Head point 1

Head point 2

Head point 3

Foot point 1

Foot point 2

Foot point 3

pointvanishing

Horizontal

pointvanishing

Horizontal




1198913D = 1119885 (119875119879119875)minus1 1198751198791198912D (6)






119867119900 =1003816100381610038161003816100381610038161003816100381610038161003816100381610038161003816(1199102Dref ℎ minus 1199102D119891 )(1199102Dℎ

minus 1199102D119891)1003816100381610038161003816100381610038161003816100381610038161003816100381610038161003816 119867ref (8)













height


Human foot point


Horizontalvanishing

point 1

Horizontalvanishing

point 2



(a)


(b)


(c)



5 Conclusions




(a)


(b)



(a)


(b)



(a) (b)

(c) (d)


1 11 21 31 41 51 61165

17175

18185

19195




Competing Interests


Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










Movingobject

detectionObject

trackingAuto

cameracalibration

Heightestimation

Errorcorrection

Inputimage

sequence

Outputdetection

result

Human detection

⋱

Ik

R O P

Hk H


(a) (b)

(c) (d)






(a) (b)

(c) (d)



119874119894 = argmax119877

( 1119899119895 119868 (119877119895 119879119894)) (4)





119900(119910119874119894 119896)

1198912D = 1119899119894 119904119899119894 119904sum119896=1

119874119904119894 (119896) (5)






Head point 1

Head point 2

Head point 3

Foot point 1

Foot point 2

Foot point 3

pointvanishing

Horizontal

pointvanishing

Horizontal




1198913D = 1119885 (119875119879119875)minus1 1198751198791198912D (6)






119867119900 =1003816100381610038161003816100381610038161003816100381610038161003816100381610038161003816(1199102Dref ℎ minus 1199102D119891 )(1199102Dℎ

minus 1199102D119891)1003816100381610038161003816100381610038161003816100381610038161003816100381610038161003816 119867ref (8)













height


Human foot point


Horizontalvanishing

point 1

Horizontalvanishing

point 2



(a)


(b)


(c)



5 Conclusions




(a)


(b)



(a)


(b)



(a) (b)

(c) (d)


1 11 21 31 41 51 61165

17175

18185

19195




Competing Interests


Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










(a) (b)

(c) (d)



119874119894 = argmax119877

( 1119899119895 119868 (119877119895 119879119894)) (4)





119900(119910119874119894 119896)

1198912D = 1119899119894 119904119899119894 119904sum119896=1

119874119904119894 (119896) (5)






Head point 1

Head point 2

Head point 3

Foot point 1

Foot point 2

Foot point 3

pointvanishing

Horizontal

pointvanishing

Horizontal




1198913D = 1119885 (119875119879119875)minus1 1198751198791198912D (6)






119867119900 =1003816100381610038161003816100381610038161003816100381610038161003816100381610038161003816(1199102Dref ℎ minus 1199102D119891 )(1199102Dℎ

minus 1199102D119891)1003816100381610038161003816100381610038161003816100381610038161003816100381610038161003816 119867ref (8)













height


Human foot point


Horizontalvanishing

point 1

Horizontalvanishing

point 2



(a)


(b)


(c)



5 Conclusions




(a)


(b)



(a)


(b)



(a) (b)

(c) (d)


1 11 21 31 41 51 61165

17175

18185

19195




Competing Interests


Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












Head point 1

Head point 2

Head point 3

Foot point 1

Foot point 2

Foot point 3

pointvanishing

Horizontal

pointvanishing

Horizontal




1198913D = 1119885 (119875119879119875)minus1 1198751198791198912D (6)






119867119900 =1003816100381610038161003816100381610038161003816100381610038161003816100381610038161003816(1199102Dref ℎ minus 1199102D119891 )(1199102Dℎ

minus 1199102D119891)1003816100381610038161003816100381610038161003816100381610038161003816100381610038161003816 119867ref (8)













height


Human foot point


Horizontalvanishing

point 1

Horizontalvanishing

point 2



(a)


(b)


(c)



5 Conclusions




(a)


(b)



(a)


(b)



(a) (b)

(c) (d)


1 11 21 31 41 51 61165

17175

18185

19195




Competing Interests


Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












height


Human foot point


Horizontalvanishing

point 1

Horizontalvanishing

point 2



(a)


(b)


(c)



5 Conclusions




(a)


(b)



(a)


(b)



(a) (b)

(c) (d)


1 11 21 31 41 51 61165

17175

18185

19195




Competing Interests


Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











(a)


(b)



(a)


(b)



(a) (b)

(c) (d)


1 11 21 31 41 51 61165

17175

18185

19195




Competing Interests


Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










(a) (b)

(c) (d)


1 11 21 31 41 51 61165

17175

18185

19195




Competing Interests


Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation









research article object detection and tracking-based...

Documents