a debris image tracking using optical flow algorithm

Available online at www.sciencedirect.com

www.elsevier.com/locate/asr

Advances in Space Research 49 (2012) 1007–1018

A debris image tracking using optical flow algorithm

K. Fujita a,⇑, T. Hanada a, Y. Kitazawa b, A. Kawabe b

a Department of Aeronautics and Astronautics, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka 819-0395, Japanb IHI Corporation, Toyosu IHI Bldg., 1-1, Toyosu 3-chome, Koto-ku, Tokyo 135-8710, Japan

Received 24 May 2011; received in revised form 24 November 2011; accepted 13 December 2011Available online 21 December 2011

Abstract

This study proposes a motion detection and object tracking technique for GEO debris in a sequence of images. A couple of techniques(called the “stacking method” and “line-identifying technique”) were recently proposed to address the same problem. Although thesetechniques are effective at detecting the debris position and motion in the image sequences, there are some issues concerned with com-putational load and assumed debris motion. This study derives a method to estimate motion vectors of objects in image sequence andfinally detect the debris locations by using a computer vision technique called an optical flow algorithm. The new method detects theseparameters in low computational time in a serial manner, which implies that it has an advantage to track not only linear but also non-linear motion of GEO debris more easily than the previous methods. The feasibility of the proposed methods is validated using real andsynthesized image sequences which contain some typical debris motions.� 2011 COSPAR. Published by Elsevier Ltd. All rights reserved.

Keywords: Debris observation; Image processing; Optical flow algorithm

1. Introduction

In recent years, the approaches to measure space debrisdistributions in Earth-centered orbits have been dividedinto two categories: “Ground-based Measurement” and“Space-based Measurement”. By using both approaches,a reliable orbital debris environment model has alreadybeen obtained for the Low Earth Orbit (LEO) region. Onthe other hand, for the Geosynchronous Earth Orbit(GEO) region around the altitude of 36,000 km, ground-based observations using high-resolution CCD camerashave been conducted by various organizations. However,the distribution of GEO objects including micro-debrisunder 10 cm in size has not been estimated sufficiently(Schildknecht et al., 2004, 2001). In order to construct areliable GEO environment model, it is desired to detectunknown debris in the images from ground-based observa-tions, employing efficient techniques of image processing.

0273-1177/$36.00 � 2011 COSPAR. Published by Elsevier Ltd. All rights rese

doi:10.1016/j.asr.2011.12.010

⇑ Corresponding author.E-mail address: [email protected] (K. Fujita).

In this work, a novel approach to tracking debris in asequence of images is proposed using a technique in the fieldof computer vision. The technique called an optical flowalgorithm is different from those in the past studies in thatit can effectively estimate motion vectors of objects in animage sequence, and finally detects unknown debris trajec-tories. It has an advantage of tracking nonlinear trajectoriesof the debris images more easily than the previous methods.

In order to show the feasibility of the proposed method, itis applied to both real observation image sequences and syn-thesized image sequences using orbital elements of real debris.

2. Methods to detect and track GEO debris image in past

studies

This study treats the “search observation” approach toobserving GEO debris. A series of observation images aretaken for same area with a telescope fixed in a topocentric,Earth-fixed coordinate system, which is equivalent totopocentric equatorial coordinate system described in Sec-tion 5. Given that images are obtained from a camera with

rved.

http://dx.doi.org/10.1016/j.asr.2011.12.010

mailto:[email protected]

http://dx.doi.org/10.1016/j.asr.2011.12.010

i

j

ij

Fig. 2. A line-identifying technique.

1008 K. Fujita et al. / Advances in Space Research 49 (2012) 1007–1018

small exposure time in such a topocentric, Earth-fixed sys-tem, stars appear to be line segments locally stretching inthe same direction. On the other hand, if GEO debris isdefined as an object not exactly on geostationary orbitbut around it, its image appears to be point-like objectand also moves across the night sky images depending onits actual orbital elements.

In recent studies aiming at detecting and tracking debrisin GEO or asteroids during the search observation, imageprocessing methods called the stacking method and theline-identifying technique were proposed (Yanagisawaet al., 2008, 2009). A schematic view of the stackingmethod is shown in Fig. 1. In the framework of thismethod, medians of luminance values for local block imageincluding a faint object are calculated for each frame, andfrom them a new image called a “median image” is synthe-sized. As the median values for the CCD image effectivelyeliminate noise signals or unexpected bright stars, onlydebris appears in the synthesized image.

During this process, the local block images have to becropped considering the debris motion direction so thatthe target image is always at the same location in the blockimage frames. Consequently, its visibility is improved withrespect to the number of stacked image frames. While thestacking method is effective at detecting faint debris whosevisibility is poor in an image frame, it needs another search-ing or tracking method for the moving faint object. Thoughrandom searching techniques are usually applied for thismethod, it has large computational cost.

On the other hand, the line-identifying technique wasproposed to detect and track linear trajectories of debris(Fig. 2). Given a previously stated condition of the searchobservation, the trajectory of the debris is regarded as aline segment across the spatio-temporal domain. If a linearmodel, r(k) = ak + b (r = [ximg(k),yimg(k)]T: the positionvector of the object in the kth frame, a, b: the 2 � 1 con-stant vectors) is derived from two point-like objects onarbitrarily selected frames (the ith and jth frames inFig. 2), the same object between these frames can be iden-tified as the points at the intersections of the image frameswith the line. Because the debris locations in the spatio-temporal domain are detected by a simple linear model,this technique is more computationally efficient than theother image tracking methods.

Fig. 1. A stacki

Note here that unlike the conventional “blinking” tech-nique, the line-identifying technique effectively tracks thedebris image using a linear model of their trajectories inspatio-temporal domain, as well as detects the debris loca-tions as the points on the linear trajectories.

Meanwhile, as the line-identifying technique does notaim at detecting faint debris, it may have an issue of miss-ing possible debris images for each image frame. Also, thistechnique only detects linear trajectories in the imagesequence, and it cannot be applied to nonlinear ones. Fur-thermore, we should note that in order to improve theeffectiveness of this technique, the images of the other celes-tial bodies such as stars have to be carefully eliminatedfrom the original image frames, and the preprocessedimage sequence which contains a limited number of possi-ble debris has to be applied.

3. Debris image tracking with optical flow algorithm

3.1. Optical flow algorithm

In this study, an image processing method called anoptical flow algorithm is treated in order to track debrisin image sequences. Optical flow is a vector field of theapparent motion of an object in a visual scene, and varioustechniques to estimate the flow vectors in sequential imageshave been proposed in the field of computer vision. Thesetechniques basically utilize unique properties of temporal

ng method.

K. Fujita et al. / Advances in Space Research 49 (2012) 1007–1018 1009

and spatial changes of luminance values in an imagesequence, and there are three typical algorithms:

� Horn and Schunck algorithm (HS algorithm) (Horn andSchunck, 1981).� Lucas and Kanade algorithm (LK algorithm) (Lucas

and Kanade, 1981).� Block matching algorithm or correlation technique (BM

algorithm) (e.g. Faugeras, 1993).

As for the HS and LK algorithms, the smoothness of theobject’s shape captured on the image plane is assumed,which implies that luminance values of the image pixelsare spatially and temporally differentiable in the spatiotem-poral domain. However, images of stars and debris in thenight sky are usually discretely-distributed, and the abovecondition is not straightforward. On the other hand, nocondition on the distribution of luminance values isassumed for the BM algorithm, which is realized by evalu-ating pixel correlations for two block images appropriatelyselected in the different image frames (Fig. 3). Therefore, inthis paper, the BM algorithm is applied to detect debris inthe image sequence.

The BM algorithm needs a function to evaluate thedegree of coincidence between two block images, and typ-ical functions are as follows:

Normalized cross correlation:

ðNCCÞ¼Pm

i¼�m

Pnj¼�nE1ðxþ i;yþ jÞE2ðx0 þ i;y0 þ jÞffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPm

i¼�m

Pnj¼�nE1ðxþ i;yþ jÞ2

q ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPmi¼�m

Pnj¼�nE2ðx0 þ i;y0 þ jÞ2

q ;

ð1Þ

Sum of absolute difference:

ðSADÞ ¼Xm

i¼�m

Xn

j¼�n

E1ðxþ i; y þ jÞ � E2ðx0 þ i; y0 þ jÞj j; ð2Þ

Sum of squared difference:

ðSSDÞ ¼Xm

i¼�m

Xn

j¼�n

E1ðxþ i; y þ jÞ � E2ðx0 þ i; y0 þ jÞf g2:

ð3Þ

Whereas the NCC is a function to evaluate the exactcorrelation between two block image pixels, SAD andSSD are simplified ones to evaluate the coincidence

E1(x, y)

x

y

E2( , )

2m+1

2n+1

2m+1

2n+1

Fig. 3. Two block images to evaluate coincidence between two imageframes for the BM algorithm.

between two block images. The latter ones have advantageswith their efficient computational loads. Furthermore, thefollowing equation is derived from the definition of SSDsuch that

Xm

i¼�m

Xn

j¼�n

E1ðxþ i;yþjÞ�E2ðx0 þ i;y 0 þjÞf g2

¼Xm

i¼�m

Xn

j¼�n

E1ðxþ i;yþ jÞ2�2E1ðxþ i;yþjÞE2ðx0 þ i;y0 þ jÞþE2ðx0 þ i;y0 þ jÞ2n o

: ð4Þ

Since the second term in the summation of the right-handside of the above equation is equivalent to NCC, if thevalues of the self correlation,

Pmi¼�m

Pnj¼�nE2ðx0þ

i; y 0 þ jÞ2 are nearly constant all over the second imageframe, SSD is almost equivalent to NCC. Thus, in thispaper, the SSD is applied to the methods derived in thefollowing subsections.

Although the BM algorithm is more suitable to nightsky images among the three typical optical flow algorithms,it cannot straightforwardly estimate the flow vectors forcelestial bodies crowded in the image sequence. Fig. 4shows a result of applying an original BM algorithm to apair of sequential images. They are binarized using appro-priate threshold values, and each pixel data for the whiteimages are applied to the above algorithm, and the esti-mated flow vectors are shown as the line segments whichshow the directions and the magnitudes of the motion vec-tors between two images.

As shown in the circles in each image frame, four brightstar images are clearly seen as the objects horizontally mov-ing from left to right. However, the estimation errors areserious for the two objects on the left-hand side, which ismainly caused by the crowded objects in left half side ofthe images. Considering these issues, some techniques topreprocess the original images and to enhance the originalBM algorithm are proposed in the following subsections.

3.2. Preprocessing of the image sequences

In this study, astronomical images as shown in Fig. 4 aretreated. They are composed of celestial bodies such asstars, planets, and debris in dark background of nightsky. As stated in Section 2, given that images are obtainedfrom a camera with a small exposure time in the topocen-tric and Earth-fixed system, relatively high luminance val-ues for the GEO debris occupy a point-like area, andthose for the stars occupy a linear or flattened ellipticalarea. In this work, the image sequence is preprocessed tobe efficiently applied to the proposed methods, which willbe described in the following subsections. The steps ofthe preprocessing method are described as follows:

Step 1: For each image frame, the threshold value forbinarization is selected so that the number of pix-els for bright objects should be under a predeter-mined value.

Step 2: Binarization is conducted using the above thresh-old values.

Fig. 4. A result of applying a standard optical flow algorithm (a BM algorithm) to sequential night sky images.


Step 3: Labeling is conducted for a series of the bright objectpixels so that the neighboring bright points in theimage frame are regarded as the same image region.

Step 4: For the labeled images in Step 3, star images whichhave the characteristic shapes (e.g. oval, broad linesegment) are removed from each image frame.

The predetermined value in Step 1 is manually adjustedconsidering observation conditions such as backgroundbrightness level and density of bright objects in the imageframe. Note here that this selection may affect the perfor-mance of the following estimation algorithms, and shouldbe carefully conducted. In the process of Step 4, as mostof the star images taken in the search observation appearto be elliptical segments, they can be removed utilizing theiroblateness, a ratio of the pixel lengths of the major and theminor axes.

Fig. 5 shows results of the preprocessing for an imageframe obtained at Mt. Nyukasa observation facility (seeTable 1). As seen in these results, a limited number of thebinarized images including possible debris remain in theimage frame after the preprocessing. As for this imageframe, the predetermined value in Step 1 was set at20,000, which implies that 20,000 white pixels are extractedamong total 2048 � 2048 pixel data. In Step 4, the oblate-ness to remove star images was set at 0.5 for the presentexposure time (3 s), which resulted in 17 labeled objectsremaining in the binarized image frames.

3.3. Direct matching between labeled binary images

When the BM algorithm is applied to the images prepro-cessed according to the previous subsection, the originaltechnique of the algorithm stated in Section 3.1 causes afew practical issues. One issue is concerned with heavycomputational load mainly due to starting the matchingprocess for the second frame from the same location inthe first frame, and to fixing a searching range. It is waste-ful for the sparsely-distributed debris candidates to applythe original matching process, and it may take much timeto process depending on sizes of the searching range andthe image frame as shown in Fig. 6.

In order to avoid the issue, a method called “directmatching between the labeled binary images” is derived.In the proposed algorithm, the correlation matching isdirectly conducted for the labeled binary images in eachimage frame, that is, the output of the correlation function(Eq. (3)) is calculated between the two block images whosecenter is coincident with the centroid of each labeled object.As schematically shown in Fig. 7, the block image that hasa maximum correlation value among those in the neighbor-ing area in the second frame is regarded to contain the tar-get object.

3.4. Variable frame matching

Another issue is found in the original BM algorithmapplied to a series of image frames as shown in Fig. 8.The correlation matching is usually applied only to twosequential image frames for each object. Thus, the opticalflow estimation for blinking debris, which discontinuouslyappears in the sequence of images tends to fail when theimage frame does not contain the corresponding debris.

A technique to tackle the issue is proposed as shown inFig. 9. Unlike the original BM algorithm, the correlationmatching for the first frame is applied to the following n

frames (1 < n < N, N: the total number of the imageframes). Accordingly, the motion of the object is estimatedfrom all the results of the n matching processes. Althoughthe number of frames increases in this process, the compu-tational load for this technique is not serious compared tothe previously stated issue on the correlation matching inthe original BM algorithm.

4. Test measurement for Mt. Nyukasa image sequences

In order to validate the effectiveness of the proposeddebris detection methods, they are applied to real imagesequences obtained from an equatorial mount telescopeat Mt. Nyukasa optical observation facility of Japan Aero-space Exploration Agency (JAXA). The observation condi-tions which are relevant to the proposed debris trackingtechnique are shown in Table 1. For this test measurement,ten data sets in which exact debris images are previously

Fig. 5. The preprocessing of the observation image to apply the debris tracking algorithm. (The image contrasts are adjusted to improve the visibilities.The white circles in figure (c) are manually imposed to show the locations of the labeled images which are obtained after removing the star images in figure(b)).

Table 1The observation conditions at JAXA’s Mt. Nyukasa optical observation facility.

Observation site (WGS84) JAXA Mt. Nyukasa ObservationFacilityLatitude: 138�1001800ELongitude: 35�5400500NAltitude: 1870 m

Specification of CCD camera CCD format: 2048 � 2048 pixelsPixel scale: 2.2500

Field Of View:1.28 deg � 1.28 deg

Observation period 2010/3/13-19, 11:00-19:00 UTC(20:00–28:00 JST)

Observation zone (The equatorial coordinates for the image center) RA: 113.88–120.48 deg, Dec:7.97-8.95 deg

Frame-to-frame time Approx. 14 s/frame(Exposure: 3 s, Read-out time:11 s)

The number of the frames for each image data set 18 frames/set


detected by JAXA are applied to the proposed optical flowalgorithm as well as to the original BM algorithm and theline-identifying technique.

Because the preprocessing of the original image framesis important to effectively apply the tracking algorithms,these preprocessing parameters were carefully selected toleave a limited number of possible debris images for eachdata set. In this study, the threshold values for the binari-zation are set so that the number of pixels of the brightimages should be under 20,000 (except for data set “a” inTable 2, which is set to under 3000) considering the averagebrightness level of the background images for each data set.

While applying the BM algorithms, an appropriate win-dow size of the block image should be selected consideringthe balance between the local density of the labeled objectsand the maximum moving distance of the debris image inthe preprocessed image frame. In the case of this study, itwas set at 100 � 100 pixels, except that the 30 � 30 pixel

block image was applied for three data sets, “h”, “i”, and“j” considering the high local density of the objects in thepreprocessed image frame. Also, the number of the maxi-mum frames (n in Fig. 9) for the matching technique pro-posed in the Section 3.4 is set at 5 as a trial.

Table 2 shows the mean numbers of the labeled objectsin the preprocessed image frames, and the computationaltimes for the three debris tracking methods, the line-identi-fying technique and the optical flow algorithms with theoriginal BM algorithm, and with the proposed method(the “direct matching between labeled binary images” andthe “variable frame matching”), respectively. These resultsare obtained using MATLAB (ver.7.11) code run on a PC(CPU: Intel Core i5 2.67 GHz, RAM: 4.0 GB).

Note here that these computational times include theones to preprocess the original images (the binarization,the labeling, and the removal of the star images), whichimplies that they are not simply proportional to the

Fig. 6. The way of correlation matching between two block images in theoriginal BM algorithm.

Fig. 7. The way of correlation matching between two block images in theproposed BM algorithm.

Fig. 8. The way of correlation matching for sequential image frames in the

Fig. 9. The way of correlation matching for sequential image frames in thecorrelation matching is set such that 1 < n < N).


number of the objects in the preprocessed image frames. Asseen in the tables, the proposed method is as effective incomputational time as the line-identifying technique is,while the original BM algorithm takes 1.5–8 times longer.

Figs. 10 and 11 show two representative results of thedebris trajectories detected in the CCD image coordinates.In these figures the lines detected by the line-identifyingtechnique and the flow vectors detected in the proposedoptical flow algorithm are depicted by the line segments.The dashed arrows are added to clearly show the estimateddirections of the flow vectors. The small circles are alsoshown to indicate the locations of the objects detected inthe CCD image coordinates.

While these results only show the lines and the flow vec-tors with the corresponding objects which are detected bythe two methods, it is clear where moving debris are inthe image frames. Fig. 10(a) and Fig. 11(b) include resultsof false detections, which were confirmed by a posterioridata analysis.

Note here that the debris image around (ximg,yimg) =(1800,1500) in Fig. 11 discontinuously emerges in theoriginal image frames, that is effectively tracked by theproposed method while it cannot be with the original BMalgorithm.

5. Feasibility study on predictive performance of the

proposed method

The proposed optical flow algorithm does not only tracklinear trajectories but can also track nonlinear ones of the

original BM algorithm. (N: the number of the frames in an image set).

proposed BM algorithm. (The number of the maximum frames for the

Table 2The results of the test measurement of Mt. Nyukasa image sequences.

Data set a b c d e f g h i j

The number of the objects in the preprocessed image frame (meanvalue for the 18 frames)

22 32 8 9 13 13 19 172 24 28

Computational time [sec] – The line-identifying technique 21 128 70 84 110 108 107 213 111 120Computational time [sec] – The optical flow algorithm (The original

BM algorithm)770 1021 249 308 471 469 630 656 170 193

Computational time [sec] – The optical flow algorithm (The proposedmethod)

26 134 73 82 114 107 112 137 114 124

ximg

yimg

ximg

yimg

Fig. 10. A result of the test measurement of Mt. Nyukasa image sequence for the data set “e”. The series of the circles show the trajectories of the objectsdetected in the CCD image coordinates.

ximg

yimg

ximg

yimg

Fig. 11. A result of the test measurement of Mt. Nyukasa image sequence for the data set “g”. The series of circles show the trajectories of the objectsdetected in the CCD image coordinates.


Fig. 12. Relationships between the Geocentric Equatorial Coordinate System (IJK) and the Topocentric Equatorial Coordinate System (ItJtKt).


debris images, which is not clearly shown in the test mea-surement using the real observation images in the previoussection. In order to discuss the effectiveness of the proposedmethod against the past ones, a numerical simulation isconducted. This is a feasibility study to show the trackingperformance of the proposed method.

In this section, a Geocentric Equatorial Coordinate Sys-tem (IJK) and a Topocentric Equatorial Coordinate Sys-tem (ItJtKt) are treated and the relationship betweenthem is schematically shown in Fig. 12. If the former isassumed to be celestial, it is suitable to deal with an orbitaldata such as Two Line Elements (TLE), and the debrislocation can be described using the right ascension, a,and the declination, d. On the other hand, the latter is suit-able for debris trajectories in the CCD image coordinatesobtained from an equatorial mount telescope fixed in atopocentric, Earth-fixed coordinate system.

If position vectors in the two coordinate systems aredefined as rIJK and rI tJ tKt , a relationship between them isreduced to the following equation:

rI tJ tKt ¼ rIJK �ðC þ hÞ cosð/gdÞ cosðhLST ÞðC þ hÞ cosð/gdÞ sinðhLST Þ

ðS þ hÞ sinð/gdÞ

264

375; ð5Þ

where /gd and hLST(=hLST(t)) describe geodetic latitudeand local sidereal time respectively, for the observer’s posi-tion. C and S are the Earth’s radii of curvature in themeridian and in the prime vertical, respectively (for thespherical model of the Earth, they correspond to the meanradius, RE in Fig. 12), and h is the height above the ellipsoi-dal earth (Vallado, 2007). On the other hand, the debrislocation in the IJK system is described using the declina-tion d and the right ascension a, defined in the geocentriccoordinate system such as:

rIJK ¼x

y

z

264375 ¼

r cos d cos a

r cos d sin a

r sin d

264

375; ð6Þ

where r is the orbital radius.

A similar equation is derived for the debris location inthe ItJtKt system such as:

rItJ tKt ¼xt

yt

zt

264

375 ¼

rt cos dt cos at

rt cos dt sin at

rt sin dt

264

375; ð7Þ

where rt, dt, and at describe orbital radius, declination, andright ascension in the topocentric equatorial coordinatesystem, respectively.

Furthermore, relationships between the equatorial coor-dinates, d, a, and the debris orbital parameters are derivedas follows (Chobotov, 2002):

sind¼ sin i �sinu;�i6d6 i ð06 i6p=2Þ�ðp� iÞ6d6p� i ðp=26 i6pÞ

�ð8Þ

tanða� XÞ ¼ cos i � tan u; 0 6 a 6 2p; ð9Þ

where the right ascension of the ascending node (RAAN),X, the inclination, i, and the argument of latitude, u definedebris orbital elements. Also, u can be reduced into

u ¼ xþ hdebðtÞ; ð10Þ

where x and hdeb are the argument of perigee and the trueanomaly of the debris orbit, respectively.

Given debris orbital parameters, r, i, u, and X, and loca-tion of observer, hLST and /gd in the geocentric coordinatesystem, IJK, the debris location in the topocentric coordi-nate system, ItJtKt is calculated from the Eqs. (5), (6),(8), (9) and (10). Since the debris location in the two-dimensional CCD coordinate system is supposed to be sim-ilar to the one in the coordinates of the topocentric localhour angle, (LHA)t and the topocentric declination, dt, itcan be derived from Eq. (7) considering the followingequation:

ðLHAÞt ¼ hLST � at: ð11Þ

Note here that (LHA)t is measured positively westward tothe celestial object, that is, in a left-handed system.

In this simulation, the observer is assumed to be on asite which is the same as Mt. Nyukasa observation facility,

Table 3The orbital parameters of the Titan 3C transtage. (The reference frame of the Earth-centered inertial (ECI) coordinates, the orbital model based on trueequator, mean equinox of epoch (TEME)).

The number of debris 9The epochs for TLE data 2011/2/14 16:07:02 – 2011/7/25 19:06:33 UTCInclination : i 8.5588–9.0317 degRAAN: X 324.97–327.99 degEccentricity : e 0.0012–0.0261Argument of perigee : x 6.5989–333.16 degMean anomaly 26.613–353.43 degSemi-major axis : a 41674 � 103 �42025 � 103 m

Fig. 13. The nine debris trajectories in a geocentric coordinate system.(The reference system is TEME.)

Fig. 14. The nine debris trajectories in a topocentric coordinate system.


and target debris are assumed to be from the Titan 3C tran-stage, whose orbital parameters are given as TLE data(Space-Track, 2011) summarized in Table 3.

In order to simplify the simulation, the debris orbits areapproximated to be circular ones, which implies that theorbital radius and the true anomaly, hdeb of the debris aredefined such that

r ¼ að1� e2Þ1þ e cos hdeb

¼ a ð* e ¼ 0Þ; ð12Þ

hdebðtÞ ¼ xkdebt; k ¼ 1; 2; � � � ; 9; ð13Þ

where a, e, and xkdeb describe the semi-major axis, the eccen-

tricity, and the mean angular rate of each debris, respec-tively. Also, the local sidereal time of the observer isdefined as time variables such that

hLST ðtÞ ¼ hGðtÞ þ kobs ¼ hLST 0 þ xEt: ð14Þ

As shown in Eq. (14), while the local sidereal time is de-fined by Greenwich sidereal time, hG and geodetic east lon-gitude, kobs, it can be reduced to the last term of the aboveequation using the rate of the Earth rotation, xE.

These debris trajectories in the geocentric coordinatesare shown in Fig. 13. They are plotted by using nine differ-ent markers (named d1 to d9) at hourly intervals fortwenty-four hours, in which the epoch for all the debrisorbital motion is set at 19:06:33, July 25, 2011 (UTC). As

shown in this result, all the trajectories are sinusoidal andsimilar with each other.

In contrast, the trajectories using two parameters in thetopocentric coordinate system are shown in Fig. 14. Theyare depicted by using the topocentric declination, dt andthe local hour angle, (LHA)t. The local hour angle isdepicted in the unit of degree in the left-handed system con-sidering the appearance in the CCD image coordinates.

As seen in the result of Fig. 14, all the trajectories seemto be distorted loops which remain in narrow areas of theobservation angles, and motion direction of each debris

Fig. 15. The exact debris trajectories in the original field of views.


appears to dynamically change in opposite directions asshown in the magnified view of the debris trajectory for“d7” in Fig. 14.

Focusing on one of these nine debris, three cameraimage motions corresponding to different observation areasin different time for the debris “d7” are applied to the pro-posed optical flow algorithm and the line-identifying tech-nique, respectively. Each image sequence contains fortybinarized image frames, which are synthesized by plottingthirteen-pixel point-like white debris images in the darkbackground considering the specifications of the CCDcamera used at Mt. Nyukasa observation facility.

Aiming at applying three characteristic patterns of debristrajectories, the center of the field of view is fixed at (i) Case1:((LHA)t,dt) = (303.0 deg,4.5 deg), (ii) Case2: ((LHA)t,dt)= (303.0 deg,4.0 deg), and (iii) Case3: ((LHA)t,dt) =(107.5 deg,28.5 deg). The start times for the observationare assumed to be 0, 1, and 2 h after the epoch of debris orbi-tal motion. The frame-to-frame time is fixed at 30 s to obtainsufficient changes of debris positions in the synthesizedimage frames, which implies 20 min observation duration.

Fig. 15 shows the exact locations for the simulated deb-ris motions in the original field of view, and Fig. 16 shows

Fig. 16. The exact debris traject

the same debris locations in the magnified view for eachcase. As seen in these figures, while the debris trajectoryfor Case 3 seems to be almost linear, the one for Case 2appears to be skewed linear and the one for Case 3 is obvi-ously curved.

Figs. 17 and 18 show the estimated flow vectors and thedetected lines as well as the corresponding object positionsfor both methods in the image coordinates. The scale foreach image coordinate is adjusted to fit the one for thetopocentric coordinates in Fig. 16. As shown in Fig. 17,despite the nonlinearity of the debris trajectories in Case1 and 2, the proposed method effectively estimates the flowvectors to fit them.

On the other hand, when the line-identifying techniqueis applied to the same image sequences, the object locationscorresponding to the detected lines seem to exactly depictthe debris locations as shown in Fig. 18. That is becausethe lines are detected for some sets of the objects on theplural lines tangential to the debris trajectory. For Case1, as seen in Fig. 18, the number of detected lines reachesto 631, and to estimate only one trajectory from these linesis not so straightforward as with the proposed optical flowalgorithm.

ories in the magnified views.

Fig. 17. The estimated debris flow vectors for the proposed method.

Fig. 18. The estimated lines for the line-identifying technique.


6. Discussion

As shown in the results of the previous sections, the pro-posed optical flow algorithm is as effective both in trajec-tory detection and in computational time as the previousline-identifying technique. However, the proposed methodis fundamentally different from the previous ones in that itfirstly estimates the velocity vector of the debris image’smotion, then detects the debris trajectory by tracking theobjects moving in the image sequence. Since the velocityvector of the debris image is estimated in a sequential man-ner, it has advantages to detect a trajectory other than thelinear ones, which can also lead to improved position pre-diction performance.

7. Concluding remarks

In this paper, an image tracking technique is proposedto detect debris images in ground-based observation imagesequences obtained of the GEO region. The proposedmethod is based on a computer vision theory called anoptical flow algorithm, which is different from the previ-ously proposed methods like a stacking method and aline-identifying technique. The effectiveness of the pro-

posed method was validated by applying to real image datasets and a feasibility study using synthesized imagesequences. In order to see the advantageous properties ofthe proposed method in detail, the method should beapplied to much more image data sets obtained in practicalobservation conditions.

Acknowledgements

The authors appreciate Dr. Yanagisawa of JAXA forsupplying the image data set obtained at Mt. Nyukasaoptical observation facility.

References

Chobotov, V.A. (Ed.). Orbital Mechanics, third ed AIAA, Reston, Va,2002.

Faugeras, O. Three-Dimensional Computer Vision: A Geometric View-point. The MIT Press, Cambridge (pp.189–196), 1993.

Horn, B.K.P., Schunck, B.G. Determining optical flow. Artificial Intel-ligence 17, 185–203, 1981.

Lucas, B., Kanade, T. An iterative image registration technique with anapplication to stereo vision, in: Proceedings of the Seventh InternationalJoint Conference on Artificial Intelligence (IJCAI), pp. 674–679, 1981.

Schildknecht, T., Musci, R., Ploner, M., Beutler, G., Flury, W., Kuusela,J., Cruz, J.D.L., Palmero, L.D.F.D. Optical observations of space


debris in GEO and in highly-eccentric orbits. Adv. Space Res. 34 (5),901–911, 2004.

Schildknecht, T., Ploner, M., Hugentobler, U. The search for debris inGEO. Adv. Space Res. 28 (9), 1291–1299, 2001.

Space-Track(2011) <https://www.space-track.org/perl/login.pl>.Vallado, D.A. Fundamentals of Astrodynamics and Applications, third ed

Microssosm Press and Springer, New York, 2007.

Yanagisawa, T., Kurosaki, H., Nakajima, A. The stacking method: thetechnique to detect small sizeof GEOdebris andasteroids, JAXA Researchand Development Report, JAXA-RR-07-032E, Tokyo, pp. 2–7, 2008.

Yanagisawa, T., Kurosaki, H., Nakajima, A. Activities of JAXA’sinnovative technology center on space debris observation, in: Pro-ceedings of the Twelfth Advanced Maui Optical and Space Surveil-lance Technologies Conference (AMOS), Poster presentations, 2009.

http://https://www.space-track.org/perl/login.pl

a debris image tracking using optical flow algorithm

Documents