Accelerated Convolutions for Efficient Multi-ScaleTime to Contact Computation in Julia

Alexander Amini, Berthold Horn, Alan Edelman

Department of Electrical Engineering and Computer ScienceMassachusetts Institute of Technology, Cambridge, MA

Abstract— Convolutions have long been regardedas fundamental to applied mathematics, physics andengineering. Their mathematical elegance allows forcommon tasks such as numerical differentiation tobe computed efficiently on large data sets. Efficientcomputation of convolutions is critical to artificialintelligence in real-time applications, like machinevision, where convolutions must be continuously andefficiently computed on tens to hundreds of kilobytesper second. In this paper, we explore how convolutionsare used in fundamental machine vision applications.We present an accelerated n-dimensional convolutionpackage in the high performance computing language,Julia, and demonstrate its efficacy in solving thetime to contact problem for machine vision. Resultsare measured against synthetically generated videosand quantitatively assessed according to their meansquared error from the ground truth. We achieveover an order of magnitude decrease in compute timeand allocated memory for comparable machine visionapplications. All code is packaged and integrated intothe official Julia Package Manager to be used invarious other scenarios.

I. INTRODUCTION

One of the most important concepts in machinevision [1], signal processing [2], and Fourier theory[3] is that of convolutions. Convolutions are used tocompute discrete derivatives in a variety of industrialapplications, such as computer graphics and geom-etry processing. Additionally, with the advent ofdeep learning, convolutions have gained a resurgenceof prominence, as Convolutional Neural Networks(CNNs) are used to compute optimized feature maps.Thus, because of their wide applicability, mathemat-ical elegance, and fundamental symmetry betweentime and frequency domains, convolutions are an es-sential part of any mathematical computing platform.

Two critical machine vision computations are timeto contact (TTC) and focus of expansion (FOE).Imagine an autonomous vehicle, with a cameramounted on its front, approaching a wall. The timeto contact (TTC) is defined as the amount of timethat would elapse before the optical center reachesthe surface being viewed [1]. This problem canintuitively be thought of as: how much time will passbefore the car collides with the wall? On the otherhand, the focus of expansion (FOE) will determinethe precise location on the image plane that thecamera is approaching (i.e., the point that wouldultimately collide first). TTC and FOE solutionsare critical for many robotic systems since theyprovide a rough safety control capability, based oncontinuously avoiding collision with objects aroundit.

The problem of accurately determining the TTCis particularly difficult because we are often notgiven any information of speed, or size of objectsin our environment. While the FOE can be coarselyestimated by looking for minima of the directionalderivatives from the video sequence, this would notgive us any information relating to velocity (andtherefore, the TTC).

In the following sections, this paper will outlinethe formulation of discrete derivatives and convolu-tions, specifically in the context of machine visionand time to contact. We then present our acceleratedconvolution approach (FastConv) and its implemen-tation in Julia, a dynamic programming languagefor high performance computing environments [4].We evaluate the performance of FastConv againstthe existing Julia machine vision toolset and otherstate-of-the-art tools. We further assess FastConvspecifically for the TTC problem by implementing

arX

iv:1

612.

0882

5v1

[cs

.CV

] 2

8 D

ec 2

016

a multi-scale approach to TTC, and evaluate theperformance against synthetically generated videodata. We show that the convolution operation canbe accelerated by an order of magnitude to decreasecompute time and allocated memory for comparablemachine vision applications over 10 times.

II. DISCRETE DERIVATIVES

Most fundamental machine vision techniques suchas time to contact [1], optical flow [5], and viewsynthesis [6] perform several convolutions on everyiteration. In the context of robotics, these techniquesare performed on a continuous video stream sampledsomewhere around the range of 10 − 30 Hz. Sincethese computations need to be done at such a highfrequency, it is imperative for the programminglanguage to perform these convolutions extremelyquickly.

A fundamental part of solving the problem ofTTC is computing the partial derivatives Ex, Ey, andEt. Since our images are sampled under a discreteinterval (determined by the number of pixels in eachdimension), it is impossible for us to compute theseas continuous derivatives. Instead, we need a way toestimate the derivative in each of these directions.

Consider a 2-dimensional (2D) image E(x, y)with a single vertical edge. Since an edge is an abruptchange in brightness intensity it is the most commonapproach to determining meaningful discontinuities.We would expect a large magnitude impulse at thex location of the edge when computing the partialderivative with respect to x (ie. Ex).

We state the partial derivative of an image E(x, y)with respect to x as: ∂E∂x = Ex. Similarly the deriva-tive with respect to y can be written as: ∂E

∂y = Ey.Subsequently, we define the magnitude and angle ofthe edge as:

|E| = Emag =√E2x + E2

y (1)

θ = Edir = tan−1

(EyEx

)(2)

Here, |E| represents the sum of squares magnitudeof the gradient at every pixel, and θ is the cor-responding direction of the gradient. The directionis estimated by using four quadrant inverse tangent

with the two directional derivatives. Note that θ ∈[−π, π]. Figure 1 gives an example of taking a singleimage (eg. of a bicycle) and computing the resultingpartial derivatives, Ex (1B) and Ey (1C). Accordingto equation (1) the magnitude is computed in 1D.Finally, the directions of the derivatives are computedand visualized in 1E, where the color representsthe direction (equation 2) of the edge at that point(following the color wheel on the top right corner).For example, a pixel colored red indicates a gradientat that location with angle of −π2 .

These partial derivatives are computed accordingto the limit definition of a derivative:

∂E(x, y)

∂x= lim

∆x→0

E(x+ ∆x, y)− E(x, y)

∆x

For images, a crude estimate would be to saythat min ∆x = 1 since the smallest unit of measurein our image, without going to the sub-pixel scale,is a single pixel. Which in turn, would allow usto conclude that the derivative estimate in the xdirection for our image could be approximated by:

∂E(x, y)

∂x= E(x+ 1, y)− E(x, y)

This operation amounts to applying a two cellmask k = [−1 , 1], in turn, at every position xalong column y, multiplying each masked elementby the mask entry, and summing the result. In thediscrete setting, this replicates a common signal pro-cessing calculation: convolutions. That is, directionalderivatives (in any direction) can be approximated bymodifying the kernel, k, above.

More specifically, given an image, E, of size(m,n) and kernel, k, we can define the spatialconvolution (⊗) as follows:

(E ⊗ k)(x, y) =

m−1∑i=0

n−1∑j=0

E(x+ i, y+ j)k(i, j) (3)

Figure 2 gives a visual outline of how a con-volution of an image with a 3 × 3 kernel mightbe performed. The kernel is applied as a slidingwindow that spans all possible areas of the inputimage, at each point the element-wise multiplication

is computed and summed, which is stored in theoutput at the location highlighted in purple.

Fig. 2. An illustration of an input image convolved with a 3kernel to produce an output at the center of the kernel.

Now, in order to approximate the partial derivativeof E along the x axis (Ex) for example, we can let:

k =1

2

[−1 1−1 1

]To find Ey, we use the transpose of the previouskernel. In the next section we detail some of mostcommon kernels that have been developed over theyears for the task of edge detection.

III. CONVOLUTIONS

Different variations of these kernels have beenstudied over the years to accomplish various tasksin machine vision [7]. In this section we explorethree pervasive kernels, which were developed in thecontext of computing image derivatives. Each kernelstructure has various benefits and disadvantages as-sociated to it.

Figure 3 shows a visual example of the fourdifferent types of kernels that have been developedfor edge detection through first derivatives.

Fig. 3. A visual description of 2D image edge detectionkernels. First and second dimensions are the top and bottomrows respectively. The different kernels are: 2× 2 Roberts (A),2× 2 Prewitt (B), 3× 3 Prewitt (C), and 3× 3 Sobel (D).

A. Roberts

The Roberts kernel (3A) uses diagonal edge gra-dients and is susceptible to fluctuations. This methodgives no information about edge orientation and isactually computing the gradient at the center of agroup of 4-neighbor pixels. Therefore, the resultingcoordinate system of the output derivatives is 45degrees away from the original Cartesian coordinatesystem.

B. Prewitt

The Prewitt 2 × 2 kernel (3B) tries to estab-lish a Cartesian coordinate system by defining two

Fig. 1. Gradient computations of a single image E(x, y) (A). Directional partial derivatives are computed along the x (B) and y(C) axes, along with the absolute magnitude (D). Directions of the gradient are visualized according to a circular colorbar rangingfrom −π to π (top right).

derivates that follow the x and y axes. However, thereis still a slight ambiguity with this kernel. Since thekernel is of size 2× 2 (like Roberts), the gradient iscomputed at the center of the 4 pixels. Therefore, wedo not know precisely where the result pixel shouldbe stored (ie. it is between two pixels).

Conversely, the Prewitt 3×3 kernel (3C) addressesthis issue through a well defined center (ie. at themiddle pixel). Due to the nature of the kernel, it isvery simple to implement but also very sensitive tonoise fluctuations.

C. Sobel

Finally, the Sobel 3×3 kernel (3D) is very similarto Prewitt kernels in that it has a well defined center.However, it applies twice the weight around thecenter pixels. This makes the Sobel kernel moresensitive to diagonal edges than the previous Prewittoperators.

IV. ACCELERATING CONVOLUTIONS

In this section, we will present a method tospeed up the computation of convolutions in a high-performance computing language, Julia [4]. We uti-lize a key Julia feature of multiple dispatch, in whichfunctions can be defined and overloaded for differentcombinations of argument types. Additionally, weextend our convolution algorithm to handle arbitraryn−dimensional convolutions, which is not supportedby the current Julia implementation. N− dimen-sional convolutions are implemented through auto-matic code generation of higher order convolutionfunctions using Cartesian.jl [8].

A. Optimization for Vision

Applications of convolutions in machine vision,image processing, and deep learning are unique inthe sense that they utilize relatively small kernel sizesthrough convolution. Previous sections provided ex-amples of computing first order derivatives with 2×2and 3 × 3 kernels, each containing a total of 4 and9 pixels respectively. Such kernels are many ordersof magnitude smaller than the size of the image. Forexample, today, it is standard for even the low-endmobile devices to possess cameras which capture 1megapixel (106 pixels) quality images. Thus, a 3×3

Sobel kernel is 106

9 ≈ 105 times smaller than a 1megapixel image.

Convolutions in deep learning applications exhibitsimilar properties. That is, CNNs are used in a va-riety of machine learning tasks, including characterrecognition [9], autonomous vehicle control [10], andmedical drug delivery [11]. The convolution kernelsleveraged in these networks (as described above forimage processing) are relatively very small, rangingfrom 2× 2 to 5× 5.

B. Convolutions in Julia

Historically, convolutions have been conceptual-ized many different ways. Although Equation 3 givesus one representation of convolutions by computingtwo summations, there have also been other represen-tations of convolutions built upon notions developedin fundamental signal processing. For example, theConvolutional Theorem [12] states that convolutionsin the time domain are equivalent to element-wisemultiplication in the frequency domain, and viceversa. This theorem can be summarized as follows:

E ⊗ k = F−1(F(E) ∗ F(k)

)(4)

where F is the Fast Fourier Transformation (FFT),and F−1 is the Inverse Fast Fourier Transformation(IFFT). In other words, the image is converted intothe frequency domain (using FFT), multiply, andconvert back to the time domain (using the IFFT).Complexity analysis of this algorithm reveals that itexhibits O(n log n) asymptotic behavior (where Eand k have n samples each). Theoretically, this in-dicates the FFT approach to computing convolutionswould out-perform the direct summation method out-lined in equation 3, which has complexity O(n2). Forthese reasons: (1) compact implementation and (2)lower computational complexity, this approach waschosen for the existing convolution implementationin the Julia software package.

However, while the FFT/IFFT approach to com-puting convolutions does exhibit low asymptoticconvergence, in many practical machine learningand machine vision scenarios, a far more efficientapproach can be provided. This is due to the factthat, when using kernels that are orders of magnitudesmaller than the raw signal (i.e., image), the over-head of the FFT/IFFT overshadows any efficiency

advantages gained through the transformation to thefrequency domain. Thus, to take advantage of thesmall kernels leveraged by machine vision and ma-chine learning, this study implements a new directmethod of convolution computation and evaluatesperformance results in these settings.

The goal of our implementation is to developa novel, hybrid implementation, which exploits thesmall kernel sizes where possible for low complexitycomputation, and leverages the existing FFT ap-proach only when dealing with large kernel sizes(which does not occur frequently). Furthermore, un-like many other comparable high performance com-puting languages, Julia does not have built-in func-tionality to compute n-dimensional convolutions. Wewill address this gap by extending our algorithmsinto an arbitrary n-dimensional implementation tohandle a much larger range of applications.

This study implements and evaluates an algorithm,FastConv, in Julia that aims to accomplish the goalsoutlined in Figure 4.

Fig. 4. A conceptual outline of the benefits of the imple-mentation developed in this paper (FastConv), to the currentimplementation that exists within Julia.

C. FastConv

Due to the high overhead in converting a signalinto the frequency domain, the algorithm starts byevaluating the kernel size. If the kernel size isrelatively small, the convolution will be performed ina direct manner (ie. via equation 3 for 2D); otherwisethe existing FFT method will be used. Kernel sizechecking incurs a small increase in computation, butavoids requiring the user to assess which method touse, and is dwarfed by the computations required forboth the direct and the FFT methods.

FastConv [13] employs use of existing base Ju-lia features (such as multiple dispatch, efficientmemory allocations, and macros to generate multi-dimensional code). The package, which has beenadded to the official Julia Package Manager, achieves

over an order of magnitude speedup for small kernelapplications (such as machine learning and machinevision), e.g., when convolving a standard sized imagewith a 3 × 3 kernel. By allocating the memory ina helper function, the algorithm also improves effi-ciency by reusing memory within the actual convolu-tion function. Additionally, we use @inbounds toeliminate array bounds checking, further increasingperformance speedups.

Our implementation covers a wide variety ofsignal types (real, complex, boolean, irrational, un-signed integers, etc) through native Julia multipledispatch. Furthermore, we utilize the Cartesian pack-age to create auto generative code that can computeconvolutions in any dimension. The process by thecompiler is as follows:

1) The dimensionality of the inputs is identified(let’s say both inputs are of dimension k)

2) The k-dimensional convolution code is gener-ated (if it has never been before)

3) The inputs are processed by the auto-generatedcode.

D. Benchmarking

To compare the performance of select program-ming languages at computing convolutions, a simplebenchmark, designed to represent machine visionapplications, was conducted. A random 1MP imagewas convolved with 4 different sized kernels. Fig-ure 5 shows the time taken to convolve these twosignals using the standard convolution function inthree different computing platforms (conv in Julia,Matlab, Octave or numpy.convolve in Python).

The number of elements present in the input imagewas maintained for each test (ie. only the languageand kernel size was varied). Since Julia does notsupport for convolutions of dimension three andabove, performance was compared to Julia only for1D and 2D inputs.

FastConv achieved well over an order of magni-tude speedup over the convolution function in thecurrent Julia version (v0.5), and performed compa-rably to Python, Matlab and Octave implementationsfor these kernels. Additionally, we evaluated ouralgorithm against the FFT based approach for kernelsof larger size. Figure 6 plots the time taken tocompute the convolution across varying kernel sizes

2 3 4 5

Kernel Length

0

1

2

3

4

5T

ime (

linear)

×10 -3 1D convolutions

Julia v0.5 Octave Python Matlab FastConv

2 3 4 5

Kernel Length

10 -3

10 -2

10 -1

10 0

Tim

e (

log)

2D convolutions

Fig. 5. Time taken to compute E ⊗ k across three differentcomputing platforms (Julia, Matlab, and Python). For eachlanguage the built in convolution function was used. Note thatJulia does not have base support for convolutions of arbitrarydimension (above 2D).

for both the pre-exsiting Julia implementation andFastConv.

0 10 20 30 40 50

Kernel Size

0

2

4

6

8

Co

mp

ute

Tim

e

×10 -3

Common MV

FFT

Direct

Fig. 6. Convolution computation time for both the direct (red)and FFT (blue) method across varying kernel sizes. FastConvuses a threshold on the kernel size to determine when computeaccording to either method. Frequently used machine visionkernel sizes (as described in the main text) are highlighted inblue background.

For the majority of machine vision (MV) ap-plications (shaded region), FastConv significantlyoutperforms the FFT implementation. As the kernelsize increases (kernel size ≈ 30) it becomes moreefficient to use an FFT based convolution algorithm.Therefore, in the published package of FastConvwe utilize the best of both worlds. When belowthis threshold (determined experimentally), whichis common in the realm of machine vision, the

algorithm automatically uses the direct method ofcomputing convolutions. However, for large kernelsizes, the algorithm switches to use the FFT basedimplementation.

V. TIME TO CONTACT

In order to test how FastConv might be used inthe context of machine vision, we also implementedthe time to contact algorithm in Julia. This sectionwill briefly outline the details of the algorithm wefollow to determine precisely the focus of expansion(FOE) and resulting time to contact (at the FOE) [1].

A. Formulation

We can define the brightness of an image sequenceover time as E(x, y, t), where x and y are Cartesiancoordinates across the image and t is the tempo-ral operator. For example, E(x, y, 0) is the imageE(x, y) at time t = 0. Figure 7, illustrates how thissequence can be visually represented as “stacking”images on top of one another to capture the temporalvariability. Studying how a certain pixel, or group ofpixels, vary over a temporal slice of the dataset cangive us insights into optical flow and time to contact.

Fig. 7. A sequence of four images of the MIT dome, zoominginto the dome as time progresses. Images vary spatially accord-ing to x and y, while time is represented along the temporal (ie.t) axis.

To start, assume that as a point moves throughthe scene, its brightness value (in E) does not varysignificantly. [5] Therefore according to the chain

rule for differential calculus, we get:

d

dtE(x, y, t) = 0

∂E

∂x

dx

dt+∂E

∂y

dy

dt+∂E

∂t= 0

Exdx

dt+ Ey

dy

dt+ Et = 0

uEx + v Ey + Et = 0 (5)

where Ex and Ey are the derivatives of the imagein the x and y directions respectively.

In order to gain a better understanding of the wayobjects are situated in our environment compared towithin our images, we can establish the following re-lationship of perspective projection. Given a opticalcenter (or center of projection) with focal length f ,we can use the rule of similar triangles to conclude:

x

f=X

Z,

y

f=Y

Z(6)

where X,Y, Z are the positions of a point inspace, and x, y is the position of that same pointprojected onto the image plane. Figure 8 gives avisual representation of a scene (3D) being projectedon to an image plane (2D), and the proportionalrelationship between any point in the scene to itscorresponding point in the image (ie. two equationsabove).

Differentiating equations (6) with respect to timeyields:

u

f=

1

Z

(U − x

fW

),

v

f=

1

Z

(V − y

fW

)(7)

where (U, V, W ) are the time derivatives of(X, Y, Z) respectively.

This can be rewritten in terms of the focus ofexpansion (x0, y0) :

u = −WZ

(x− x0), v = −WZ

(y − y0)

In other words, define the focus of expansion andtime to contact as follows:

x0 = −AC

y0 = −BC

TTC =1

C(8)

Fig. 8. Diagram illustrating how 3D object in the environmentcan be projected into a 2D image plane. From the point of viewof the center of projection (eye), points in the image plane andin the environment form similar triangles that we use to definethe perspective projection equations.

where A = f UZ , B = f VZ , and C = WZ . Finally,

equations (7) can be substituted into the originalformula for constant optical flow, equation (5). Aftersimplifying:

(A+ Cx)Ex + (B + Cy)Ey + Et = 0

AEx +BEy + C(xEx + yEy) + Et = 0

AEx +BEy + CG+ Et = 0

with G = xEx + yEy representing a measure ofthe “radial gradient”. The least squares optimizationproblem is formulated as follows:

minA,B,C

∫∫(AEx +BEy + CG+ Et)

2 dx dy

Taking derivatives with respect to A, B and Cindividually and setting equal to zero, a system ofthree linearly independent equations can be obtainedand written in matrix form as:

∫∫E2

x

∫∫ExEy

∫∫GEx∫∫

ExEy

∫∫E2

y

∫∫GEy∫∫

GEx

∫∫GEy

∫∫G2

ABC

=

− ∫∫ExEt

−∫∫

EyEt

−∫∫

GEt

Finding the inverse of the 3× 3 matrix on the left

enables solving for the vector of interest: [ABC]T .

Once these values are determined, the TTC and FOEcan be computed by plugging into the respectiveequation (8) and finding [x0, y0, TTC].

B. Results

The TTC and FOE determination algorithm de-scribed in the previous subsection was also im-plemented and integrated into the official PackageManager built into Julia. We tested our implemen-tation across a variety of synthetic videos that weregenerated by progressively “zooming” into randompoints on the image, each time performing a cubicinterpolation maintain image size.

Given any input video pointer (ie. pointer tolocal video, or to webcam), the algorithm movesthrough the data to progressively determine the timeto contact at that instant, along with the location thatwill collide first (ie. FOE). Figure 9 shows the resultsof the TTC algorithm compared to ground truth datafor a certain video sequence using 5 different filteringmethods to enhance results.

A common disadvantage of having an image witha very large number of pixels is that it becomesincreasingly harder to mitigate the effects of noiseand distortions. To combat this, and reduce noise wecan down sample each image by filtering it with ablock averaging filter, and then passing the resultthrough a low-pass filter. This process will create asmaller image, with inevitably less noise. We cancontinue this further, each time removing more andmore noise (but also loosing more and more imagedetail). The five methods shown in Figure 9 representdifferent levels of down-sampling. In other words,“No Filter”, means the original images were used asinputs to the TTC algorithm, while “Sample” 2, 4, 5represent the act of down sampling the inputs 2, 4,5 consecutive times before even passing them to theTTC algorithm.

We observe that filtering certainly helps up to acertain point, which may be hard to determine at firstglance. However, a “Multi-Scale” technique can alsobe employed to progressively run a search over thesampling space to identify the best possible amountto down sample at every video frame. For example,we start with no sampling, compute the expectedTTC, down-sample and recompute the TTC. If theestimate improved (ie. reduced) then we continue to

down sample until no additional progress is made.

This technique achieves significantly better resultsthat ad hoc sampling of the entire video sequence,and requires no additional prior knowledge aboutthe environment, or ground truth TTC. These resultsare illustrated by the green line on the left plotof Figure 9, while the respective mean squarederror (MSE) for each of the methods is shown onthe right. Figure 9 illustrates the multi-scale TTCalgorithm significantly outperforms any of the staticscaling methods, but also requires more computationto progressive search through the sampling space.Testing our algorithm on a synthetically generatedstandard definition video sequence, we are able toobtain real-time performance of both the originalTTC algorithm (≈ 30Hz) as well as the multi-scalevariant (≈ 10Hz).

VI. FUTURE WORK

We are currently capturing video data from multi-ple cameras mounted on a self-driving automobilein the MIT Distributed Robotics Laboratory. Thisdata presents new challenges, such as multiple pointsof interest, and potentially excessive noise (snow,rain, reflections, etc), and will enable extending andevaluating our algorithms against real-world drivingconditions. We are also extending our implementa-tion to automatically detect separable kernels (i.e.,matrices that are capable of being represented asthe multiplication of two vectors) and compute theconvolution piece-wise over each segment of theseparable kernel. This would reduce the complexityof the direct algorithm from O(n2) to O(n), andbring additional performance gains to our imple-mentation. Separable kernels are a special set ofkernels that appear frequently in machine vision, as(1) directional derivatives (2) Gaussian smoothingfunction, and (3) block averages.

Additionally, we aim to extend our implemen-tations for applications outside of machine vision.Specifically, we plan to extend our direct method ofcomputing n-dimensional convolutions into the FFTmethod, in order to handle convolutions with largekernel sizes.

0 20 40 60 80 100 120 140

Time (frames)

0

50

100

150

200

250

300

350

400

Pre

dic

ted T

TC

Predicted vs Ground Truth

No Filter

Sample 2

Sample 4

Sample 5

Multi-Scale

Ground Truth

No Filter Sample 2 Sample 4 Sample 5 Multi-Scale

Method

0

0.5

1

1.5

2

2.5

3

Mean S

quare

Err

or

×10 4 Error Through Filtering

Fig. 9. Predicted time to contact vs the ground truth for a synthetically generated video sequence (left). Five down samplingmethods are used, from none to five consecutive down samplings. The right subplot illustrates the mean squared error for each ofthe methods used.

VII. CONCLUSION

In this paper, we implement a fast N-dimensionalconvolution algorithm, optimized specifically for ma-chine vision applications, and integrate into the highperformance computing Julia platform. We showthat our algorithms achieve an order of magnitudeperformance improvement over the existing imple-mentation.

Furthermore, we develop a real time implemen-tation of time to contact and focus of expansiondetermination using a single stereo video sequence.We also evaluate our accelerated convolution imple-mentation in the context of real-time TTC and FOEdetermination. In autonomous systems, TTC andFOE provide a low complexity method to estimatethe danger of their trajectory and trigger evasive ma-neuvers as needed. Our accelerated, low complexity,and reduced memory implementation is especiallyattractive for resource-constrained embedded devicesand autonomous vehicle control.

Results are measured against synthetically gen-erated videos and quantitatively assessed accordingto their mean squared error from the ground truth.Utilizing multiple scales of resolution we show thatit is possible to drastically improve the overall per-formance of the TTC estimation.

Finally, we packaged and published our code[13], [14], for both the accelerated convolutions andthe TTC algorithm, into the official Julia PackageManager, allowing our results to be leveraged on anyJulia capable device with only a few lines of code.

REFERENCES

[1] Horn, Berthold KP, Yajun Fang, and Ichiro Masaki. “Timeto contact relative to a planar surface.” IEEE intelligentvehicles symposium. 2007.

[2] Philips, Charles L., John M. Parr, and E. Riskin. Signals,systems, and transforms. Prentice Hall, 1995.

[3] Champeney, David C. A handbook of Fourier theorems.Cambridge University Press, 1987.

[4] “The Julia Language.” Julia. http://julialang.org[5] Horn, Berthold KP, and Brian G. Schunck. “Determining

optical flow.” Artificial intelligence 17.1-3 (1981): 185-203.

[6] Pollard, Stephen, et al. “View synthesis by trinocular edgematching and transfer.” Image and Vision Computing 18.9(2000): 749-757.

[7] Shrivakshan, G. T., and C. Chandrasekar. “A comparisonof various edge detection techniques used in image pro-cessing.” IJCSI International Journal of Computer ScienceIssues 9.5 (2012): 272-276.

[8] Holy, Tim, et al. “Cartesian.jl: Fast multidimensional al-gorithms for the Julia language.” Github.

[9] LeCun, Yann, et al. “Backpropagation applied to handwrit-ten zip code recognition.” Neural computation 1.4 (1989):541-551.

[10] Bojarski, Mariusz, et al. “End to End Learning for Self-Driving Cars.” arXiv preprint arXiv:1604.07316 (2016).

[11] Wallach, Izhar, Michael Dzamba, and Abraham Heifets.“AtomNet: A Deep Convolutional Neural Network forBioactivity Prediction in Structure-based Drug Discovery.”arXiv preprint arXiv:1510.02855 (2015).

[12] Arfken, G. “Convolution theorem.” Mathematical Methodsfor Physicists,: 810-814.

[13] Amini, Alexander, et al. “FastConv.jl: Accelerated Convo-lutions for the Julia language.” Github.

[14] Amini, Alexander, et al. “TimeToContact.jl: Real-timeTime to Contact algorithm for the Julia language.” Github.