Virtual CinematographyUsing Optimization and Temporal Smoothing

Alan LittenekerUniversity of California, Los Angeles

alitteneker@cs.ucla.edu

Demetri TerzopoulosUniversity of California, Los Angeles

dt@cs.ucla.edu

ABSTRACTWe propose an automatic virtual cinematographymethod that takesa continuous optimization approach. A suitable camera pose or pathis determined automatically by computing the minima of an ob-jective function to obtain some desired parameters, such as thosecommon in live action photography or cinematography. Multipleobjective functions can be combined into a single optimizable func-tion, which can be extended to model the smoothness of the optimalcamera path using an active contour model. Our virtual cinematog-raphy technique can be used to find camera paths in either scriptedor unscripted scenes, both with and without smoothing, at a rela-tively low computational cost.

CCS CONCEPTS• Computing methodologies→ Computer graphics; Anima-tion; • Mathematics of computing→ Continuous optimization;

KEYWORDSvirtual cinematography, optimization, camera motion

ACM Reference format:Alan Litteneker and Demetri Terzopoulos. 2017. Virtual CinematographyUsing Optimization and Temporal Smoothing. In Proceedings of MiG ’17,Barcelona, Spain, November 8–10, 2017, 6 pages.https://doi.org/10.1145/3136457.3136467

1 INTRODUCTIONThe problem of virtual cinematography is one of the oldest in 3Dcomputer graphics. The settings chosen for a virtual camera in-trinsically affect both what objects are visible and how the viewerperceives them. In general, the goal of any declarative virtual cine-matography system is to automatically select camera settings thatmatch a user specified set of parameters for the final renderedframe.

However, the demands of even the simplest of photographicand cinematographic techniques require sophisticated parametersthat may be difficult to quantify at best [Datta et al. 2006]. Giventhe inherent subjectivity of these art forms, users may frequently

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from permissions@acm.org.MiG ’17, November 8–10, 2017, Barcelona, Spain© 2017 Copyright held by the owner/author(s). Publication rights licensed to Associa-tion for Computing Machinery.ACM ISBN 978-1-4503-5541-4/17/11. . . $15.00https://doi.org/10.1145/3136457.3136467

disagree both on how well a frame matches a particular aesthetic,and on the relative importance of aesthetic properties in a frame.

As such, virtual cinematography for scenes that are plannedbefore delivery to the viewer, such as with 3D animated films madepopular by companies such as Pixar, is generally solved manu-ally by a human artist. However, when a scene is even partiallyunknown, such as when playing a video game or viewing a pro-cedurally generated real-time animation, some sort of automaticvirtual cinematography system must be used.

The type of system with which the present work is concerneduses continuous objective functions and numerical optimizationsolvers. As an optimization problem, virtual cinematography hassome difficult properties: it is generally nonlinear, non-convex,and cannot be efficiently expressed in closed form. Furthermore,ensuring smoothness in camera paths can amplify the difficulty ofthis problem as the dimensionality of the search increases.

2 RELATEDWORKAs previously mentioned, a diverse set of systems have been devel-oped in this field [Christie et al. 2005]. The following are some ofthe projects most relevant to our work:

A number of systems have been developed that employ contin-uous optimization methods [Drucker and Zeltzer 1995; Gleicherand Witkin 1992]. Issues such as the sensitive nature of the shotweighting and difficulty with over-constraining led to the use ofstochastic optimization [Abdullah et al. 2011; Burelli et al. 2008], aswell as the development of systems based on boolean constraints[Bares et al. 2000] and logical reasoning [Funge et al. 1999]. Someresearchers have even attempted to control in-game cameras, withvaried success [Burelli 2012; Halper et al. 2001].

There have also been more lightweight methods, such as se-lecting shots from predefined libraries of geometry configurations[Elson and Riedl 2007], or by assuming preprocessed elements ofenvironment geometry will remain constant over the motions of ascene [Lino et al. 2010].

Efficiently producing occlusion-free and collision-free cameraposes or paths has proven problematic for several researchers. Cam-Droid [Drucker and Zeltzer 1995] supported the avoidance of col-lisions and occlusions through an expensive preprocessing of thescene geometry, which required the scene to remain static whilethe camera moved.

More recent work has taken a purely stochastic approach toocclusion satisfiability [Ranon and Urli 2014] or omitted supportfor collisions and occlusions altogether [Lino and Christie 2015],instead focusing on efficiency over a limited set of shot parameters.While much of this work has been restricted to static poses or simpleinterpolated paths, there has been some work on path smoothnessthus far limited to specific types of splines or scene actions. [Assaet al. 2008; Galvane et al. 2015]

MiG ’17, November 8–10, 2017, Barcelona, Spain Alan Litteneker and Demetri Terzopoulos

3 VIRTUAL CINEMATOGRAPHY SYSTEMOur prototype virtual cinematography system operates based on acontinuous objective function that corresponds to a user-specifiedset of shot parameters. When operating on a moving scene, thisfunction must be extended to handle variable parameterization aswell as the modeling of smoothing. Once this objective functionhas been fully constructed, our system attempts to minimize thefunction using standard optimization techniques. The system isintegrated with Unreal Engine 4 for rendering and user input.

3.1 Objective FunctionsTo find a camera pose that matches the desired shot parameters, oursystem must be able to evaluate how well a particular camera posematches the given shot parameters. This is done with an objectivefunction f : RN 7→ R for each shot parameter. To simplify thenotation, these functions will be denoted as f (x) despite requiringdifferent numbers and types of parameters.

Multiple shot parameters are combined into a single objectivefunction f (x) =

∑ni=k αk fk (x) as a weighted sum of the their as-

sociated objective functions, a strategy that critically allows forcompromises between shot parameters for which the minima donot precisely intersect. The influence of each parameter can betuned by adjusting the values of α .

Evaluating the same functions over an entire camera path re-quires the parameterization of x over time to x(t), which can beevaluated as

∑i f (x(ti )) over the set of discrete keyframes.

3.2 Temporal SmoothingTo model the inertia and momentum viewers expect from real-world cameras, we extend the user-specified temporal objectivefunction with an active contour model [Kass et al. 1988]. In thisaugmented model, our system attempts to minimize the energyfunctional

E(x(t)) = Eint(x(t)) + Eext(x(t)) (1)comprising an internal energy

Eint(x(t)) =12

∫α | Ûx(t)|2 + β | Üx(t)|2 dt , (2)

where α and β are given constants and the overstruck dots denotedifferentiation with respect to t , and an external energy

Eext(x(t)) = f (x(t)). (3)

Determining suitable values for α and β generally requires ex-perimental parameter tuning. If either is set too high, the resultantcamera path may violate the desired parameters, while too low asetting may fail to alleviate the undesirable artificiality. It was foundexperimentally that α = 0.01 and β = 0.2 produced satisfactoryresults.

3.3 Optimization TechniquesGiven the nonlinear nature of many of our objective functions,finding a solution analytically is impossible for any moderatelycomplex parameter set. As long as a given objective function isconvex, a simple gradient descent (GD) algorithm can be used.However, this is generally inadequate as the underlying objectivefunctions that must be optimized are likely to be non-convex andtherefore cause our system to be caught in local minima. Simulated

annealing (SA) provides a potential solution. While even quenchedSA is less sensitive to local minima, the solutions it produces arefar less precise than those found by GD when the latter does findthe global minimum. By running an SA optimizer before a GDoptimizer, the robustness of the former can be combined with theprecision of the latter.

Optimizing for camera paths is a straightforward extension ofoptimizing for static poses. Without smoothing from the activecontour model, a temporal objective function can be optimized byrunning the optimization algorithms locally on each frame. Op-timizing with smoothing is somewhat more involved. Each timeinterval must first be initialized with a locally computed roughoptimum that ignores the smoothing, then gradient descent can beperformed such that every component of ∇E(x(t)) is computed forevery time step t before any new values can be set.

3.4 Use CasesWhile there are many different ways of utilizing the cinemato-graphic tools outlined above, our system supports two primarycategories of use cases, which are labeled as scripted and unscriptedscenarios.

3.4.1 Scripted Scenarios. In a scripted scenario, our system at-tempts to find a satisfactory camera path for a preplanned scenebefore any part of it is to be displayed to the viewer. As this is anoffline operation, our system can take as much time as is necessaryto find a satisfactory solution. Furthermore, complete knowledge ofall past, current, and future states of the scene are available at everyinstant of time. This is analogous to the traditional preproductionworkflow of a live action or animated film.

Scripted scenarios can be solved in a straightforward manner bythe optimization strategy described in Section 3.3.

3.4.2 Unscripted Scenarios. In an unscripted scenario, the goalof our system is to find a camera pose for the current instant intime given complete knowledge of the current and past states ofthe scene, but without any direct knowledge of the future. This isan online operation during the playback of the scene; therefore, oursystemmust be able to find a satisfactory solution in real time giventhe viewer’s desired frame rate. This is analogous to the shootingof an unscripted documentary or news material, or to video gamecamera control.

Without smoothing by an active contour model, the first framecamera pose is set either by the user or by running SA, then GD isrun every frame starting from the values of the previous frame.

Unscripted scenarios with smoothing are significantly trickier,since in order for the active contour model to work effectively, oursystem must know something of the unknown future. As our cur-rent system is lacking contextual or behavioral data, as employed by[Halper et al. 2001] to form predictions more precisely, the predic-tion scheme currently supported is a Taylor series approximationwhere, for future keyframes t1, . . . , tn ,

x(tj ) ≈N∑k=0

1k!(tj − t0)

k dkx(t0)dtk

, (4)

where dkx(t)/dtk is the k th-order derivative of x(t). To alleviateinstability stemming from discrete time steps, our system calculates

Virtual Cinematography Using Optimization and Temporal Smoothing MiG ’17, November 8–10, 2017, Barcelona, Spain

the necessary derivatives as local averages

dkx(t)dtk

≈1

M∆t

M∑i=0

dk−1x(t − ∆t i)

dtk−1−dk−1x(t − ∆t (i + 1))

dtk−1, (5)

where ∆t = ti −ti+1. Of course, this is a tradeoff as too large a valueofM can create an undesirable sense of variable inertia. Values ofM = 10 and ∆t = 0.1 sec were used.

Once these predictions are made, the same types of optimizationtechniques can be used to find a suitable camera path through thepredicted period, and the first future keyframe x(t1) can be used asthe next camera state.

4 PREDEFINED SHOT PARAMETERSWhile any valid objective function can be input by the user, our sys-tem supports common cinematic and photographic compositionalaesthetics by providing a set of predefined shot parameter types,modeled as novel objective functions. These mimic the types of shotdescriptions commonly used on “shot lists” in the preproduction ofa film by live action directors and cinematographers.

While the components of our system integrated from the UnrealEngine rely on its particular coordinate system (left-handed, z-up,scale, etc.), the functions listed here are written to be as genericas possible. As several rules require the projection of a point ontothe image plane, the function Π(R3) → R2 is used to denote such aprojection from world space to screen space, where any point p thatwill appear in frame will have |v | ≤ 1,∀v ∈ Π(p). Note that thisprojection function must have knowledge of the current cameraparameters (e.g., position, orientation, field of view, etc.), which arenot explicitly listed as parameters for many functions.

All of the functions described below are point-based, meaningthat all details of the scene are collected in the form of observingthe position, orientation, etc., of specific points connected to objectsof interest in the scene. This requires that complex elements in thescene be represented by multiple points. This point-based modelvaries from the explicit geometric models of [Bares et al. 2000;Jardillier and Languénou 1998], where each camera pose is evalu-ated based on observations of geometric primitives approximatingcomplex objects, as well as from the rendering-based methods of[Abdullah et al. 2011; Burelli et al. 2008], where each camera poseis evaluated by analyzing a full rendering of the scene from thatpose.

4.1 VisibilityAn object the user desires to be visible must appear on-screen,but this apparently simple behavior requires two different typesof parameters, denoted frame bounds and occlusion, both of whichmust be satisfied for an object to be visible.

4.1.1 Frame Bounds. The frame bounds rule is simply that anypoint that the user desires to be visible must appear inside thebounds of the frame. In many circumstances, some points are com-monly required to be inset from the edge of the frame, a propertycommonly called headroom in live action cinematography. For hu-man subjects, too much room between the top of the head and theedge of the frame causes the subject to appear small and insignif-icant, while too little causes the subject to appear restricted andboxed in [Brown 2013, p. 52], as is shown in Figure 1.

Figure 1: A frame that was specified to have too much head-room, as chosen by our system.

Given world space point p and constants xlow < xhigh, where−xlow = xhigh = 1 ensures a point is simply in frame and 0 <(|xlow |,xhigh) < 1 will produce headroom, the frame-bounds rulecan be evaluated as

f (p) =∑

x ∈Π(p)

д(x), (6)

where

д(x) =

(x − xlow)

2 x < xlow;0 xlow ≤ x ≤ xhigh;(x − xhigh)

2 x > xhigh.

(7)

4.1.2 Occlusion and Collision. The occlusion rule is simply thatthere must be an unoccluded line of sight from the camera to anobject that the user desires to be visible. A common distinction ismade between clean shots where all of an object is unoccluded anddirty shots where only part of object is unoccluded.

Similarly, the collision rule simply states that the camera mustnot collide with scene elements. Just as it is generally undesirablefor real world cameras to pass through walls and actors, users ofvirtual cameras generally expect their cameras not to pass throughapparently solid virtual matter.

Unfortunately, building suitable objective functions for occlu-sion and collision is much more complicated. Unlike all of the otherfunctions described, these parameters represent an observationabout the geometry of the scene as a whole and therefore cannotbe defined in the same concise analytic style used elsewhere. Fur-thermore, there will be a discontinuity in any direct observationof visibility at the boundary of occlusion and collision. To addressthese issues, our system provides an n-dimensional linear inter-polator that, given camera position c and point position p, can beexpressed as

f (c) = д3(c), (8)

where

дi (c) =cai − ci

cbi − caiдi−1(cb ) +

1 − cai + ci

cbi − caiдi−1(ca ). (9)

MiG ’17, November 8–10, 2017, Barcelona, Spain Alan Litteneker and Demetri Terzopoulos

Figure 2: Examples of a close-up shot (left) and a wide shot(right), both chosen by our system.

Here, ca and cb are the closest sample locations on axis i such thatcai ≤ ci and cbi ≥ ci and caj = cbj = cj ,∀j , i , and

д0(c) = B(c, p), (10)

where

B(c, p) =

{1 if the path from c to p is occluded;0 if the path from c to p is not occluded.

(11)

As this function is not analytically differentiable, differentiation iscomputed numerically as df (x)

dx ≈f (x+∆x )−f (x)

∆xwith ∆x = 1 cm.

To improve smoothness, our system supports the convolution ofthe samples of the interpolator by a Gaussian kernel, which can beexpressed mathematically as changing д0(c) to

д0(c) =1

√2πσ 2

h3(c). (12)

Here,

hi (c) =K/2∑

j=−K/2e−

(j∆i )2

2σ 2 hi−1(c+j∆iUi ), (13)

where ∆ is the set of convolution step sizes, U is the set of basesfor c, and h0(c) = B(c, p). Values K = 7, σ = 1, and ∆ = 25 cm areused. Of course, the convolution requires more samples than itsunconvolved counterpart, decreasing efficiency.

Note that in all uses, B(c, p) is sampled in a uniform grid withsteps that are specified at function construction. Samples in the cur-rent search neighborhood can be fetched on demand, then cachedin a hashmap for efficient retrieval in later computations. This ap-proach differs from [Drucker and Zeltzer 1995] in that it requires nopreprocessing of the scene geometry, does not require the occludingor colliding geometry to remain static, and can scale to scenes ofany size, given a utility for sampling B(c, p) that is equally efficientin scenes of any size.

4.2 Shot SizeThe amount of space an object occupies in the frame reflects itsimportance in the scene. For human subjects, this is usually ex-pressed in terms of how much of a particular person is in frame.For example, a view of a person’s entire body, commonly calleda wide or long shot, is frequently used to establish the relativegeometry of objects in a scene, while a view of only their head andshoulders, commonly called a close-up shot, is often used to showdialog [Bowen and Thompson 2013, p. 8–11]. Examples of both areshown in Figure 2.

While there are several different ways of evaluating this ruleusing projective geometry, most tend to suffer from severe instabil-ity given even moderately suboptimal inputs. Given object points

Figure 3: Examples of a shot from a low vertical angle (left)and a profile angle (right) chosen by our system.

a and b, camera position c, and angle θ , the most stable objectivefunction1 we have found is

f (a, b, c,θ ) =(θ − arccos

(θ −

(a − c) · (b − c)| |a − c| | | |b − c| |

))2. (14)

As described above, the object points a and b are chosen to be theedges of the object to be in frame (e.g., top of head and mid chest fora close-up shot). For most purposes, the value of θ can be chosento be the camera’s vertical field of view. However, when headroomis desirable for the same points (see Section 4.1.1), this must bedecreased somewhat so as to avoid unnecessary parameter conflict.

4.3 Relative AnglesThe angle from which an audience are shown a particular objectcan dramatically change the way that an object or character isviewed. This can generally be divided into two categories, verticaland profile angles, examples of which are shown in Figure 3.

Objects, and people especially, appear more powerful, intimidat-ing, or ominous when viewed from below, and conversely moreweak, frightened, or small when viewed from above. This relativevertical angle is often used to demonstrate the changing relation-ships between characters in a story, making each relative rise ordecline visually clear to the audience [Bowen and Thompson 2013,p. 34–39].

The way in which people are perceived is strongly affected bythe angle of view relative to the direction in which they are looking,or profile angle. In general, characters that are viewed directly fromthe front are felt to be more connected to the viewer than charactersviewed in profile, mimicking the way humans tend to face directlytowards things uponwhich they are focused [Bowen and Thompson2013, p. 40–43].

Given object point p, camera position c, unit object up directionu, and desired angle θ relative to unit object look direction d, theobjective function for vertical angle is

f (p, c, u,θ ) =(θ − arccos

(p − c

| |p − c| |· u

))2, (15)

while the profile angle function is

f (p, c, u, d,θ ) =(θ − arccos

(p − u((p − c) · u) − c

| |p − u((p − c) · u) − c| |· d

))2.

(16)

1For example, the function f (a, b) = (1− |Π(a)y −Π(b)y |)2 is highly unstable, wherethe subscript denotes the y component of the vector.

Virtual Cinematography Using Optimization and Temporal Smoothing MiG ’17, November 8–10, 2017, Barcelona, Spain

Figure 4: An example of a frame specified for an actor not tohave adequate look space, as chosen by our system.

4.4 Rule of ThirdsA standard rule of thumb in photographic composition is to placeimportant objects near the third lines, both vertical and horizontal,in the frame. The most important objects in the scene are oftenframed nearest the intersections of these lines [Brown 2013, p. 51].All the example frames shown in this section conform to this pa-rameter.

Given a point p in world space as well as constants 0 ≤ x0 ≤ 1and 0 < a ≤ 1, this parameter is modeled as

f (p) =∑

x ∈Π(p)

(x + b)2

(x + b)2 + a+

(x − b)2

(x − b)2 + a(17)

where

b =

√√√√√2(x20 − a

)+

√4(x20 − a

)2+ 12

(x20 + a

)26

. (18)

To use the standard third lines, x0 should be set to 1/3. Selecting avalue for a is a bit trickier, as too high a setting causes the functionto affect only points in the immediate vicinity on-screen whiletoo low a setting can cause the penalty for |x | < x0 to decreasesignificantly. It was found experimentally that a = 0.07 producedsatisfactory results.

4.5 Look SpaceA common technique used to balance a shot is to give substantialspace between a character and the frame bound in the directionthey are looking, a property that has been given numerous names inthe cinematographic literature, including “look space”, “lead space”,“nose room”, “action room”, etc. When this space is not provided,as is shown in Figure 4, viewers tend to perceive the character aspsychologically on edge or boxed in, a property that can sometimesbe utilized to interesting compositional effect [Brown 2013, p. 52].

Given object point p, unit object look direction d, camera positionc, and unit camera right direction cR , the look space parameter canbe evaluated using the visibility frame bounds objective functionfvis(p) from Section 4.1.1, as

f (p) = fvis

(Π(p) +

Π(p + d) − Π(p)| |Π(p + cR ) − Π(p)| |

). (19)

5 RESULTSAll of the described tools and techniques were implemented in asingle-threaded prototype camera control system that was inte-grated with Unreal Engine 4.9, running online with the engine’sstandard game thread. Several different experimental scenes werethen tested for both scripted and unscripted scenarios on a desk-top computer with a 3.07 GHz processor and an Nvidia GTX 980graphics card.

In all experiments, each camera pose required that our systemoptimize over 5 variables—position in R3, yaw, and pitch. Thisrequired that each shot had a static user-specified field of viewand aspect ratio. In principle, our system is capable of optimizingover both static variables as well as camera roll, but we have notattempted to do so.

Note that for all the scripted scenes tested, a keyframe rate of 40per second was subjectively found to produce a sufficiently smoothcamera path. This is partially a side effect of the speed of action,which was often over 3m/s. Scenes with slower action could use alower keyframe density.

5.1 SpeedMeasuring the speed of our system is difficult as it entirely dependson the tightness of the satisfaction boundaries. There is a tradeoffbetween processor time and solution quality, so a user willing toendure slower performance can set a higher bar for parametersatisfaction.

For most of the scripted scenes tested, a satisfactory unsmoothedcamera path could be found using approximately 3,000 iterations ofsimulated annealing, followed by approximately 2,000 iterations ofgradient descent, taking about 1.15 seconds per frame.With smooth-ing, the same two-phase optimization can be used to roughly locallyoptimize using about 50–75% of the iterations required withoutsmoothing, followed by 1,000–1,200 iterations of gradient descentwith smoothing at a total cost of about 1.75 seconds per frame.

In unscripted experiments, the unsmoothed algorithm was sub-jectively found to be capable of finding satisfactory solutions using100 optimization iterations of gradient descent at an average of60 frames per second. The smoothed algorithm was found to pro-duce a subjectively satisfactory solution—given the predicition of4 keyframes with t = 0.25 sec and with 10 iterations of gradientdescent to initialize each keyframe, followed by another 10 itera-tions of active contour model gradient descent—at an average of25 frames per second, which is more than fast enough to supportreal-time usage.

5.2 Camera PathsA demonstrative scene involved a character running around anopen area, where the parameter set was configured for the camerato follow the character from the front. At one point in the scene, thecharacter passes close enough to a wall that requires deflecting thecamera from the optimal path. The path of the camera, as shownin Figure 5 (top), is similar in all use cases tested.

An example of a scene in which our systemmet with more mixedsuccess was also noted. In this scene, a character runs past a set ofpillars while the camera follows him with a long lens in profile. Asthe optimal camera path for the relative angle parameter becomesoccluded by the pillars, the system must compromise between the

MiG ’17, November 8–10, 2017, Barcelona, Spain Alan Litteneker and Demetri Terzopoulos

Figure 5: Overhead views of the camera paths computed for asimple moving scene (top) and a more complex scene wherethe camera’s optimal path is blocked by a number of pil-lars (bottom). Color key: Green is unscripted-unsmoothed,yellow is unscripted-smoothed, red is scripted-unsmoothed,and blue is scripted-smoothed.

occlusion parameter and the relative angle parameter. As shownin Figure 5 (bottom), the paths of the various use cases vary muchmore significantly, and each suffers from its own particular issue.

The unscripted-unsmoothed strategy becomes stuck when itcomes to the first occlusion boundary, while the unscripted-smoothedstrategy manages to find its way around the first occlusion to trailthe actor from a distance. While the latter solution seems intuitivelybetter, the user has not explicitly specified which is preferable.

Meanwhile, both of the scripted strategies attempt to balancemore evenly between all parameters, resorting to jagged solutionsthat momentarily collide with the scene geometry. However, thesesolutions represent a technically fairer compromise between thedesired shot parameters.

6 FUTUREWORKWhile we have received anecdotal feedback from users of the cur-rent prototype, we have not attempted to run a proper user studyexamining whether the rules and algorithms described are satisfac-tory for the rigors of professional use. Even the design of such a userstudy would be nontrivial [Lino et al. 2014]. The expressivenessof the system can likely be increased by including more commoncinematographic parameters, such as frame balance, leading lines,

separation, contrast, and depth of field, as would adding additionalmethods for combining multiple parameters.

Additionally, the integration of other optimization algorithmswould likely improve overall performance and accuracy. Specializ-ing the camera model for specific objective function types has beenshown to increase efficiency [Lino and Christie 2015]. In unscriptedscenarios, more accurate and stable predictions of future statescould dramatically expand the use of multiple elements.

In real-world cinematography, actors are frequently positionedbased on the physical constraints on the camera in the environmentalong with the content of the scene, a process that to our knowledgeno existing automated cinematography system has attempted toemulate. The adjustment of character positioning may necessitatea significantly different optimization approach.

REFERENCESRafid Abdullah, Marc Christie, Guy Schofield, Christophe Lino, and Patrick Olivier.

2011. Advanced Composition in Virtual Camera Control. In Smart Graphics.Springer, Berlin, 13–24.

Jackie Assa, Daniel Cohen-Or, I-Cheng Yeh, Tong-Yee Lee, and others. 2008. MotionOverview of Human Actions. ACM Transactions on Graphics (TOG) 27, 5, Article115 (2008), 10 pages.

William Bares, Scott McDermott, Christina Boudreaux, and Somying Thainimit. 2000.Virtual 3D Camera Composition From Frame Constraints. In Proc. 8th ACM Inter-national Conference on Multimedia. 177–186.

Christopher J. Bowen and Roy Thompson. 2013. Grammar of the Shot. Taylor &Francis.

Blain Brown. 2013. Cinematography: Theory and Practice: Image Making for Cinematog-raphers and Directors. Taylor & Francis.

Paolo Burelli. 2012. Interactive Virtual Cinematography. Ph.D. Dissertation. IT Univer-sity of Copenhagen.

Paolo Burelli, Luca Di Gaspero, Andrea Ermetici, and Roberto Ranon. 2008. Vir-tual Camera Composition with Particle Swarm Optimization. In Smart Graphics.Springer, Berlin, 130–141.

Marc Christie, Rumesh Machap, Jean-Marie Normand, Patrick Olivier, and JonathanPickering. 2005. Virtual Camera Planning: A Survey. In Smart Graphics. Springer,Berlin, 40–52.

Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z. Wang. 2006. Studying Aestheticsin Photographic Images Using a Computational Approach. In Computer Vision –ECCV 2006. Springer, Berlin, 288–301.

Steven M. Drucker and David Zeltzer. 1995. CamDroid: A System for ImplementingIntelligent Camera Control. In Proc. 1995 ACM Symposium on Interactive 3D Graphics.139–144.

David K. Elson andMark O. Riedl. 2007. A Lightweight Intelligent Virtual Cinematogra-phy System for Machinima Production. In Proc. Artificial Intelligence and InteractiveDigital Entertainment Conf. 8–13.

John Funge, Xiaoyuan Tu, and Demetri Terzopoulos. 1999. Cognitive Modeling: Knowl-edge, Reasoning and Planning for Intelligent Characters. In Proc. ACM SIGGRAPH.29–38.

Quentin Galvane, Marc Christie, Christophe Lino, and Rémi Ronfard. 2015. Camera-on-Rails: Automated Computation of Constrained Camera Paths. In Proc. 8th ACMSIGGRAPH Conference on Motion in Games. ACM, 151–157.

Michael Gleicher and Andrew Witkin. 1992. Through-the-Lens Camera Control.Computer Graphics 26, 2 (1992), 331–140. (Proc. ACM SIGGRAPH ’92).

Nicolas Halper, Ralf Helbing, and Thomas Strothotte. 2001. A Camera Engine forComputer Games: Managing the Trade-Off Between Constraint Satisfaction andFrame Coherence. Computer Graphics Forum 20, 3 (2001), 174–183.

Frank Jardillier and Eric Languénou. 1998. Screen-Space Constraints for CameraMovements: The Virtual Cameraman. Computer Graphics Forum 17, 3 (1998), 175–186.

Michael Kass, AndrewWitkin, and Demetri Terzopoulos. 1988. Snakes: Active ContourModels. International Journal of Computer Vision 1, 4 (1988), 321–331.

Christophe Lino and Marc Christie. 2015. Intuitive and Efficient Camera Control withthe Toric Space. ACM Trans. Graph. 34, 4, Article 82 (July 2015), 12 pages.

Christophe Lino, Marc Christie, Fabrice Lamarche, Guy Schofield, and Patrick Olivier.2010. A Real-Time Cinematography System for Interactive 3D Environments. InProc. 2010 ACM SIGGRAPH/EG Symposium on Computer Animation. 139–148.

Christophe Lino, Rémi Ronfard, Quentin Galvane, and Michael Gleicher. 2014. HowDo We Evaluate the Quality of Computational Editing Systems?. In Proc. AAAIWorkshop on Intelligent Cinematography And Editing. 35–39.

Roberto Ranon and Tommaso Urli. 2014. Improving the Efficiency of ViewpointComposition. IEEE Transactions on Visualization and Computer Graphics 20, 5(2014), 795–807.