automatic view finding for drone photography based on image … · 2020. 4. 15. · planning. in...

Automatic View Finding for Drone Photographybased on Image Aesthetic Evaluation

Xiaoliang Xiong, Jie Feng and Bingfeng ZhouInstitute of Computer Science and Technology, Peking University, 100871, Beijing, China

{jonny xiong, feng jie, cczbf}@pku.edu.cn

Keywords: Drone Control, Automatic View Finding, Drone Photography, Image Aesthetic Evaluation.

Abstract: Consumer-level remotely controlled smart drones are usually equipped with high resolution cameras, whichmake them possible to become unmanned “flying camera”. For this purpose, in this paper, we propose anautomatic view finding scheme which can autonomously navigate a drone to an proper space position wherea photo with an optimal composition can be taken. In this scheme, an automatic aesthetic evaluation forimage composition is introduced to navigate the flying drone. It is accomplished by applying commonly usedcomposition guidelines on the image transmitted from the drone at current view. The evaluation result is thenconversely used to control the flight and provide feedback for the drone to determine its next movement. Inflight control, we adopt a downhill simplex strategy to search for the optimal position and viewing directionof the drone in its flying space. When the searching converges, the drone stops and take an optimal image atcurrent position.

1 INTRODUCTION

Consumer-level remotely controlled smart drones area special kind of Unmanned Aerial Vehicles (UAVs)that are equipped with an on-board computer fornavigation and communication. The drone usuallyhas 4 propellers and can be controlled convenientlyby a ground remote controller, which is usually also acomputer with a two way communication link withthe drone. In this paper, we describe a controlscheme that combines drone’s high programmablemaneuverability and the theory of image aestheticmeasurement to achieve the automatic view findingon the drone’s autonomous flight. Particularly,our drone control scheme for optimal view findingcomprises the following steps:

• Detect the photographic subject;

• Locate the subject at a proper position in currentview and evaluate the image aesthetics;

• Adjust the drone position based on the aestheticevaluation so that a photo with better compositioncan be obtained.

In the first step, the photographic subject is detectedby searching predefined specific features (such as

This work is partially supported by NSFC grants#61370112, #61602012.

human face or buildings) in the image sequence. Withthe detected subject, we evaluate the image aestheticsby considering several commonly used compositionrules to calculate its aesthetic score. According tothe evaluation, we control the flight by heuristicallyadjusting the drone flying status until a maximal scoreis reached, and then an image is captured as theoptimal photo.

For the image aesthetic evaluation, it is asubjective activity and many factors (like personalsentiment) can influence the judgement. However,there are still some widely accepted guidelines for thephotographer when shooting a photograph, which aresuitable for the computational aesthetic evaluation.These guidelines include: rule of thirds, diagonaldominance, visual balance and proper region size etc.Liu et al. first quantize these guidelines and formulatean aesthetic score criteria (Liu et al., 2010). Weadopt similar aesthetic measurements in this paper toautomate the process of image aesthetic evaluation,and use the aesthetic score to control the flight.

Vision-based navigation is widely used in theautonomous control for the robot or the automobile(Lenz et al., 2012; Bills et al., 2011). In theseapplications, images provide position information forthe device to locate itself in the environment for pathplanning. In this paper, our drone is navigated to aproper position based on the aesthetic score. As the

282Xiong, X., Feng, J. and Zhou, B.Automatic View Finding for Drone Photography based on Image Aesthetic Evaluation.In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017) - Volume 1: GRAPP, pages282-289ISBN: 978-989-758-224-0Copyright c© 2017 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved

aesthetic evaluation depends on the relative positionbetween the photographic subject and the camera,it can give feedback to the drone to determine itsnext movement. During this adjustment, a downhillsimplex strategy (Press et al., 1992) is adopted tonavigate the drone to the position corresponding toa higher aesthetic score.

In summary, the main contributions of this paperinclude:• Propose an aesthetic evaluation algorithm which

is oriented to the real-time optimal view findingfor drone photography;

• Develop a real-time flight control algorithm usingdownhill simplex method based on the imageaesthetic evaluation;

• Implement the aesthetic evaluation and flightcontrol algorithms on a remotely controlleddrone platform, which enables our drone toautomatically fly to an proper position where aphoto with optimal composition can be taken bythe on-board camera.

2 RELATED WORK

2.1 UAV Navigation

Unmanned Aerial Vehicles (UAVs) are currentlywidely used in applications like surveillance andaerial photography (Joubert et al., 2015; Roberts andHanrahan, 2016). The researching topics focusingon UAVs include obstacle avoidance and autonomousnavigation. And the solutions can be categorized intotwo classes: active-sensor-based methods (Bachrachet al., 2011; Benet et al., 2002) and vision-basedmethods (Lenz et al., 2012; Soundararaj et al., 2009).

Active-sensor-based Methods. Active sensors suchas laser range finders (Bachrach et al., 2011), sonar,and infrared detectors (Benet et al., 2002) are oftenused for obstacle avoidance during UAV or robotnavigation in indoor environments. These devices arecheap and have fast response for distance detection.However, they are not suitable for unstructuredoutdoor environments. Also, most of these sensorshave high power requirements and can not beadequately supplied in consumer level aeriel vehicles.

Vision-based Methods. Vision signals includingimage and depth information are commonly usedin UAV autonomous flight. They can be easilycaptured using lightweight cameras which are small-sized, require low power supply and offer long-range sensing. Without any extra equipment, Lenz

et al. use a single monocular camera and propose aparallel algorithm based on Markov Random Fieldclassification for an aerial robot to avoid obstaclesautonomously (Lenz et al., 2012). Soundararaj etal. fly a miniature helicopter in indoor environments,using a data-driven image classification method toachieve real-time 3D localization and navigation(Soundararaj et al., 2009). These works analyse theimage captured by the onboard camera to navigatethe vehicle. However, they all need prior knowledgeof the flying environment. Bills et al. utilizethe perspective cues to estimate the desired flyingdirection to navigate the flight (Bills et al., 2011).They avoid reconstructing the 3D environment for itscomplexity. Harbar et al. uses a stereo camera tocapture and build 3D environment map for obstacledetection and dynamic path updating (Hrabar, 2008).This takes considerable computational time andpower, which is not suitable for UAVs.

2.2 Automatic Photography

Using robots to take photos is not a novel problem.Byers et al. had developed an autonomous robotsystem for taking photographs of people at socialevents (Byers et al., 2003). Their robot walkson the floor, and needs remote path planning andmotion control. Kim et al. designed their ownhardware and created a “robot photographer” to takepictures for human by skin color detection (Kim et al.,2010). The camera direction is controlled via humanvoice recognition, but the position of the camera cannot move according to the human motion. Theserobots take photos by selecting proper photographicopportunity based on customized clues, which is notgeneral enough. We solve these problems by adoptinga flying vehicle that controlled by image aestheticevaluation.

There are also some works on semi-automaticphotograph. Fu et al. present a data-drivenpose suggestion tool, serving as a guidance for thephotographer (Fu et al., 2013). They identify a similarpose for the current subject from a large collection ofreference poses, based on which the subject shoulddo some refinement to match the selected pose.They only focus on the pose suggestion and otherphotography tips are guaranteed manually.

2.3 Image Quality Assessment

In computer vision, different levels of image featuresare adopted to evaluate the image quality (Ke et al.,2006; Luo and Tang, 2008). For computationalphotography, image composition is an important

Automatic View Finding for Drone Photography based on Image Aesthetic Evaluation

283

Figure 1: Photographs automatically taken by our drone. The image on the top-right corner of each column is an temporaryview, from which the drone begins to search and navigate by the aesthetic score, finally stops at the optimal view.

measurement for the photographer to create aes-thetically pleasant photos (Yao et al., 2012). Itrefers to the arrangement of visual elements duringview finding. There are no absolute rules to createa good photograph, but heuristic principles, whichmay lead to more pleasant composition, can beconcluded based on the experience of professionalphotographers.

The principles include rule of thirds, shapesand lines, visual balance, and diagonal dominanceetc.(Krages, 2005). Many efforts have been madeon image editing to improve the composition, likeimage cropping (Liu et al., 2010; Ni et al., 2013),warping (Jin et al., 2012) and resizing (Li et al.,2015). But all these approaches are post-processingafter the images are taken, and they will more or lesslose or distort the image content during compositionoptimization. Different from these works, we evaluatethe image aesthetics online and search for an optimalcomposition during photographing.

3 AUTOMATIC VIEW FINDING

Human photographers take aesthetically pleasantphotos according to certain widely accepted guide-lines. For drone photography, we propose anautomatic view finding scheme on the basis of imageaesthetic evaluation, so that a remotely controlleddrone may imitate such human behaviors. The droneis equipped with an onboard camera which can takelive video stream during the flight. The photographicsubjects are firstly detected from the video imagesequence. Then, we evaluate the image aestheticsby analysing its composition, to determine whetherit satisfies general composition rules. According tothe aesthetic score, the drone can be navigated toa better viewpoint, until an optimal view with thehighest score is reached. In this way, an aestheticallysatisfactory photo can be captured by the drone. Themain work flow of our method is shown in Fig.2.

3.1 Feature Detection

Given an image, we calculate its aesthetic score basedon an analysis of its spatial structures, considering thedistributions of photographic subjects and prominentlines in the image. Hence, the photographic subjectsshould be automatically detected first. And theconstituent of the subjects depends on what typeof photo we want to take. For human portraitphotography, we can detect the human body by theface features. For natural sceneries, the subjects canbe detected by their geometry structures.

Photographic Subject Detection

For human portraits, we first estimate whether thereare people in the image, using a face detection methodbased on Haar features (Viola and Jones, 2004). Acascade classifier is pre-trained on the Haar featuresof sampling dataset. Then, it is used to determinewhether the selected region of the input image is aface by sliding a window with different size over theimage. With the detected faces, we can estimate thebodies of the subjects.

Line Detection

The prominent lines in an image are also importantelements for aesthetic evaluation. We first detect theline segments existing in an image based on HoughTransform (Duda and Hart, 1972). Then these linesegments are merged if they are on approximately thesame line.

3.2 Image Aesthetic Evaluation

There are various guidelines for shooting well-composed photographs. Here, we consider threemost effective guidelines: rule of thirds, visualbalance and proper region size, which are well-defined and prominent in many aesthetic images(Fig.3). These guidelines are widely used in rule-based image composition optimization (Liu et al.,2010; Jin et al., 2012; Li et al., 2015), and we

GRAPP 2017 - International Conference on Computer Graphics Theory and Applications

284

Figure 2: The work flow of our automatic view finding.During the flight, the drone captures images, evaluates theiraesthetics and adjusts its flying status to searching for anoptimal view.

make some adaption on them to evaluate the imageaesthetics during automatic view finding.

For the rule of thirds, photographers are en-couraged to place the main subject around fourthird points (green dots in Fig.3a) intersected bytwo equally spaced horizontal lines and two equallyspaced vertical lines (red dash lines in Fig.3a, i.e.third lines) in the image. Also, prominent lines shouldkeep align with these four third lines (Fig.3b). Invisual balance, multiple subjects are suggested to bedistributed evenly in the image. And proper regionsize tells the photographer what’s the proper size thatthe subjects should occupy the whole image.

According to (Liu et al., 2010), the aesthetic scoreof a given image is calculated as:

E =w1ERT +w2EV B +w3ERS

w1 +w2 +w3, (1)

where ERT ,EV B,ERS are the quantization of ruleof thirds, visual balance and proper region size,respectively. w1,w2,w3 are weights of each guideline.

ERT is a combination of the point and lineconstraints,

ERT = λpointEpoint +λlineEline. (2)

It measures how close the photographic subjects lie tothe third points (Epoint ) and how close the prominentlines lie to the third lines (Eline). In Fig.3a, the toweris placed near the right-top power point to followpoint constraint. And Fig.3b shows the prominent lineplacing near the bottom third line to follow the lineconstraint.

EV B quantizes the harmony of an image-composition. An arrangement of all salient regionsis considered balanced if their weighted center isnear to the image center. In Fig.3c, two subjects areplaced on two sides of the image to create a visuallybalanced composition.

Figure 3: Three composition guidelines. (a,b)Rule ofThirds, (c)Visual Balance, (d)Proper Region Size.

ERS is a measurement of the proper region sizeof the photographic subjects in an image. Liuet al. surveyed over 200 professional images andobtained a distribution of salient region ratio, whichincludes three dominant peaks at 0.1, 0.56 and 0.8,corresponding to small, medium, and large sizedregions, respectively. Similarly, we encourage subjectregion size that follows this distribution. Fig.3dshows a subject occupying about 0.1 of the wholeimage.

Specially, for single-subject photographing, thesubject must be placed near the third points to satisfythe rule of thirds, or near the image center to satisfythe visual balance. Hence, there should be a tradeoffbetween the two rules, or else both rules will beviolated. Therefore, we change the weights w1,w2for each rule based on our photograph situation. Ifwe tend to place the subject near the third point, wetake w2 = 0. For multiple subjects, the visual balanceis more important and we take λpoint = 0, or else allthe subjects will be placed on one side of the image toform a visually unbalanced composition.

In summary, we adopt similar formulations forthe guidelines as in (Liu et al., 2010) for the imageaesthetic evaluation. Some modifications are madein our implementation: 1) Salient regions are definedbased on the photographic subjects; 2) The diagonaldominance is not used in our aesthetic evaluation asthe UAV is always flying horizontally; 3) Differentweights are adopted for each guideline, to takephotographs with different style.

3.3 Automatic View Finding

As described in the last section, the image aestheticevaluation is a combination of three quantizedcomposition guidelines. It measures how thephotographic subjects distribute in the capturedframe, and describes the relative position of the droneand the subjects. Based on this evaluation, an optimal


285

Figure 4: Flight adjustment. (a) Yaw at a fixed position toadjust camera direction, (b) Throttle to fly up and down,(c) Roll to move left and right, (d) Pitch to move front andback. These movements cause the relative position of thephotographic subject changing in the image, resulting innew aesthetic score.

view with the highest aesthetic score can be found.If current frame does not reach the highest score, thedrone should adjust its flight to the direction wherethe score increases. Thus, the drone flight controldepends on the aesthetic score, and the navigationbecomes the searching of the highest score.

3.3.1 Flight Control Model

Generally, a drone has 4 flying status: throttle (flyup and down), roll (move left and right), yaw (rotatealong fixed point) and pitch (move front and back).Note that, since the onboard camera has fixed focallength, we move the drone front and back to changethe subject region size.

Fig.4 shows the four flying status. Given amovement xi, i ∈ {t,r,y, p} at each status (t,r,y, pfor throttle, roll, yaw, pitch), the drone moves incorresponding direction and consequently causes thevarying of image aesthetic score. Specially, themovement xi describes both the moving direction andstep length. Thus, the score E in Eq.1 can alsobe written as E = f (xt ,xr,xy,xp). Here, f is animplicit function of the four flying status, and thereis no precise model of how each variable affects theaesthetic score.

In order to take a photo with optimal composition,our target function for automatic view finding is

max E = f (xt ,xr,xy,xp), (3)

where xi ∈ (xi − εi,xi + εi) and (xi − εi,xi + εi) is asmall interval defining the searching space in eachdimension.

This function can be optimized by a downhillsimplex method (Press et al., 1992) in the 4Dspace of xt ,xr,xy,xp. Downhill simplex method isefficient in multi-dimensional function optimization,

Figure 5: Variable variation during optimal view searchingusing downhill simplex method.

which requires only function evaluations rather thanderivatives. In our 4-dimensional case, a simplex isthe geometrical figure consisting of 5 vertices and alltheir interconnecting line segments. The method thentakes a series of steps including reflection, expansionand contraction on the simplex, until it reaches themaximum of the target function.

3.3.2 Optimal View Searching

Different from mathematical function optimization,we should consider that: in the actual drone move-ments, drastic changes are not allowed and multi-dimensional variation is not preferred. Consideringthat human photographers adjust the camera settingsstep by step, we also navigate our drone in onedimension each time.

After the photographic subject is detected duringour drone turning around the yaw-axis, it beginsto search an optimal view to increase the aestheticscore. For each dimension, we give an initial estimateof xi, i ∈ {t,r,y, p}. Then they are transformedinto drone controlling commands, and navigate thedrone to a new viewpoint. The image under thisnew viewpoint is evaluated. If the aesthetic scoreincreases, the movement in this direction continues,or else the drone should fly to the opposite directionwith a smaller step length. Fig.5 shows thexy variation tendency in the optimal view finding(xr,xp,xt variation is similar). At the beginning of thesearching, it changes with a large decrement and thestep length becomes smaller gradually as it gets closerto the optimal view.

We use a multi-thread mechanism to performthe flight control based on the aesthetic evaluation.For the image aesthetic evaluation, it calculates theaesthetic score E1 of current frame I1 transmitted fromthe drone, and sends a signal to the flight controlthread when the evaluation is completed. The flightcontrol thread then begins to search a better viewwhere the aesthetic score increases.


286

The detailed optimal view finding algorithm isshown in Alg.1. When the subjects are not detected,we give the drone a yaw movement and set χy = 0.25,where 0.25 is the speed relative to the maximumspeed that the drone can reach and the value is setaccording to our experiments. When subjects occur inthe camera view, we test if the aesthetic score changesbetween current frame I1 and previous frame I0. Ifso, a flight adjustment that affects the correspondingcomposition rules is needed. For example, if ERTincreases, it means the subject center gets closer to thethird point (with v gets smaller). And we set xt =

vH xt

to decrease the movement vibration. With adjustedxt ,xr,xy,xp, controlling commands are sent to thedrone, which navigates the drone to a new viewpoint.Then image aesthetic score under this view is inputinto Alg.1 for further optimal view searching. Inthe algorithm, W,H are the image width and height,respectively. τ and δ are constant threshold and wetake τ = 0.95,δ = 0.1 .

Our target function converges until the dronevibrates small enough in each dimension (|λi| <δ). Naturally, the image aesthetic score reaches itshighest value. Then we stop the drone movement andtake the image at current viewpoint as the optimalphotograph. When the subjects in the frame move, theaesthetic score of the image will change and it is nolonger the optimal view. Therefore the view searchingwill be repeated until a new optimal view is found. Infact, that implicitly leads to object tracking.

4 EXPERIMENTAL RESULTS

We implement our automatic view finding schemeon a remotely controlled drone platform whichconsists of an off-the-shelf flying vehicle “ParrotAR. Drone” and a common laptop. The dronecontains two cameras: one facing forward for imagecapture (with resolution 1280x720) and anothervertically downwards, a sonar height sensor, andan onboard computer for command processing andcommunication with the PC. Commands and imagesare exchanged via a WiFi adhoc connection betweenour host machine and the drone. The image aestheticevaluation and optimal view finding algorithm run ona common laptop (2.10GHz Pentium dual core, 1GBRAM), with a Linux OS of Ubuntu 14.04.

As described in section 3, our automaticview finding is based on the image aestheticevaluation. The image aesthetic score reflects howthe photographic subjects distribute in the image.For human portrait photography, we first detect thehuman faces, then estimate the bodies and place them

Algorithm 1: Optimal view finding using downhill simplexsearching.Input: the aesthetic score E1 of current frame I1, andits three components E1

RT ,E1V B,E

1RS;

Output: the controlling commands xt ,xr,xy,xp foreach dimension;Initialization: Set the initial aesthetic score E0 = 0of previous frame I0, and its three components E0

RT =0,E0

V B = 0,E0RS = 0; Set xt = 0,xr = 0,xy = 0,xp = 0;

Set λi = MAX FLOAT, i = 1, · · · ,5;1: if E1 == 0 then . Subjects not detected

2: xy = χy;3: else . Optimal view searching

4: if E1 > τ and |λi|< δ then5: Capture the image I1; . Optimal view found

6: else7: xt = 0.25,xr = 0.25,xy = 0.25,xp = 0.25;8: Calculate the vector (u,v) between the

center of mass and the nearest third point;9: if E1

RT 6= E0RT then

10: xt = λ1xt ; . Throttle to satisfy RT, λ1 = vH

11: xr = λ2xr; . Roll to satisfy RT, λ2 = uW

12: xy = λ3xy; . Yaw to satisfy RT, λ3 = u2+v2

W2+H2

13: Calculate the vector (s, t) between thecenter of mass and the image center C;

14: if E1V B 6= E0

V B then15: xr = xr +λ4xr; . Roll to satisfy VB, λ4 = s

W

16: xy = xy +λ5xy; . Yaw to satisfy VB, λ5 = s2+t2

W2+H2

17: Calculate the distance d between the arearatio of current frame and the nearest perfect arearatio r;

18: if E1RS 6= E0

RS then19: xp =

dr xp; . Pitch to satisfy RS

20: Send command xt ,xr,xy,xp to the drone;21: E0 = E1,E0

RT = E1RT ,E

0V B = E1

V B,E0RS = E1

RS;

at the proper position satisfying the compositionguidelines. Face detection is the most time-consuming step in our method, so we down-samplethe captured images by 2x to reduce the searchingspace. After the subjects are detected, we turn tosubject tracking between the adjacent frames usingCamshift (Comaniciu and Meer, 2002) to improvethe detection accuracy. Subjects in the former frameare back projected onto the latter frame and the newsubject is searched near the projected center.

Under current view, we compare the currentaesthetic score with previous score to determine thedrone movement. If the score increases, currentmovement continues and the step length decreases.Or else the drone stops and moves back to thepreviously better view. The optimal view searching


287

(a) (b) (c) (d)Figure 6: The process of our automatic view finding using downhill simplex searching. (a) An initial view is found where thesubjects are first detected. (b) One temporary view. (c) The optimal view. (d) The aesthetic score variation during optimalview searching.

continues until the aesthetic score changes slightly.All computation can be accomplished in real-time(with frame rate at about 20 fps).

Validation

To validate the effectiveness of our method, we sim-plify the aesthetic evaluation by detecting concentriccircles and placing it at the center of the image (Fig.6:first row). The optimal view search begins at theinitial view with score 0.486 and the score increaseswhen the center of the circles gets closer to the imagecenter. Even with external interference, the drone canfinally stop at the view aiming at the concentric circlecenter (with aesthetic score 0.975).

Single Subject

For single subject, the rule of thirds and the visualbalance can not be guaranteed at the same time. If wewant to place the subject at the image center, we cantake λpoint = 0 and eliminate the point constraints inrule of thirds (Fig.6: first row). If we tend to placethe subject near the third point, we can take w2 = 0and do not consider the visual balance (Fig.6: secondrow, the score changes from 0.739 and finally reaches0.952 with several steps searching).

Multiple Subjects

For multiple subjects, the visual balance is more

important than the point constraints in rule of thirds.So we take λpoint = 0 and place the subjects evenlyin the image to avoid unbalanced composition. InFig.6, the third and forth row show two cases of ourdrone taking photos for multiple subjects. Since thesesubjects may not occur in the camera view at thesame time, our method search the optimal view onlyfor the detected subjects. In the forth row, the left-most person is not detected first and the initial viewis actually optimal for the two detected subjects (withscore 0.952). When new subjects are detected, currentview is not the optimal and the search keeps going onuntil a new optimal view with score 0.958 is reached.

Fig.6(d) shows the aesthetic score variation duringautomatic view searching. With the evaluated score ofimages where subjects are first detected, we estimatethe flight adjustment and send control commandsto the drone. After several steps of searching, itarrives at the optimal view and then takes photos.Fig.1 shows the photos taken by the drone using ourautomatic view finding.

5 CONCLUSION

In this paper, we propose an automatic view findingscheme based on image aesthetic evaluation, whichmakes a remotely controlled drone capable of


288

automatically taking photographs satisfying severalbasic composition guidelines. The drone is navigatedby the aesthetic score gradually to the view satisfyingthese guidelines. And we adopt a downhill simplexmethod to heuristically search for the optimalview. Experiments on human portrait photographydemonstrate the efficiency of our method. In fact, ourdevice can also take photos for any other subjects witha clearly defined features like the human face.

As a prerequisite, the subject detection is crucialto guarantee that our method can work well. In humanportrait photography, the face detection will fail if thesubject turns his head away from the camera. Theaesthetic score drops to 0 and our drone will stopcurrent movement and go back to find a higher score.If the face detection still fails, it will stop currentsearching and start a new one.

We are exploring more rules and clues inpractical photographing, such as color, illumination,or geometry, to make our automatic photographermore intelligent. Meanwhile, we notice that rule-based aesthetic evaluation is not general enough tocapture the diversity of possible photographs. Manyrules are not convenient to be quantized. We aretrying to overcome these problems with data drivenmethods.

REFERENCES

Bachrach, A., Prentice, S., He, R., and Roy, N. (2011).Range robust autonomous navigation in gps-deniedenvironments. Journal of Field Robotics, 28(5):644–666.

Benet, G., Blanes, F., Sim, J. E., and Prez, P. (2002). Usinginfrared sensors for distance measurement in mobilerobots. Robotics & Autonomous Systems, 40(4):255–266.

Bills, C., Chen, J., and Saxena, A. (2011). Autonomousmav flight in indoor environments using single imageperspective cues. In IEEE International Conferenceon Robotics and Automation (ICRA),2011, pages5776–5783.

Byers, Z., Dixon, M., Goodier, K., Grimm, C. M.,and Smart, W. D. (2003). An autonomous robotphotographer. In IROS 2003, volume 3, pages 2636–2641 vol.3.

Comaniciu, D. and Meer, P. (2002). Mean shift:a robust approach toward feature space analysis.IEEE Transactions on Pattern Analysis & MachineIntelligence, 24(5):603–619.

Duda, R. O. and Hart, P. E. (1972). Use of the houghtransformation to detect lines and curves in pictures.Communications of The ACM, 15(1):11–15.

Fu, H., Han, X., and Phan, Q. H. (2013). Data-drivensuggestions for portrait posing. In SIGGRAPH Asia2013, Technical Briefs, pages 29:1–29:4.

Hrabar, S. (2008). 3d path planning and stereo-basedobstacle avoidance for rotorcraft uavs. In IROS, pages807–814.

Jin, Y., Wu, Q., and Liu, L. (2012). Aesthetic photocomposition by optimal crop-and-warp. Computers& Graphics, 36(8):955–965.

Joubert, N., Roberts, M., Truong, A., Berthouzoz, F.,and Hanrahan, P. (2015). An interactive tool fordesigning quadrotor camera shots. ACM Transactionson Graphics, 34(6):238.

Ke, Y., Tang, X., and Jing, F. (2006). The design ofhigh-level features for photo quality assessment. InCVPR’06, volume 1, pages 419–426.

Kim, M.-J., Song, T. H., Jin, S. H., Jung, S. M.,Go, G.-H., Kwon, K. H., and Jeon, J. W.(2010). Automatically available photographer robotfor controlling composition and taking pictures. InIROS, pages 6010–6015.

Krages (2005). Photography: The Art of Composition.Allworth Press.

Lenz, I., Gemici, M., and Saxena, A. (2012). Low-powerparallel algorithms for single image based obstacleavoidance in aerial robots. In IROS, pages 772–779.

Li, K., Yan, B., Li, J., and Majumder, A. (2015). Seamcarving based aesthetics enhancement for photos.Signal Processing-image Communication, 39:509–516.

Liu, L., Chen, R. C., Wolf, L., and Cohenor, D. (2010).Optimizing photo composition. Computer GraphicsForum, 29(2):469–478.

Luo, Y. and Tang, X. (2008). Photo and video qualityevaluation: Focusing on the subject. In ECCV 2008,Marseille, France, October 12-18, pages 386–399.

Ni, B., Xu, M., Cheng, B., Wang, M., Yan, S., and Tian,Q. (2013). Learning to photograph: A compositionalperspective. Trans. Multi., 15(5):1138–1151.

Press, W. H., Teukolsky, S. A., Vetterling, W. T., andFlannery, B. P. (1992). Numerical Recipes in C: TheArt of Scientific Computing. Cambridge UniversityPress, New York, NY, USA, 2nd edition.

Roberts, M. and Hanrahan, P. (2016). Generating dynami-cally feasible trajectories for quadrotor cameras. ACMTransactions on Graphics, 35(4):61.

Soundararaj, S. P., Sujeeth, A. K., and Saxena, A. (2009).Autonomous indoor helicopter flight using a singleonboard camera. In IROS, pages 5307–5314.

Viola, P. and Jones, M. J. (2004). Robust real-time facedetection. International Journal of Computer Vision,57(2):137–154.

Yao, L., Suryanarayan, P., Qiao, M., Wang, J. Z.,and Li, J. (2012). Oscar: On-site compositionand aesthetics feedback through exemplars forphotographers. International Journal of ComputerVision, 96(3):353–383.


289

automatic view finding for drone photography based on image … · 2020. 4. 15. · planning. in...

Documents