Interactive Autonomous Driving through Adaptation from Participation

Anqi Xu∗, Qiwen Zhang∗, David Meger∗ and Gregory Dudek∗

Abstract— We present an intelligent driving software system thatis capable of adapting to dynamic task conditions and adjustingto the steering preferences of a human passenger. This interactiveautonomous system is a realization of a natural human-robotinteraction paradigm known as Adaptation from Participation (AfP).AfP exploits intermittent input from the human passenger in orderto adapt the vehicle’s autonomous behaviors to match the intendednavigation targets and driving preferences. In our system, this isrealized by dynamically adjusting the parameter settings of a vision-based algorithm for tracking terrain boundaries. We deployed theresulting interactive autonomous vehicle in diverse task settings,and demonstrated its ability to learn to drive on unseen paths, aswell as to adapt to unexpected changes to the environment and tothe vehicle’s camera placement.

Keywords— human-robot interaction, autonomous driving, fieldrobotics

I. INTRODUCTION

A utonomous driving promises innumerable benefits tosociety and is nearing wide acceptance. This is illus-

trated by the ever-increasing intelligent assistance featuresin commercial automobiles, and also by the rapid establish-ment of government regulations for self-driving cars [1].A common goal shared by many is for an autonomousdriving solution that can operate robustly within diverseoutdoor environments, which poses a number of identifiedchallenges. For instance, many existing systems are likely tofail when driving off of standard roadways, since outdoorpaths can vary highly in their definition (e.g. gravel, grass,and dirt trails driven by other vehicles). Also, navigationsolutions based on the Global Positioning System (GPS) areinappropriate for operations in previously unseen or changingpaths. Furthermore, reactive way-finding using local sen-sors, such as cameras, is susceptible to constrained field ofview and other restrictions, which limit their autonomousnavigation capabilities. In contrast, since human drivers canreadily handle navigation in various dynamic and challengingsituations, the coupling of an autonomous driving systemwith the help from a human participant becomes an attractivesolution concept, which we explore presently.

In this work we present an interactive autonomous drivingsystem that incorporates input from an onboard humanpassenger to provide high-level navigation guidance andsituational driving expertise. The realization of a human-robot interaction paradigm known as Adaptation from Partic-ipation (AfP) is used to mediate this interface. AfP enablesthe autonomous system to learn from intermittent humancontrol in order to adapt its own navigation objectives and

∗Authors are members of the Mobile Robotics Lab, at the Centrefor Intelligent Machines, McGill University, Montreal, Canada. Email:{anqixu,qzhang32,dmeger,dudek}@cim.mcgill.ca

Fig. 1. We deployed an interactive and adaptive autonomous driving systemonboard the SL-Commander all-terrain vehicle.

driving style on-the-fly. In this manner, our solution isable to alleviate much of the tedious driving burdens bynavigating predominantly through autonomous control evenin unanticipated conditions, while providing flexibility andcomfort through seamless transitions to and from manualdriving.

This work contributes a feasibility demonstration of aninteractive autonomous driving system that realizes the AfPparadigm. To accomplish this, we have adapted an existinginstrumented vehicle, the SL-Commander (as seen in Fig. 1),to be compatible with the Robot Operating System (ROS)middleware. Sec. II describes the capabilities of the SL-Commander platform, as well as its hardware and softwareinfrastructure. Our autonomous vehicular controller employsan existing vision-based algorithm for detecting and steeringalongside terrain boundaries, which is described in Sec. III.Sec. IV discusses the interaction paradigm of Adaptationfrom Participation (AfP), and also highlights its implemen-tation on the SL-Commander platform. Sec. V elaborates onresults of preliminary field experiments with this interactiveautonomous vehicle, which demonstrated successful drivingon previously unseen paths, as well as adaptation to differenttypes of disturbances. We conclude in Sec. VI with adiscussion of experimental outcomes and future work.

II. SYSTEM INFRASTRUCTURE

An intelligent driving hardware platform must providea number of services in order to achieve practical trans-port of its human passengers. Some of the most essentialrequirements include safety, navigational competency, anda flexible command interface. In this work, we base anautonomous driving solution around the SL-Commandervehicle, which was developed by MacDonald Dettwiler

and Associates (MDA), Bombardier Recreational Products(BRP), and Quanser Consulting [2], for the Canadian SpaceAgency’s planetary research [3]. This vehicle’s base drive isan electric-powered side-by-side All-Terrain Vehicle (ATV)produced by BRP [4], with the capability to traverse terrainranging from paved roads to undulating back-country trails.The SL-Commander variant has been instrumented by MDAand BRP with a programmable interface for steering, accel-eration, breaking, vehicle state control, and status monitoringcapabilities. Furthermore, numerous sensors including cam-eras and laser range-finders have been integrated onboardto allow for autonomous perception of the environment.This vehicle’s capabilities are exposed through the JointArchitecture for Unmanned Systems (JAUS) SAE AS-4standard [5], and for this work we extended sensor andcontrol interface support to the open-source Robot OperatingSystem (ROS) [6] to facilitate easy re-use and sharing ofcode with other similar platforms. This section will continueto elaborate on each layer of this platform in detail.

A. SL-Commander Mechanical Platform

Side-by-side ATVs are a class of wheeled vehicles well-suited for both rugged terrain and low-traffic roads. Thismakes them attractive for the study of autonomous drivingin general outdoor environments, beyond the paved roadsthat have been the focus of previous research efforts [7].The SL-Commander utilizes Ackerman steering and a four-wheel drive with selectable locking differentials, independentsuspension, rugged construction and a high-torque propulsionsystem based around a 48V electric AC induction motor. Thisvehicle has been designed for a comfortable ride on roughterrain, and provides a number of passenger safety features.These include a steel roll-cage for protection from roll-overson steep inclines, as well as front and rear ventilated discbrakes for rapid stopping power.

B. Actuation and Sensing

A drive-by-wire system onboard the SL-Commander repli-cates many of the operations typically conducted by a humandriver during manual control, including actuation of the brakeand accelerator pedals, the gear shift, and also the steeringwheel. The vehicle state, including wheel odometry, actualspeed and turning angle, battery state-of-charge, etc., can allbe accessed remotely. These subsystems are interfaced via aController Area Network (CAN) Bus with an onboard DriveComputer, which is responsible for managing real-time low-level tasks such as steering control. This Drive Computerprovides a velocity-based software interface, which is usedby another networked Smart Computer that implements high-level ROS and JAUS interfaces and also hosts autonomysoftware.

In autonomous control mode, both the steering wheeland throttle pedal are engaged by the drive-by-wire systemand are locked from human input. Nevertheless, the humanretains the ability to use the brake pedal and can stopthe vehicle at any time. In addition, kill switches locatedon the dashboard and on the exterior frames can be used

to disengage power flow to the entire vehicle in case ofemergency.

The sensor suite onboard the SL-Commander includes aSICK LMS-151 laser range-finder, a Vivotek high-definitioncamera with 18x optical zoom mounted on a pan-tilt unitfrom FLIR Systems Inc., and two Tyzx stereo cameras.Our autonomous driving system employs the central pan-tilt-zoom camera unit, which allows for flexible visual trackingof diverse paths and trails, as will be described in Sec. III.

C. Software Automation Interface

The Robot Operating System (ROS) is a distributed mid-dleware solution for robotic sensing and control [6]. Itis open source and has gained wide adoption within theresearch and industrial robotics communities, making it anenabling technology for code sharing and re-use. Inspired bythe efforts of research groups that have previously adoptedROS for autonomous cars [8], we leveraged the benefitsof ROS while building an autonomous driving system forthe SL-Commander. In particular, we realized autonomoussteering using an adaptive vision-based control system thatwe had originally developed for other ROS-enabled wheeledrobots [9]. Its integration onboard the SL-Commander ne-cessitated the development of a standardized ROS interfacefor interacting with the vehicle’s sensors and actuators,which extends from the JAUS control software previouslydeveloped by MDA. Our ROS interface complies with thecommon set of sensing and actuation functionalities ofwheeled ground vehicles, and was essential in facilitating theswift deployment and extensions to our existing automationsystem, targeting the SL-Commander platform.

Sensory data from the vehicle’s cameras and laser range-finders are exposed within ROS by device driver nodes, andare published using standard message formats, accompaniedby meta-data on sensor characteristics. The pose of eachsensing and control element of the vehicle is tracked overtime, allowing information from various sources to be regis-tered temporally and spatially. Locomotion commands areprocessed by a ROS node that wraps the Quanser DriveComputer API, to appropriately actuate the vehicle.

III. VISION-BASED PATH FOLLOWING

During autonomous operation, the vehicle utilizes vision-based control in order to safely follow pathways. Thereis a wealth of literature on the use of cameras to guideautomobiles [10], [11], [12]. In this work, we deployedan existing visual boundary tracking system [9] to real-ize autonomous driving onboard the SL-Commander. Thisautomation system identifies a dominant terrain boundarywithin each frame acquired by a frontal-mounted camera,corresponding to the contours of roads, pathways, fields, etc.The identified boundary line is fed into a control law thatregulates the steering direction of the vehicle, which operatesunder user-regulated velocity. This image processing pipelineis illustrated in Fig. 2.

When tracking boundaries on the ground plane from afrontal view camera, it is important to begin by excluding

(a) exclude horizon (b) segment target region (c) extract boundary curve (d) fit line and map into steering

Fig. 2. A simplified pipeline of the boundary tracking algorithm: (a) first the top of the image is excluded to remove the horizon and image content nearthe vanishing point; (b) next the target region is segmented using a specified image feature (e.g. HSV); (c) the dominant boundary curve is then extractedfrom the filtered segmented image; and (d) the resulting line fit of the boundary is mapped into a steering command (blue arrow).

the upper portions of each image from analysis. Theseportions contain the horizon and highly distorted parts ofthe image near the vanishing point. Next, the image issegmented to differentiate between the target terrain andbackground regions. To accommodate identifying terrainsfrom diverse types of surroundings, segmentation can beconfigured to use different image features, such as hue,grayscale intensity, or a representation derived from the Hue-Saturation-Value (HSV) colorspace [13]. Akin to the seminalwork of Stauffer et al. [14], the feature centroids of the targetand non-target labels are adapted after every frame, whichimproves robustness to illumination and changes in featurecharacteristics of the terrain over time.

Following feature segmentation, small regions in the la-beled image (e.g. as seen in Fig. 2(b)) are blended into theirneighboring regions, which filters out all but the most salientand wide-scale terrain boundaries. This filtering step can beconfigured to isolate two different types of terrain bound-aries, namely an edge boundary between two visually distinctterrains such as a grass-curb divide, and a strip boundarywhere a path falls between between two background regions.

After identifying the dominant target region, a line isfit to its boundary curve. The line-fitting algorithm useshistorical information from past frames to ensure that itconsistently tracks the same boundary over time. The bound-ary line compactly represents the pose of the target terrainin image space, which is essential to determining steeringcommands for following the target. This problem could besolved through the use of geometric information such asthe camera’s pose, its intrinsic and extrinsic parameters,the incline of the ground plane, and a desired boundary-following distance. Since this information may not be readilyavailable for a given platform, and could also change duringoperation depending on task conditions and user preferences,we instead utilize an adaptive parameterized linear mapping:

yr = M1χ +M2φ +M3

This mapping relies on the x-intersection χ between theboundary line and the bottom of the image frame, as wellas the slope of the boundary φ . The steering command yr isgenerated through a linear combination of these two values.The weighting factors, M1 and M2, and the bias term, M3, arefree parameters that can be adapted on-the-fly both in the face

of changing user preference (e.g., to move closer or fartherfrom the target boundary), as well as to accommodate non-rigid system geometry (e.g., perturbations to the camera’spositioning).

Overall, our boundary tracking algorithm has several pa-rameters that can be adapted to perform various driving tasks.These include discrete-value settings such as the boundarytype Tb ∈ {Edge, Strip} and the segmentation feature choiceTs ∈ {Hue, Grayscale, HSV}. Additional parameters withcontinuous values include the horizon cut-off threshold H0,and the mapping coefficients M1, M2, and M3. The followingsection will describe our automated approach for adaptingthese parameters based on intermittent human input.

IV. ADAPTATION FROM PARTICIPATION

Several parameters of our boundary tracking algorithmmust be specified in order to achieve desirable drivingbehavior. In principle, parameter values could be set manu-ally by a human user or expert engineer, although, beyondproviding a rough estimate, this fine-tuning process would behighly laborious. In addition, manual parameter adjustmentsduring operation to cope with changes in task objectivesand conditions could easily overload the human and mightintroduce safety hazards if carried out while in motion.Instead, we instantiated the Adaptation from Participation(AfP) paradigm to achieve the task of adjusting parametervalues automatically in a robust and flexible manner.

AfP capitalizes on a supervisor-worker style relationshipbetween a human passenger and the autonomous drivingsystem, as shown in Fig. 3. Specifically, we rely on thehuman to steer the vehicle to specify an initial navigationtarget, to correct erroneous motions made by the autonomoussystem, and to make navigation-level adjustments such asdeciding where to go at an intersection or switching toa different road or track. During these periods of humandriving, the AfP module continuously updates the parametervalues of the autonomous system to settings that mostaccurately resemble the passenger’s actions. Once the au-tonomous system can demonstrate competent driving thatfollows the human’s intents through on-screen feedback, thepassenger can relinquish vehicular control and proceed topassively supervise the road-following progress. This allowsfor largely autonomous operation of the vehicle, with humaninputs occurring rarely and only for short periods of time.

AutonomousVehicle

Supervisor

SL-Commander B. Tracker

(AI cmds)(actions)

(intervening cmds)

(updated params)

AfP Module

(world state)

Trails/Paths

Fig. 3. The Adaptation from Participation (AfP) paradigm calls for theautonomous boundary tracker to do the bulk of the driving workload,whereas the human passenger takes on a supervisory role and only needs tointervene to correct misbehaviors or to change task objectives. Parametersof the boundary tracker are adjusted automatically during periods of humanintervention by the AfP module.

The resulting adaptive system can also adjust to dynamicchanges to task conditions through occasional periods ofhuman assistance.

More formally, the objective of AfP is to optimize therobot autonomy’s parameters θ throughout periods of con-tiguous human interventions. Conceptually, we consider thatfor any moment in time t there exists a set of optimal pa-rameter values

{θ ∗ (t)

}for the robot’s autonomous system,

which would lead to the best possible behaviors. However,these optimal settings are typically not known, and mustbe determined through computational search. They may alsochange dynamically with variations in the environment (e.g.scene lighting), the robot’s configuration (e.g. tire wear), orthe desired navigation objective. By assuming that the humanwill react to these changes through manual control interven-tions, AfP aims to maintain an up-to-date θ ∗ by deducinginformation from the human’s intervening commands yh.

The dynamic adaptation to changing task objectives andconditions differentiates AfP from the related problem ofLearning from Demonstration (LfD) [15], [16], since demon-strations in LfD are meant to teach an autonomous agent howto perform a single target task. Moreover, AfP encouragescontinuous interactions between the autonomous system andthe human participant, working towards the task objectiveat all times. In contrast, conventional LfD agents and theirteachers must take turns in demonstrating and repeating taskbehaviors, in a sequential manner [17], [15].

We used the Adaptive Parameter EXploration (APEX)algorithm [9] to realize the AfP paradigm onboard our au-tonomous driving system. APEX is an online, any-time, andparallelized implementation of model-free policy search [18].Specifically, APEX operates by simulating multiple instancesof the boundary tracker, each with constantly-changing pa-rameter settings that are being optimized using gradient-based search. The resulting steering commands yr fromeach simulated instance are compared against the human’slatest commands yh to form a similarity score, which isaccumulated over time. APEX selects the tracker instancethat best matches the human driver’s past actions throughan temporal-discounted score metric, and uses the winning

Fig. 4. Satellite view of the test site at the Canadian Space Agency. Thehighlighted circuit depicts the gravel pathway used for our experiments,which has an approximate 1 km length.

Fig. 5. During experiments, a human driver is responsible for the vehicle’ssafety, through control over the brakes and emergency stop. Anotherhuman passenger supervises the autonomous driving system through alaptop display, and occasionally makes manual control interventions usinga gamepad.

tracker instance’s steering commands to drive the actual ve-hicle. APEX’s generic design allows it to optimize differentparameter types, including continuous values and discretesettings. Further implementation details are elaborated in [9].

V. FIELD EXPERIMENTS

We validated our interactive autonomous driving systemthrough a series of field trials, where onboard passengerscollaborated with our system to drive through several terraintypes and to adapt to diverse task conditions. The primarygoal of these field trials was to examine the system’s abilityto adapt to each user’s preferred driving behaviors, to copewith dynamic challenges in the environment, and to adaptto in-situ modifications of the hardware system. This sectionwill present our experimental setup as well as various resultsof our field trials.

A. Evaluation Scenario

Four different participants collaborated with our au-tonomous system in order to jointly perform driving tasks.

(a) (b) (c) (d)

Fig. 6. Snapshots of a run through the primary test circuit: (a) the human passenger began by training the boundary tracker at 3 km/h speed to follow apreviously-unseen gravel road, with the user’s command shown as a green arrow; (b) shortly after, the passenger relinquished control to the autonomoussystem, which continued to track the path, with its autonomous steering command shown as a blue arrow; (c) the passenger incrementally ramped up theautonomous driving speed to 20 km/h after witnessing robust tracking of the gravel-grass border through diverse surroundings; and (d) the run concludednear a large tent that produced a confusing secondary boundary, although autonomous tracking of the gravel path remained unfaltering.

(a) (b) (c) (d)

Fig. 7. Sample segment demonstrating interactive re-learning following a change in the camera’s positioning: (a) following initial manual training, thevehicle autonomously tracked the gravel pathway, with steering command shown as a blue arrow; (b) the passenger began issuing intervening commands,shown with the green arrow, in preparation to change the camera’s pose while the vehicle is in motion; (c) after panning the camera to the right anddownwards, this unexpected change in image perspective caused the robot’s command to differ from the user’s desired steering; however (d) after AfPswiftly adapted parameter settings to match the updated camera pose, the human quickly returned control back to autonomous driving system.

These experiments were performed within a controlled arealocated at the Canadian Space Agency, as part of the 2014NSERC Canadian Field Robotics Network (NCFRN) fieldtrials. The primary task goal for each user was to circum-navigate a closed-loop gravel pathway, highlighted in Fig. 4.This pathway featured several challenging scenario segments,including a forested patch, narrow land bridges passing overpipes, and water-filled ditches. In addition to this primarytest circuit, several participants also directed the vehicle tonavigate less well-defined paths, such as along the tire tracksof another vehicle that had previously passed through amuddy field.

As seen in Fig. 5, during these experiments a human in thedriver seat assumed the role of safety manager, and had con-trol over the brakes and vehicle’s kill switch. The passenger-side participant interacted with the adaptive boundary trackerusing a gamepad, and also received visual feedback on alaptop screen. The boundary tracker exerted control over thevehicle’s heading, except when the passenger intervened toprovide manual steering directions. The vehicle operated ata constant velocity, and although the passenger could changethis velocity setting by fixed increments, all participantswere instructed to be cautious during the requisite safetybriefing. Therefore, actual vehicle speed correlated with eachparticipant’s comfort levels, according to user testimonials.

Every participant was able to successfully complete the1 km primary test circuit safely, demonstrating the basiccompetency of our driving system. The remainder of this

(a) (b)

Fig. 8. During a trial run, the participant steered away from the gravelroad and onto wheel tracks on a muddy field; (a) following a brief periodof manual steering commands, which are shown as green arrows, (b) oursystem adapted to autonomously steer on the left wheel track, where theautonomous steering commands are shown as blue arrows.

section will provide further analysis of the achieved results.1

B. Sample Adaptation Sequences

Overall, our adaptive driving system demonstrated a robustability to navigate the gravel path, operating predominantlyin autonomous mode. Human intervention occurred at thebeginning of each session to update the randomly-initializedparameter settings, and a few more times throughout eachrun at the discretion of the user. Fig. 6 illustrates a typicalrun, where the initial human training allows the system tosubsequently perform the task robustly through a number ofsituations such as changes in lighting and variations in ter-rain. Fig. 8 shows the vehicle adapting to drive along a set of

1See bit.ly/ICIUS2014AFP for videos of additional experiment runs.

0 5 10 15 20 25 30

Time (sec)

He

ad

ing

Co

mm

an

d

Human Interv.

Autonomous

(a)

0 5 10 15 20 25 30

Time (sec)

Inte

rcep

t G

ain

M1

Human Interv.

Autonomous

(b)

Fig. 9. Quantitative results reflecting interactive adaptation during a portion of a field session: (a) the robot’s autonomous commands quickly converge tothe human’s intervening steering controls; (b) for the same period, the intercept gain mapping parameter M1 is continuously optimized during interventionsas our adaptive system seeks for settings that best match the user’s latest steering commands. All other parameters tuned by APEX exhibited similaradaptation trends.

0 50 100 150 200 2500

5

10

15

20

Time (sec)

Sp

ee

d (

km

/h)

Fig. 10. Progression of vehicle’s speed during an adaptive autonomousdriving session.

muddy wheel tracks, which demonstrates the generalizabilityof our system across terrains.

A key characteristic of our adaptive autonomous drivingsystem is the ability to quickly adapt to changing taskscenarios. Evaluations of such adaptations were carried outthroughout our field experiments, both naturally in responseto changing environmental conditions (as illustrated byFig. 6(c)), and through artificially induced scenarios suchas adjusting the vehicle’s camera pose mid-way through adriving session, as shown in Fig. 7. This latter inducedscenario simulates accidental perturbations to the robot’sphysical configuration. Even in this drastic scenario, onlya small amount of human intervention was required for thesystem to adapt its parameter values accordingly.

C. Quantitative Analysis

During our field experiments, the vehicle’s speed wasdirectly controlled by the human passenger, and was onlyincreased as each person felt comfortable with system perfor-mance. Fig. 10 depicts the progression of speed throughouta run, which reflects a pattern typical to all users. Startingfrom conservative speeds as low as 2 km/h, participants

would quickly ramp up the velocity as the autonomoussystem proceeded to drive smoothly and without incident,up to 20 km/h during some sessions, and at an overallaverage speed of 8.98 km/h. The average speed duringuser interventions was 8.76 km/h, compared to 9.35 km/hachieved during autonomous control. This is an indicationof the confidence in the autonomous system, developed byour study participants.

Fig. 9 provides a detailed view of the parameter adaptationprocess that occurred during periods of human interventions.Whenever the passenger intervened, the parameters of theboundary tracking system were continuously optimized inorder to bring the autonomous heading commands into align-ment with the human’s. At the beginning of each sequence ofhuman intervention, the parameter settings of the boundarytracker would fluctuate significantly in order to reduce theinitially large error. As this adaptation process continued,the parameter settings would eventually converge to a stableconfiguration that consistently matched the user’s commands.

VI. CONCLUSIONS

In this work, we demonstrated the use of the Adaptationfrom Participation paradigm to realize an interactive au-tonomous driving system that can handle diverse challengingterrains. This comprised of a proof-of-concept system thatadapts to drive autonomously using vision on diverse pathsand trails, with occasional control interventions from ahuman passenger. We also developed a standardized ROSinterface for the SL-Commander vehicle, which facilitatedthe swift integration of, and extensions to, our existingvisual navigation system. Our experimental results with thisinteractive autonomous vehicle have validated the promisesof AfP for driving use, both through positive user feedback,and through consistent rapid adaptations to drive on unseenpaths and in dynamically changing environments. The rapidadaptation to novel changing situations is a key demonstrated

feature of our system, which will be beneficial during up-coming deployments in a number of challenging outdoorapplication domains, such as for agriculture, mining, andforestry.

In addition to the adaptation abilities, another beneficialfeature of AfP is that the human participants were alwaysin full control over the vehicle, through immediate overrideof the autonomous system’s steering commands, and alsothrough command over the autonomous vehicle’s velocitysettings. This led to an interesting side-effect where severalusers pro-actively took over manual control despite observ-ing consistent autonomous performance during hazardoussections of the circuit, for example when driving througha narrow land bridge surrounded by ditches and trees, andwhile traveling along a segment bordering a watery trench.This highlights the fact that the AfP paradigm preserves thehuman’s sense of being in charge of the driving process. Inaddition, such pro-active interventions are beneficial to AfP,as they provide further opportunities to refine system param-eters, leading to greater robustness. The ease with which AfPcan gather such training information in an interactive manneris largely unique amongst competing approaches.

Several remaining challenges were also apparent duringour field experiments. The user interface for our prototypesystem was based on a laptop screen for visual feedbackand a gamepad controller for user input. Although sufficient,these interfaces proved somewhat awkward to use in theconfined passenger seat area. More natural forms of feed-back, such as visualization on an augmented display, anduse of the steering wheel for control input are likely interfaceimprovements for our ongoing investigations.

A primary technical aspect of our adaptive system is thedynamic updating of system parameters based on intermittentmanual steering commands. While our current adaptationapproach is highly capable, we are continuing to enhanceits functionality and generality. One promising area forimprovement is the explicitly modeling of user intentionfor different interaction periods. As we have noted, mostmanual interventions during our field trials were meant tocorrect unsafe autonomous behavior or to specify a newtask objective. However, in few instances the causes ofintervention were independent of the performance of the au-tonomy, such as over-caution in anticipation of hazards in thevehicle’s non-immediate surroundings. Correctly capturingthese intentions will allow our system to produce desirabledriving behavior more rapidly in the future. Given that oursystem is largely agnostic to the underlying system beingadapted, our Adaptation from Participation framework alsohas the potential to increase efficiency and add value to manyother collaborative human-robot teams in the near future.

ACKNOWLEDGMENT

We would like to acknowledge the NSERC CanadianField Robotics Network (NCFRN) for its funding support.We would also like to thank the Canadian Space Agency(CSA) for allowing us to use the SL-Commander vehicleand for providing access to testing terrains. Furthermore, weare grateful to MacDonald Dettwiler and Associates (MDA)for supporting this work through discussions during ourdevelopment of the ROS interface for the SL-Commander.

REFERENCES

[1] “U.S. Department of Transportation Releases Policy on AutonomatedVehicle Development,” 1.usa.gov/18w19BA, accessed: 2014-06-25.

[2] M. Nimmelman, “SL-Commander - Design highlights,” in UnmannedSystems Canada, 2012.

[3] C. Langley, R. McCoubrey, J. Ratti, N. Ghafoor, P. Fulford, C. Gagnon,T. D. Barfoot, and M. Picard, “HERCULES: Analogue testing of aCanadian lunar rover prototype,” in Proc. of the 64th InternationalAstronautical Congress (IAC), 2013.

[4] “Commander E LSV,” http://can-am.brp.com/off-road/side-by-side/commander/commander-e-lsv.html, accessed: 2014-06-25.

[5] “SAE AS-4 JAUS Standard,” http://www.sae.org/standardsdev/jaus.htm,accessed: 2014-06-25.

[6] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. B. Foote, J. Leibs,R. Wheeler, and A. Y. Ng, “ROS: An open-source robot operatingsystem,” in ICRA Workshop on Open Source Software, 2009.

[7] C. Thorpe, M. Hebert, T. Kanade, and S. Shafer, “Vision and naviga-tion for the Carnegie-Mellon NavLab,” IEEE Transactions on PatternAnalysis and Machine Intelligence, vol. 10, no. 3, pp. 362 – 373, 1988.

[8] “ROS robotics news: autonomous cars Archives,”http://www.ros.org/news/robots/autonomous-cars/, accessed: 2014-05-31.

[9] A. Xu, A. Kalmbach, and G. Dudek, “Adaptive Parameter EXploration(APEX): Adaptation of robot autonomy from human participation,” inProc. of the IEEE Int. Conf. on Robotics and Automation (ICRA’14),2014.

[10] D. A. Pomerleau, “Efficient training of artificial neural networks forautonomous navigation,” Neural Computing, vol. 3, no. 1, pp. 88–97,1991.

[11] Y. Ma, J. Koscka, and S. S. Sastry, “Vision guided navigation fora nonholonomic mobile robot,” IEEE Transactions on Robotics andAutomation, vol. 15, no. 3, pp. 521–536, 1999.

[12] J. Crisman and C. Thorpe, “Color vision for road following,” in Proc.of SPIE Conference on Mobile Robots, 1988.

[13] S. Sural, G. Qian, and S. Pramanik, “Segmentation and histogramgeneration using the HSV color space for image retrieval,” in Proc.of the IEEE Int. Conf. on Image Processing (ICIP ’02), 2002, pp.589–592.

[14] C. Stauffer and W. E. L. Grimson, “Adaptive background mixturemodels for real-time tracking,” in Proc. of the IEEE Computer SocietyConf. on Computer Vision and Pattern Recognition (CVPR’99), vol. 2,1999, pp. 252–258.

[15] A. Billard, S. Calinon, R. Dillmann, and S. Schaal, “Survey: Robotprogramming by demonstration,” Handbook of Robotics, ch. 59, 2008.

[16] B. D. Argall, S. Chernova, M. Veloso, and B. Browning, “A surveyof robot learning from demonstration,” Robotics and AutonomousSystems, vol. 57, no. 5, pp. 469–483, 2009.

[17] P. Abbeel and A. Y. Ng, “Apprenticeship learning via inverse rein-forcement learning,” in Proc. of the Int. Conf. on Machine Learning(ICML’04), 2004.

[18] M. P. Deisenroth, G. Neumann, and J. Peters, “A survey on policysearch for robotics.” Foundations and Trends in Robotics, vol. 2, no.1-2, pp. 1–142, 2013.