coordinator project manager linda meiby … · 2020. 11. 20. · page 1 of 44 finalreport_safer...

of44FinalReport_SAFERPre-studiesStageV_RemoteControl_Submitted.docx

Reference FP04

ProjectTitle Humanfactorsrelatedtoremotecontrolofautomatedheavyvehicles

Coordinator Scania

ProjectManager LindaMeiby([email protected])

ProjectDuration 2020-01-15–2020-04-30


ContentsSummary ................................................................................................................................................ 4

HUMANFACTORSRELATEDTOREMOTECONTROLOFAUTOMATEDHEAVYVEHICLES ............ 5

1. Background .................................................................................................................................... 5

2. Projectsetup ................................................................................................................................. 6

2.3 Purpose ...................................................................................................................................... 6

2.4 Objectives .................................................................................................................................. 6

2.5 Projectperiod ........................................................................................................................... 6

2.6 Partners ..................................................................................................................................... 6

3. Methodandactivities .................................................................................................................... 7

4. ResultsandDeliverables ............................................................................................................... 8

5. Theoreticalframeworkforremoteoperation ............................................................................. 8

6. Remoteoperationanditscomplexity .......................................................................................... 9

7. Stateoftheart .............................................................................................................................. 13

7.1 Literaturereview .................................................................................................................... 13

7.2 Remotedrivingcompanies .................................................................................................... 17

7.3 RelevantSwedishresearchprojects ..................................................................................... 18

iQMatic&iQPilot .......................................................................................................................... 19

DriverSense ................................................................................................................................... 20

AVTCTphase1andphase2 ......................................................................................................... 21

ITRL’slabfacilities ....................................................................................................................... 22

REDO ............................................................................................................................................. 22

MERGEN ........................................................................................................................................ 23

7.4 RelevantEuropeanprojects ................................................................................................... 23

7.5 Interviewsandworkshops .................................................................................................... 24

InterviewREDOproject:WP2 ..................................................................................................... 24

InterviewREDOproject:WP3 ..................................................................................................... 25

InterviewVoysys ........................................................................................................................... 25

Workshop1 ................................................................................................................................... 27

Workshop2 ................................................................................................................................... 27


8 Researchgapsandchallenges ..................................................................................................... 29

8.1 Literaturereviewresearchquestions ................................................................................... 29

8.2 Workshop1researchquestions ............................................................................................ 29

8.3 Workshop2researchquestions ............................................................................................ 30

9 Conclusions,LessonsLearntandNextSteps ............................................................................. 37

10 DisseminationandPublications ............................................................................................ 40

11 Acknowledgement .................................................................................................................. 40

12 References ............................................................................................................................... 40


SummaryCurrently,mosthighlyautomatedvehiclesstillrequirethepresenceofahumansafetyoperatorinthevehicle,anditisevidentthatautomateddrivingwithouthuman“fallback”mightbedistant.Ontheotherhand,havingahumanoperatorinthevehiclejeopardizesmajoranticipatedbenefitsofautomateddriving–productivity.Thisisespeciallyevidentwhenitcomestoheavyautomatedvehicles.Tobridgethisgap,stakeholdersareexploringteleoperations technology, which enables highly automated vehicles to be remotelyoperated ifnecessary.Butremoteoperationcomeswith itsownchallenges,bothfromtechnicalandhumanbehaviorperspectives.InthisSAFERco-financedprestudy,ScaniaandRISEhaveidentifiedpotentialsafetychallengesandresearchgapsrelatedtohumanbehaviorinthecontextofremoteoperationofheavyautomatedvehicles.Ageneralviewofthehumanfactorsrelatedchallengeswithintheremoteoperationtopiccan be summarized by highlighting phenomena such as physical and psychologicaldistancing, screen delays, network latency delays, inefficient interface designs, andhumanoperator’scognitivelimitations.Thesearenotexclusivetoonesingleoperationallevel,orapplicationtype,andareofteninterrelated.Alargerbodyofscientificworkcanbefoundrelatedtohumanfactorsinremoteoperationinotherdomains(e.g.,robotics,aerialdrones,military).Someofthefindingsfromthesedomainscanhavevaluefortheautomotivedomain,however,generallydesignrequirementsarenotdirectlytransferablebetweendomainsastherearedomainspecificchallenges.Anoverall conclusion from theprestudy is thathuman factors in remoteoperationofhighlyautomatedroadvehicleshavebeensomewhatneglectedbyindustryandresearchcommunity. By providing an overall conceptualization of remote operation and itscomplexity,a theoretical framework,astateof theartoverview,anda listofgapsandchallenges,theexpectationisthatthispre-studywillstimulatemoreactivitiesinthearea.


HUMANFACTORSRELATEDTOREMOTECONTROLOFAUTOMATEDHEAVYVEHICLESAuthors:Habibovic,A.,Andersson,J.,Castor,M.,Meiby,L.andRizgary,D.

1. BackgroundCurrentquestionsaboutautomatedvehicles(AVs)donotrevolvearoundwhethersuchtechnologiesshouldorshouldnotbeimplemented;theyarealreadywithus.Rather,suchquestions aremore andmore focused on how such technologieswill impact evolvingtransportation systems, our social world, and the individuals who live within it andwhethersuchsystemsoughttobefullyautomatedorremainundersomeformofdirecthumancontrol.Itmaybethathumaninterventiondoesnotnecessarilyneedtobefrominsidethevehicle;insteadthephysicallocationofthehumancontrollercouldberemotefromtheactualvehicleitself.Regardlessofthespecificspatialrelationsbetweenthecontrollerandvehicle,thehumanoperatorwillrequireeffectivesituationawarenesscalibratedtoambientenvironmentaldemandsatall timesbecause thepointatwhichanysuchhuman interventionwillberequired remains unpredictable as yet. Again, this raises the spectre of prolongedvigilance and its well-known decrement and response failure. This human-as-backuparchitecture,whichremovesthepersonfrommomentarycontrolandinsteadplaceshimorherinasupervisorycontext,inmanywaysdefeatstheveryideaofautomationinthefirstplace.However,itiscurrentlyseenasanecessarytransitionphase,andawaytoputautomatedvehiclesonthemarketinthenearfuture.DuringthelastfiveyearsoftheAVsdevelopmentboom,therehasbeenagrowthinthenumberofcommercialcompaniespromotingcontrolconceptsandtechnicalsolutionsforteleoperationwhichenablesmoredirecthumancontrolofheavyroadvehicles,althoughfrom a remote location, i.e. remote driving. Many concepts and solutions for remotecontrolexistwithcontroltargetingdifferentlevelsofcontrolofavehicle,asexemplifiedin Figure 1. In this pre-study, we aim at identifying potential safety challenges andresearch gaps related to human factors in the context of remote control of heavyautomatedvehiclesacrossthesedomains.

Thepre-studyhasaquitelimitedbudget(intotal200.000SEK)andanswerstomanyoftheidentifiedresearchquestionsremainthusunanswered,andmanyquestionshavenotyetbeenformulated.Theprimarypurposeofthereportistoidentifyresearchquestionsthatcanbeusedasthestartingpointfornewresearchprojects,internalorexternaltotheSAFERconsortia.


Figure1.Conceptualsketchofcontrolofavehicleondifferentlevels.

2. Projectsetup2.3 Purpose

Currently,mostheavyautomatedvehicles(HAV)requireahumansafetyoperatorinthevehicle,anditisevidentthatHAVswithouthuman“fallback”mightbedistant.Atthesametime,havingahumansafetyoperatorinthevehiclejeopardizesmajoranticipatedbenefitsof the automated driving – transport efficiency and safety. To bridge this gap,stakeholdersareurgentlyexploringremoteoperation,whichenablesseveralHAVstoberemotelyoperatedbyonehumanoperator.But remoteoperation comeswith itsownchallenges, both from technical and human factors perspectives. In particular, humanfactor challenges are currently rather unexplored. Thepurpose of this pre-study is toidentify potential safety challenges and research gaps related to human factors in thecontextofremotecontrolof(heavy)automatedvehicles

2.4 ObjectivesTheoverallobjectiveofthepre-studywastogathertheoreticalandpracticalexperiencesandknowledge,both from theareaof remote controlof automatedvehiclesand fromotherdomains(e.g.,trafficcontrolcentresatairports,nuclearpowerstations,etc.)whereremotecontrolhasbeeninuseformanyyears.

Thefollowingsub-objectiveswereenvisionedatthestartofthepre-study:1)literaturereview to identify the state-of-the-art related to human factors and remote control ofautomatedvehiclesandremotecontrolinotherdomains,2)interviewsandworkshopswith relevant stakeholders, 3) workshops to identify research questions for furtherresearchinthecontextofheavyautomatedvehicleswithinatransportationhubaswellasbetweentransportationhubs.Alltheseobjectiveshavebeenaddressedasplanned.

2.5 ProjectperiodThepre-studyhasbeencarriedoutintheperiodJanuary-April2020.

2.6 PartnersTheprojectpartnershavebeenScaniaandRISEResearchInstitutesofSweden.Severalother SAFER-partners have participated in workshops and interviews and therebyindirectlysupportedtheproject.


3. MethodandactivitiesThefollowingmethodswereutilizedintheprestudy:

• Literaturereviewofthescientificliterature,see7.1• Websearchofrelevantcommerciallyavailablesolutions,see7.2.• WebsearchofrelevantSwedishandEuropeanprojects,see7.3and7.4• Interviewswithresearchersanddevelopers,see7.3and7.5• Workshops,see7.5.

Theprestudywasinitiatedbyareviewofrelevantscientific literature,bothspecifictoautomotiveapplicationsaswellasotherdomains,aswellasageneralreviewofremotedriving companies (Figure 2). Concurrent with the literature review, a number ofrepresentatives fromrelevantSwedish researchprojectswas interviewed.The resultsfromthisstate-of-the-artreviewwastransformed,duringtheinternalWS0into50draftresearch questions. These 50 questions were then discussed in a larger group withexternal representatives during WS 1, where the number of research question werelimiteddownto21,whichinturnwasdiscussedduringasecondworkshopwithotherexternalexperts.

Figure2.Schematicrepresentationoftheapproachduringtheprestudy.

In the literature review, search phrases were divided into Teleoperation and humanfactorsandTeleoperationandhumanfactorsrelatedtoon-roadvehicles.Intotal,19searchphraseswereusedinfivedatabases/searchengines,whereineightsearchphraseswere


used in the firstmentionedsegment,andelevensearchphraseswereused in the last-mentioned segment.Choiceof search termswerebasedonour reviewof focusareas:humanfactors,automatedvehiclesandteleoperation.Variationsofsub-categorieswithinhuman factors were used such as “cognition”, and variation of related topics toteleoperation was used such as “remote control”. Relevant papers and reports wereselectedfromthesearchresultsbasedontitleandabstractdescriptions.Mainly,paperandreportswereselectedbasedoncontentrelevanttohumanfactorsandteleoperation,whilst application domainwas not restricted. The documents that considered humanfactorswithinteleoperationofotherdomainareasbesidesautomotive,suchasmilitaryvehicles,spacerovers,dronesandotherrobotswerealsoselected,reasoningthatbestpracticescanbefoundinareasthathaveinvestigatedteleoperationforalongertime.Thedocuments were categorized in a three-step manner firstly based on thereview/experimental orientation of the publication, secondly based on relevancecategories,i.e.whethertheywerefocusedoncognition,HMI,neuroscience,behaviour,orsimplymoregeneralwithinhumanfactors.Lastly,theywerealsocategorizedbasedondomainarea,i.e.automotive,aerial,maritime,roboticsetc.

4. ResultsandDeliverablesRoadtrafficpresentsacomplex,ever-changingenvironmentwheresafetyshouldbetheprimary concern. The results from our prestudy show that the success of remoteoperation will, however, be affected by many inter-dependent factors specific to theremote nature of the tasks. For example, challenges related to situational awareness,hand-over,telepresence,andworkloadmight,ifnotproperlyaccountedforinthedesignofHMI,leadtoriskysituationsaswellaspoorexperienceandworkconditionsforremoteoperators.Thedeliverablesfromthisprestudycanbesummarizedinthefollowingway:

• Atheoreticalframeworkforremoteoperation• Conceptualizationofremoteoperationanditscomplexity• Understandingof thecurrent stateof theart, aswell asunderstandingofwhat

questionshavebeen(orarebeingaddressed)inrelatedresearchprojects.• Listofchallengesandresearchgapsfromahumanfactors’perspective.• EstablishedanetworkamongSwedishresearcherswithinterestinthefield.• Developedaknowledgebasetobeusedincontinuationresearchproject(s).• Apaperdraftisunderdevelopment.

Theseresultsanddeliverablesaredescribedinmoredetailinthefollowingsections.

5. TheoreticalframeworkforremoteoperationAsapparentfromthereviewofthefield,researchanddevelopmentconcerningremotecontrolisacomplexfieldwithmanyvariablesthatinfluencedesigndecisions,asisthecaseinmosthumanfactorsrelatedprojects.Thissectiondescribesaframeworkandamodelonhowthefieldandremotedrivingapplicationscouldbestructuredandanalysedfurther.


To characterise remote driving, a system of system approach with emphasis on theinterplay between humans, technology and organisation was deliberately chosen.Technologyishereusedinabroadsense,whereitcanbeseenasboth“hard”(i.e.vehicles,infrastructure and control systems) and “soft” (i.e. legislation, regulation) systems(Vicente, 2003).Thedimensions in Figure4 arebasedon a frameworkdeveloped fordomaincomparison(Osvalderetal.,2009),whilethe“properties”withinthecircleareitemsthathaveemergedthroughoutthepre-study.Thepurposeofthefigureisnottogiveanexhaustivelistoffactorsthatinfluenceremotedriving,butratherwithbroadstrokesaidunderstandingandpaintthecanvasofremotedrivingcomplexity.

AmodelfromtheResilienceEngineeringtradition,whichcanbeusefulforanalysisistheExtended Control Model, ECOM, by Hollnagel and Woods (2005). The ECOM modelpresentedinFigure3consistsoffourinterconnectedlevelsofcontroltasks.Thetrackingleveldescribesthecontrolsthatareneededforreal-timecontrolofavehicle,i.e.controlonanoperativelevel.Theregulatinglayerdescribestargetvaluesandcontrolsthatareneededfortacticalcontrol.Themonitoringlayercheckswhetherthevehicleisenroutetothe destination and monitors signals from the environment. The targeting leveldeterminesthedestination.Eachlayerofcontrolischeckedbytheabovelayerinorderforthevehicletomeetthegoal,butchangesonalowerlevelcanaffectupperlevels,e.g.ifthereisapotholeintheroadthetrajectoryofthevehicleshouldbechanged.The ECOMmodels levels can be compared with the distinction between operational,tacticalandstrategicallevelofcontrolthatwasusedintheprestudydiscussions.

Figure3.ExtendedControlModel,ECOM(Hollnagel&Woods,2005).

6. RemoteoperationanditscomplexityDespitetheanticipatedbenefits,remoteoperationcomeswithitsownchallenges,bothfromtechnicalandhumanbehaviourperspectives.Remoteoperationisarathercomplexand often involves several parallel activities. Figure 4 provides some insight to howdifferentfactorsinfluencethedesignofaspecificremoteoperationapplication.


Figure4.Dimensionofremoteoperation.

Theoperationaldomain inFigure4 is the static anddynamic environmentwhere thedrivingtakesplace.Thetrafficandtheinfrastructuretogetherwithlaws,regulationsandimplicitculturedefinetheconstraintsthattheremoteoperatorhastoactwithin.Controlcan be exerted on operational, tactical or strategical level, often intertwined, and incombinationwithautomateddrivingfunctions.Apartfromoperationaldrivingtherearealsoothertasksthatemergewhenthedriveractsremotelysupportedbyautomation.Intacticalandstrategicalcontrol,itislikelythattheremotedriverwillinsteadmonitorandplan for several vehicles at a time and intervene only when something does not goaccordingtoplan.Thedifferentcontrolmodesandnewtaskscallfornewtypesofhuman-automation interfaces to support bothmonitoring and operational driving as well asefficientshiftbetweenthesemodes.Humanoperatorsinremotedrivingmightneedotherskillsandtrainingcomparedtodriversofmanuallyoperatedtrucks.Cognitiveworkloadcanbeexpectedtovaryandrapidlychangeifdrivershavetoaddressissueswithseveraltrucksneedingassistanceatthesametime(dependingonnumberoftruckscontrolledsimultaneously).Thetraditionalorganisationoftransportcompaniesislikelytoundergochangeinthefaceofincreasinguseofautomation.Whenremotedriverscontrolseveraltrucks,fewertruckswillbeneeded,thetrainingandlicensingmightbesubjecttochange.Thequestionofliabilityincaseofanaccidentmayalsobeuncleariftheresponsibilitybetweenthehumandriverandautomaticcontrolsystemisblurry.ThebasicdimensionsdescribedinFigure4arealsointerconnected.Figure5describessomeofthedependenciesthatinfluencethedesignofremotecontrolsystems.TheyellowcirclesarethedimensionsfromFigure4andthepinksquaresdescribe(non-exhaustive)examplesofhowthedimensionscanbeinterrelated.Thepurposeistoillustratetheneedforasystemperspectiveindesignofremotedrivingsystems,sinceadesigndecisionwillhaveaneffectonotherpartsofthesystem.

Dimensions of remote operation



Figure5.Remotecontroldependencies.

On top of that, in remote operation a number of different users can be identified,dependingontheusecase,withsomeexampleusersidentifiedinFigure6.

Figure6.Exampleuserroles.


One primary driver for remote operation is the expected need to be able to overridedecisions,trafficregulationsandimpassesthatthelogiccontrollingvehiclesonSAElevel4cangetstuck in, leadingtounsafesituationsor traffic jams.Oneexamplecouldbeasituationwheresensors in thevehicle interpretaplasticbag in thestreetasa foreignobjectthatblocksthetraffic.Aremoteoperatorcouldthenbenotified,quicklyassumecontrol,assessthesituationandallowthevehicletodriveintotheplasticbag,orguidearoundtheobjectbytakingcontrolthrougharemotewheel.Aremoteoperatorcouldalsobemanagingthelocalcoordinationforthevehiclewithinahub,e.g.alogisticsterminal,withallthecomplexitiesandsynchronizationwithothervehiclesandlogisticsprocessesthat need to be managed within the hub. The notion is thus that remote drivingfunctionalityreducestheneedtosolvemanyoftheedgecasesthatcouldappearinrealtrafficsituations.Thiscouldenhancethepacewithwhichvehiclesonlevel4and5canbeintroducedtothemarket.

Thatis,remoteoperationcanbedoneonoperational,tacticalorstrategiccontrolmodes,which are often intertwined, and carried out in combination with automated drivingfunctions.Apartfromoperationaldriving,therearealsoothertasksthatemergewhentheoperatoractsremotelysupportedbyautomation.Intacticalandstrategicaloperation,itislikelythattheremoteoperatorwillinsteadmonitorandplanforseveralvehiclesatatimeandinterveneonlywhensomethingdoesnotgoaccordingtoplan.Remoteoperationisenvisionedtobeutilizedforthefollowingapplications:

• Remoteassessment.Enablestheremoteoperatortoinvestigate(debug)issues.

Inremoteassessmenttheinformationflowisone-way,i.e.,thevehiclesendserrormessagesandsystemstateinformationtothehumanoperator,buttheoperatorcannotdirectlycontrolthevehicle.Thisisalwaysrelevantandcanbeseenasabasecaseforremoteoperation.

• Remoteassistance:Enablestheremoteoperatortohelpthevehicleunderstandandhandleagivensituation.Thisissometimesrelevant.

• Remoteemergencydriving:Enablestheremoteoperatorto“drive”,orevacuate,thevehicleinanemergencysituation(e.g.,atroadworks,orwhenthevehicleisstuckinacomplexsituation).Thisisrarelyrelevant,butverycriticalwhenit isneeded.

To summarize, remote operation is mainly envisioned for the following applications:remoteassessment,remoteassistanceandremoteemergencydriving.Thedesignoftheremote operation for these applications will be defined by a variety of inter-relatedfactors including operational design domain, control level (operational, tactical,strategic),tasknature,HMI,operatorskillsandtrainingaswellasorganizationtypeanditsoverallapproachtoremoteoperationandautomateddriving.


7. StateoftheartThissectionpresentsthestateoftheartregardinghumanfactorsinremoteoperation.Itis based on literature review as well as interviews and workshops with relevantstakeholders.

7.1 LiteraturereviewTeleoperatedvehicleshavebeenusedinmanydomainsduringthelast100years.NicolaiTeslademonstratedthefirstradio-controlledprototypeofafour-footshipin18981.TheDH-82BQueenBeehasbeensaidtobethemotherofallairbornedrones,knownalsoasUnmanned Air Systems (UAS)2 and became operational about 1935. The SovietTeletank3thatbecameoperationalintheatthesametimeisanotherexample,thistimeaheavygroundvehicleoperatinginpairwithamannedtankwithadistanceofupto1500metersbetweenthevehicles.Theteletankhasalsoasof2016been“reborn”intheVikhrremotecontrolledtank4.Inmodernwarfarethenumberofsystemsandapplicationsaremany,withtheUSsystemMQ-9Reaperbeingonewellknownoperationalexample.FortheMQ-9 Reaper a crew of two operates the UAS’s navigation, sensors andweapons“anywhere”ontheglobefromaremotecontrolstation.Theavailabilityofsmallerdronesforbothrecreationalandprofessionalpurposeshasalsoincreasedextensivelythatlastfewyears.Intheprofessionalmaritimedomain,RemotelyOperatedVehicles(ROVs)havebeenusedformanyyears.Thenumberofcivilianandmilitaryapplicationsofremotelyoperatedvehiclesaretoomany to list in the current report, but the US DoD is the world’s largest operator ofunmanned systems, with many different operational systems. The US Department ofDefenseUnmanned Systems IntegratedRoadmap 2013-2038 (DoD, 2013) exemplifiesmanyoftheunmannedmilitarysystemsoftoday,i.e.coveringbothsystemsthatarecalledautonomous to some degree and systems which are remote controlled rather thancapable of autonomous behaviour. The more recent version of the same roadmap,covering2017-2042(DoD,2017)highlightsfourcriticalresearchthemesforthefutureofunmanned systems formilitaryapplication.The four research themes identified thereare:

• Interoperability(Common/OpenArchitectures,ModularityandParts

Interchangeability,Compliance/Test,Evaluation,VerificationandValidation,DataStrategies,DataRights)

• Autonomy(ArtificialIntelligenceandMachineLearning,IncreasedEfficiencyandEffectiveness,Weaponization,Trust)

1 https://patents.google.com/patent/US613809A/en 2 https://en.wikipedia.org/wiki/De_Havilland_Tiger_Moth#Training 3 https://en.wikipedia.org/wiki/Teletank 4 https://rtd.rt.com/series/combat-approved-series/vikhr-reborn-as-robot/


• NetworkSecurity(CyberOperations,InformationAssurance,ElectromagneticSpectrumandElectronicWarfare)

• Human-MachineCollaboration(Human-MachineInterfaces,Human-MachineTeaming)

Someofthecriticalneeds,e.g.,weaponization,willnotbeapplicablefortheapplicationsconsideredbythecurrentprestudy,butmostoftheneedswillberelevantduringfuturedevelopmentofbothremotedrivingapplicationsandmoreautomatedvehicles.However,the current report from the prestudy focuses on the Human-Machine Collaborationaspects.

Withinthemilitarydomaintherehavebeenmanystudiesconcerningtheoperatortovehicleratio, i.e.howmanyvehiclesoneoperatorcancontrol.CummingsandGuerlain(2007)andChen,Barnes,andHarper-Sciarini(2010;2011)providereviewsofnumerousexperiments targeting the question of how many autonomous vehicles (for severaldifferentmilitary domains and automation levels) a human operator can control in asupervisorycontrolmode,rangingfrom1-12vehicles.Theconclusionsconcerningtheratiofromthesetypesofstudiesareverydependentontheoperationalcontextandthelevelofautonomythathasbeenimplementedandwillthereforenotbesummarisedinthisreport.Theliteraturealsocontainspublicationssceptictotherelevanceofquestionsregardingoperator-vehicleratio(e.g.,Galster,Knott&Brown,2006)andusefulnessofautomationlevelsdiscussionstoguidedesign(e.g.,DepartmentofDefence,2012)astheoperationalcontextandtheactualtasksthatanoperatorhastoconductduringdifferentphasesofamissionaremoreusefultodiscussinordertoinformsystemdesign.

Concerning safety analysis there is much to be learned from other transportationdomains, such as railways and aviation, where elaborate processes concerning riskanalysis and definition of safety cases have been developed. These safety cases arereviewedandapprovedbyregulatoryagenciesaftereachsignificantchangetovehiclesoroperations.Safety/riskanalysismethodssuchasCSM-RA,i.e.CommonSafetyMethod-RiskAssessment(EuropeanRailwayAgencyi.e.ERA,2016)andRAMS(Mahboob&Zio,2018)areexamplesfromtherailwaydomain.Similarsafetyanalysismethodsfromtheaviation domain have for example been defined by the International Civil AviationAuthority,i.e.ICAO,seeICAO(2018)andICAO:sSafetywebsite5.Forunmannedremoteshipoperation,Wahlströmetal.(2015),identifiedthefollowingmostprominentchallenges(andresearchgaps):informationoverload,boredom,mishapsduringchangeoversandhandoffs,lackoffeelofthevessel,constantreorientationtonewtasks,delaysincontrolandmonitoring,andtheneedforhumanunderstandinginlocalknowledgeandobjectdifferentiation(e.g., indifferentiatingbetweenhelp-seekersandpirates).Theyhavealso identifieda fewpositiveaspectsofremoteoperationofships:lackofseasicknessandphysicaldamagetothecrewinharshweatherconditions,andthe

5 https://www.icao.int/safety/safetymanagement/Pages/default.aspx


possibility to functional specialization. A recommendation from the study is that thecontrolcentresshouldreflectagilecommandandcontrol.In remote operation of container terminals and container cranes, the followingchallengeshavebeenhighlighted(Karvonenetal,2012):constantreorientationtonewtasks(theremotecontaineroperatorshavecontinuouslytoreorientthemselvestothedemands of new tasks at small intervals such as lifting a new container from trucks’chassis every 30 seconds,whereas in conventional operation the task durationsweremuchlonger)anddeterioratedsenseofspatialdimensionsinvideofeed-basedcontrol.Forremoteoperationofforestryvehicles,obstacleavoidanceandrouteplanning(e.g.,knowingwhentosteerbetweenoraroundsmallobstaclesandwhentonotavoidthem)arehighlightedaspotentialissuesaswellas(lackof)comprehensiveobjectevaluation(Ringdahl,2011).Severalreviewstudiesonthehumanfactorsofremoteoperation invariousdomainspresentcommonfactors,namely;thetimeittakestocompleteatask,errorsofoperators,responsetime,situationalawarenessandoperator’swell-being(Branniganetal.,2008;Chen,Haas&Barnes,2007;Haans&Ijsselsteijn,2012;Wahlströmetal.,2015;Gatsoulis,Virk, & Deghani-Sanij, 2010; Zunjic, 2015). Latency is described as a challenge thatincreasesoveralltimetocompletetasks,candecreaseefficiency,andresultsinincreasednumberoferrors.Similarly,adecreaseinframeratecanresultindecreaseofusability,decreaseinefficiency,andincreasenumberoferrors.Moreover,insufficientsituationalawarenessleadstofailure(Gatsoulis,Virk,&Deghani-Sanij,2010).Hand-over/Hand-offbetweenoperatorsisalsoacriticalphasewhereriskoffailureisincreased,andthereforchangingoperatorsneedtobeplannedandexecutedcarefully(Wahlströmetal.,2015).Regarding embodiment and telepresenceHaans and Ijsselsteijn (2012) state that twotechnological factors can have an effect: interactivity and vividness. Vividness implieslevel of detail in the digital environment and number of sensory modalities that areincludedfromarealoperationscenario.Bestpracticeinoperationcentresisappliedbyanalysingworkflowsoftherealtaskanddesigningtheremoteoperationcentrearoundtheworkflowsandthehumanoperator(Branniganetal.,2008).Inotherwords,applyingausercentreddesignfocusingonfactorsuch as optimum seating distance, informationpresentation, task simplification, noiseandlightoptimization.The studies of a more experimental nature show also some commonalities. Theresearchersinvestigatedhumanfactorsusingdifferenthumanmachineinterfaces(HMI)mainly through varying field of view by e.g., using head-mounted displays (Almeida,Patráo,Menesez,&Dias,2016;Boutetal.,2017;Cabralletal.,2019;Cherpillod,Floreano,& Mintchev, 2019; Hosseini & Lienkamp, 2016) and augmented reality solutions(Hedayati,Walker,&Szafir,2018).Someinvestigationsexploredusingmultiplesensoryfeedback systems. Latency/video delay showed to be a prevalent factor (Bidwell,


Holloway,&Davidoff,2014;Cabralletal.,2019;Chucholawski,Sauer,&Lienkamp,2016;Davis,Smyth,&McDowell,2010;Liuetal.,2016;Lu,Zhang,&Yang,2019;Wintersbergeretal.,2019;Wojtusch,Taubert,Graber,&Neergard,2018),andtestingdifferenttypesoflatency (fixed vs. variable) in teleoperation appears to be common (Davis, Smyth, &McDowell,2010;Liuetal.,2016;Wintersbergeretal.,2019).Someofthearticlesfocusedon mitigating the effects of poor latency by using predictive displays – displays thatpredict movement of the teleoperated vehicle and produce a smoother video duringlatency (Chucholawski, Sauer, & Lienkamp, 2016; Davis, Smyth, McDowell, 2010; Lu,Zhang,&Yang,2019).Moreover, several experimental studies investigated situation awareness (Bout et al.,2017;Hosseini&Lienkamp,2016;Wojtush,Taubert,Graber,&Neergard,2018),spatialawareness (Bout et al., 2017), tele-embodiment/tele-presence (Almeida, Patrão,Menesez,&Dias,2014),workload(Chucholawski,Sauer,&Lienkamp2016;Davis,Smyth,&McDowell,2010;Haduch&Mitchell,1995;Liuetal.,2016;Lu,Zhang,&Yang,2019;Wintersbergeretal.,2019;Wojtush,Taubert,Graber,&Neergard,2018),userexperience(Hedayati,Walker,&Szafir,2018;Wojtush,Taubert,Graber,&Neergard,2018),emotions(Nieetal.,2019),anticipatoryinteractionratio-amountofactionsabletomakewithoutwaitingforvisualresponsefrominterface(Bidwell,Holloway,&Davidoff,2014),drivingperformance (Liuetal.,2016;Hosseini&Lienkamp,2016;Wintersbergeretal.,2019;Davis,Smyth,&McDowell,2010;Cherpillod,Floreano,&Mintchev,2019),neuralactivity(Bhatetal.,2017;Nieetal.,2019)andbehaviour(Bhatetal.,2017).ResultsfromtheseexperimentalstudiesindicatethatremoteoperationismorefeasibleusingHMIthatmitigatethepsychologicallackofpresence(Almeida,Patrão&Menesez,2014;Boutetal.,2017;Haduch&Mitchell,1995).Head-mounteddisplaysareafeasibleoptionformitigatingpsychologicallackofpresence(Almeida,Patrão&Menesez,2014;Boutetal.,2017;Hosseini&Lienkamp,2016)andforbetterdrivingperformance(Cabralletal.,2019).Augmentedrealityhead-mounteddisplay(ARHMD)interfaceissignificantlysuperiortopopularmoderninterface.Faster,moreaccurateandsafertaskcompletionwere observed in teleoperation with ARHMD (Hedayati, Walker, & Szafir, 2018).Situationalawareness,userworkloadanduserexperiencehaveanimpactonefficiencyand effectiveness of teleoperation interfaces. Comprehension of the situation andreduction of unnecessary workload is thus crucial design factors for operator tasks(Wojtush,Taubert,Graber,&Nergaard,2018).Regardingemotions,operatorsperformedworse when in a higher arousal state (Nie et al., 2019). Variable latency is moredetrimentaltoteleoperatorperformancethanfixedlatency(Davis,Smyth&McDowell,2010;Liuetal.,2016;Wintersbergeretal.,2019).Latenciesabove300mshaveaneffectonperformance(Wintersbergeretal.,2019).AIRscoresdecreaseastimedelayincrease(Bidwell,Holloway,&Davidoff,2014).Mitigatinglagandlatencycantosomedegreebedone using predictive visual displays (Chucholawski, Sauer,& Lienkamp, 2016;Davis,Smyth&McDowell,2010;Lu,Zhang,&Yang,2019).Onafinalnote,theassessmentsintheseexperimentalstudiesweremainlydoneusing:workload(NASA-TLX,pupillometer,secondarytaskperformance);situationawareness


(SART);drivingperformance(lateral&longitudinaldistancetoothervehicles,collisions,lateral lane conformance,max steering angle, lane offset); and user experience (UEQ,SUS).Otherscalesandassessmentmethodswereused forexamplesimulatorsickness(SSQ), electroencephalography (EEG) for neural activity recording, self-assessmentmanikin(SAM)formeasuringarousalandemotions,anticipatoryinteractionratio(AIR),taskcompletiontime,taskaccuracy,controllability,andsensationofvelocity.Ageneralviewofthehumanfactorsrelatedchallengeswithintheremoteoperationtopiccan be summarized by highlighting phenomena such as physical and psychologicaldistancing, screen delays, network latency delays, inefficient interface designs, andhumanoperator’scognitivelimitations.Thesecategoriesarenotexclusivetoonesingleoperationallevel,orapplicationtype,andareofteninterrelated.Alargerbodyofscientificworkcanbefoundrelatedtohumanfactorsinremoteoperationinotherdomains(e.g.,robotics,aerialdrones,military).Someofthefindingsfromthesedomainscanhavevaluefor the automotive domain, however, generally design requirements are not directlytransferablebetweendomainsastherearedomainspecificchallenges. In futurework,there isvalue inprovidinga frameworkthat listshumanfactorschallengeswithin thedomain categories that have been presented in this literature review. Such literaturecouldfacilitatethedevelopmentofbestpracticesinHMIdesignforteleoperation.

7.2 RemotedrivingcompaniesDuringthelastfewyears,anumberofcommercialcompanieshavelaunchedplatformsforremotedriving,withquitesimilarconcepts.Avisittoallthelistedcommercialactors’websites reveals concepts that, at least on a conceptual HMI related level, are quitesimilar.Totheextentpossibletoevaluateduringtheprestudy,thesesolutionsappearstoprimarily implement the most basic driving tasks, e.g. controlling the wheel, speedcontrol,butmanyoftheothercontroltasks(blinkers,lighting,etc)neededinanormaltruckorbusappearstohavebeenleftout.Theprestudyhasnotinvestigatedthedifferentactors’solutionsfromadeepertechnicalperspectiveduetothelimitedprestudybudget.Theprestudyteamidentifiedthefollowingactorswithsolutionsthataremarketed,buttheresurelyexistsanumberofothercompanieswithexistingorforthcomingsolutions.Notethatthefollowinglistisinnoparticularorder.

• Voysys,https://www.voysys.se• StarskyRobotics,https://www.starsky.io• PhantomAuto,https://phantom.auto• KodiakRobotics,https://kodiak.ai• Roboauto,https://www.roboauto.tech• Qibus,http://qibus.com• Ottopia,https://ottopia.tech• Fleetonomy,https://www.fleetonomy.ai• DesignatedDriver,https://designateddriver.ai• PilotAutomotiveLabshttp://www.pilotlab.co


o https://www.youtube.com/watch?v=CJiJ3n73OBIo https://www.youtube.com/watch?v=7tpjarMyB5Qo https://www.youtube.com/watch?v=s-2IbpuFcy4

• AutonomousSolutions,ASIhttps://www.asirobots.com/platforms/nav/?tracking=Excavator+Explore+Nav

• ScottyLabsboughtbyDoorDashin2019• drive.AIboughtbyApple2019• OshkoshDefencewiththeTerramaxproductline

https://oshkoshdefense.com/advanced-technologies/terramax-unmanned-ground-vehicle-technology/#performancehttps://www.autonews.com/shift/military-working-make-its-autonomous-technology-smarter

o https://www.youtube.com/watch?v=yqPUH5SwY54o https://oshkoshdefense.com/wp-

content/uploads/2019/02/17327_TerraMax_OvrVw_A4ss_LowRes_4.20.2015.pdf

• Thereisalsoarangeofsolutionsforoperatorsclosetothevehicleandremotecontrolofbodybuildingequipmentsuchascranesandotherequipment,examplesare

o http://remoquip.com/o http://www.nerospecoscon.com/portfolio-item/hard-line/o https://torc.ai/remotetask/

7.3 RelevantSwedishresearchprojectsAsevidentfromthedifferentreviewsinSection7above,manycompanieshavedevelopedremotely controlled or teleoperated vehicles, for a long time. However, the newrequirementsofdrivingvehiclesonpublicroadsathighspeedleadstonewrequirementsofbothtechnical,legislative/trafficrulesandhumanfactorsrelatednature.TheprestudyteamidentifiedanumberofrecentorcurrentSwedishresearchprojectswhicharedescribedinthissection.AroughindicationofprojecttimelinesispresentedinFigure7below.


Figure7.Timelinefordescribedprojects

iQMatic&iQPilotScaniahasfrom2015to2020beenmanagingtwoautonomousvehicleprojectsfromtheVINNOVAFFIprogramwhichhas relation to remotedriving capabilities, although themain project focus was on vehicles on SAE level 4. Both the iQMatic and the iQPilotprojectswereearlytechnicaldevelopmentprojectsrelatedtoAVs,withsomeHMIrelatedwork packages, e.g. development design concepts and prototyping for a control roomenablingcontroloffleetsofAVsintheiQMaticproject.During the larger AV-demo at Scania in June 2017 Scania and Ericsson, the iQPilotconductedademoofaremotedrivingapplicationwheretheAVtestbusshowninFigure1wasdrivenaroundScania’stesttrack.ThebuswascontrolledfromtheremotedrivingstationshowninFigure2.Ericssonhadestablisheda5GnetworknodeatScaniatoenablethedemo.Avideopresentingthedemo-applicationispubliclyavailable6.

6 https://www.youtube.com/watch?v=lPyzGTD5FtM

2013 2014 2015 2016 2017 2018 2019 2020 2021 2022

iQMatic iQPilot DriverSense AVTCT phase 1 AVTCT phase 2 ITRL's lab facilities REDO MERGEN


Figure8.Scania’sAutonomoustestbus.

Figure9.Remotedrivingstationusedtocontrolthetestbusduringthe2017demo.

The focus of the iQPilot demo was to demonstrate low-latency video streaming thatenabledremotedrivingandinthatregardwasaclearsuccess(Inametal.,2016).Giventhe5Gfocus,thedesignoftheHMIofthedrivingstationwasnotevaluatedintheproject.TheHMIconsistedofaverylowlatencyvideostream(centrescreen),displaysshowingcurrentandassignedspeed,gearselection,latencytimesandnetworkcoverage(lowerright), overviewviews from the commandand control system ICE (IntelligentControlEnvironment,aScaniaproduct)onthesidesandpedalsandwheelforcontrol.Contact:AndersStällbergScania,GeorgeDibbenScania,JimmySellingScania.DriverSenseWithin theDriverSenseprojectanexperimentalplatformbasedonagameenginehasbeendevelopedinordertodesignandvalidatedigitalonboarduserinterfacesforself-drivingandremotelycontrolledvehicles.AsetofselectedcasestudiesincludingHead-Up


Displays (HUDs), Augmented Reality (AR) and directional audio solutions have beenconducted, see Lungaro, Tollmar, Saeik & Mateu Gisbert (2018). Psychophysiologicalmeasurementsuchaseye-trackingwasusedduringtheanalysisofresults.TheprojectwasmanagedbyKonradTollmaratKTHMobile ServiceLab (MSL)withparticipationfromEricsson,Tobii, andUniversidadeFederaldeMinasGerais and ranfrom2017 to2018.MSL has also participated in 5GAA activities together with Audi, Ericsson, Italdesign,Pirelli,Qualcomm,TIMandTobii7inordertoinvestigatehow5Gconnectivityandvehicleto vehicle information systems can be used to improve safety by AR highlighting ofpedestrians, aquaplaningwarning between connected tires, as well as connected andadaptiveroadsigns.Contact:KonradTollmarKTHMobileServicesLab.AVTCTphase1andphase2The SAFER prestudy team interviewed leading researchers in the AVTCT project8(AutomatedVehicleTrafficControlTower),fundedbyDriveSweden.TheAVTCTphase1project was a prestudy, while phase 2 is running from 2019-04-01--2021-03-31.ParticipantsintheAVTCTprojectphase2aretheIntegratedTransportResearchLabatKTH(coordinator),MobileServicesLabatKTH,Carmenta,Ericsson,Scania,Volvo,andtheSwedishTransportAdministration.MoreinformationisavailableatITRL’swebsite9ThescopeoftheAVTCTprojectistoshedlightonquestionssuchas:

1. Thearchitectureofasystemwithautomatedvehiclesandoneorseveralcontroltowers,includingresponsibilitiesandinformationflows.

2. Investigating the requirements on cellular connectivity, computationalinfrastructure,andsensorsinthetraffictowerenvironment,inordertoguaranteetheperformanceoftheAVTCToperation

3. Increasingknowledgeabouthumaninteractionwiththesystematdifferentlevelsandfordifferenttasks,byutilizingthetestbedforexperimentsandsimulation.

4. Theroleoftrafficcontroltowerswithinthetransportationsystem.IntheanalysistheAVTCTprojectusesabroadscopeandconsidercontrolonstrategical,tacticalandoperationalcontrollevelsalongwiththeidentificationofrelevantusercentricusecases.Theprojectalsodiscusseshowindirectcontrol,e.g.throughtheuseofinfrastructure, canbeutilised.Hierarchiesof control towers, fromoperators toregulatoryagenciesareincludedintheanalysis.

5. A significant development effort concerning lab facilities from remote andautonomous driving is also conducted within ITRL, where the AVTCT projectcontributes.

Contact:JonasMårtenssonKTH,KonradTollmarKTH.BasOremusScania.

7 https://www.kth.se/aktuellt/nyheter/kth-gor-bilen-smartare-1.944051 8 https://www.drivesweden.net/en/projects-5/avtct 9 https://www.itrl.kth.se/research/ongoingprojects/automated-vehicle-traffic-control-tower-phase-2-1.917776.


ITRL’slabfacilitiesExamples of ITRL’s lab facilities investments relevant for the prestudy are the RCV-Evehicles,seeError!Referencesourcenotfound.,andtheCampus2030effort,whichencompassesaninstrumentedanddigitalizedcampus.ITRLisapplyingforapermittodrivetheRCV-EvehiclesremotelyandautonomouslyonKTHCampusandifsuccessfultheRCV-Evehicles couldbedrivenwithpermitover thewhole campusafter summer2020.MoreinformationisavailableatITRL’swebsite.10Contact:JonasMårtenssonKTH,MikaelNybacka,KTH

Figure10.RCV-EtestvehiclesatKTH/ITRL.

REDOTheREDOprojectisacurrentSwedishprojectfocusedonremotedrivingthatstartedinJanuary 2020 and ends in November 2022. Participating organisations are VTI, KTHVehicleDynamics/ITRL,Ericsson,Einride,Voysys,CEVT,NEVSandIctech.Anumberofdifferent aspects and foundational research questions of both technical and humanfactorsrelatednaturewillbeaddressedandstudiedexperimentallyintheREDOproject.The prestudy team conducted two interviews with leading researchers in the REDOproject,see5.1forREDOworkpackage2–Remotedriverstudiesduringteleoperateddriving and 5.2. for work package 3 – Remote driving feedback and control duringteleoperateddriving.TheREDOprojectalsoencompassesaworkpackage6concerningLawsandregulations,whichnotisdescribedfurtherinthecurrentreport,butwhichwillbeakeyfactorforremoteandautonomousdrivingintroduction.Contact:JanAnderssonVTIforWP2,MikaelNybackaKTHforWP3.

10 https://www.itrl.kth.se/research/ongoingprojects/research-concept-vehicle-model-e-1.917925


MERGENTheMERGENprojectPhDprojectatKTH– IntegratedTransportLab (ITRL)byRobinPalmberg that started in the fall of 2019. His thesis work investigates the drivermonitoringaspectsofremotedriving,i.e.canheart,brainandeyerelatedmeasuressuchasheartratevariability,EEGandpupildilationbeusedtoassessthecognitivestateoftheremote driver. MERGEN stands forMulti-purpose biometric Evaluation Research toolGrounded in Emerging Network technologies. More information is available at ITRLswebsite11.Contact:RobinPalmberg,KTH

7.4 RelevantEuropeanprojectsForautomated roadvehicles, very few initiativesaddressinghuman factors in remoteoperationsareidentifiedonEuropeanlevel(andinternationallyingeneral).TheongoingEU-project SHOW(whereRISE coordinates theMega/Twin site Swedenand leads theDashboard WP) utilizes the Ericsson Connected Traffic Tower solution for remoteassessment (dashboard) and for sending tactical missions to vehicles, mainly shuttlebuses.ThefocusofSHOWismainlyondemonstrationratherthanin-depthhumanfactorsdesignandevaluation.ThereareafewotherEU-projectsrelatedtotechnicalaspectsofremotecontrol,e.g.,TransAID,MAVEN,5GNetMobile.Inaddition,afewstudiesrelatedtohumanfactorshavebeencarriedoutbyresearchersatTechnischeUniversitätMünchen.Examples of topics explored there include head-mounted display (HMD) and hapticfeedback for improved telepresence, connection and latency concerns, and predictionmethodsforpathplanningaswellasinteractivepathplanning.Thereare,however,severalexamplesofrelevantprojectsfromotherdomains.Inorderto improve safety and traffic regulation, many European countries are in process ofdeveloping remote tower management. The idea is that the person monitoring andcontrollingthetrafficdoesnothavetobelocatedinthetoweritself,butcanoperatefromadistantsite.Thisnewsystemissupposedtoallowmonitoringthetrafficinsmallairportsthankstocameras,radarscreensandradiotransmission.Theoperatorwouldcontroltheairportbylookingatscreenswherethesituationisdisplayedinasimilarwayaswhats/hecouldsee"outofthewindow"inatowerontheairport.IntheEU-projectMOTO12(theembodiedreMOteTOwer),theobjectiveistodeveloptechnologiestoenhancethecurrent Remote Tower concept, by integratingmultimodal human-system interaction.Theoutcomesoftheprojectincludescenariosofembodiedcognitionintoweroperations,in order to understand and explore the role of the embodied cognition during high-performing operations of remote control, user requirements for multimodal remotetowersandaugmentingmannedairporttoweroperations,andmultimodalsolutionsableto reduce workload. In particular, the project explored Brain-Computer Interfaceapplicationasameansforaddressingworkload.

11 https://www.itrl.kth.se/research/ongoingprojects/mergen-multi-purpose-biometric-evaluation-research-tool-grounded-in-emerging-network-technologies-1.946255 12 http://www.moto-project.eu


InacollaborationbetweenSwedishandGermanresearchers,complexityfactorsrelatedtoremotetowersatairportswereidentified.Thefocuswasonworkloadmanagement.TheworkwasconductedwithintheCAPMOD-project13andisbasedonasimulation.Onemainfactor istheavailabilityofrelevant information.Withintheswitchingconditions,emergencies at one airport reduced handling qualitieswhichwas not the case in theconditionwherebothairportswerevisible to thecontroller.Furthermore, theratioofsituations with critical handling qualities was increased. Complexity is influenced byunforeseen events or traffic with unforeseen behavior. In many countries, e.g., inGermany,VisualFlightRules(VFR)trafficdoesnotrequireaflightplan,hence,VFRtrafficconstitutes unforeseen events for the air traffic controller’s preplanned actions. Oneimportanttakeawayfromthestudyisthatthereisnotasinglefactor,buttheinterplayofeventsatbothairports,thatdrivesthecomplexity.InGermanprojectVICTOR14(VirtualControlTowerResearch)from2010thefocuswasalso on remote air traffic control. The goal was to develop an air traffic controllerworkstationforatowerfacilitywhichisusedtoremotelymonitorandmanageoneormore regional airports. The project derived requirements from a human factorsperspective and developed a workstation based on these, again highlighting thecomplexityoftheremoteoperationandinter-dependencybetweenseveralfactors.Thenewly startedEUprojectSAFEMODE15projectaims todesignandvalidatea risk-informed framework to support Human Factors analysis in design and operations(HumanRisk-InformedDesign,HURID).Casestudieswilladdressknownandemergingrisks,suchasincreasedlevelsofautomationandremoteoperations(unmannedshipsanddrones).Assuch,thefocusisbothonmaritimeandaviation.Asforthetimebeing,theprojecthasnopublicresults.

7.5 InterviewsandworkshopsInterviewREDOproject:WP2Jan Andersson at VTI ismanaging awork package focusing on remote driver studiesduring teleoperated driving in theREDOproject. The prestudy team interviewed him2020-03-02.Theprimarycontentofthisworkpackageconcernsexperimentalstudiesoffundamentalhumanfactorsaspectsofremotedriving.Withintheworkpackageaseriesof experimentswill be conducted in VTI’s Simulator III. The independent variables oftheseexperimentswillbe:

• Latency.• Information presentation (placement of information, field of view and

perspective)• Typeofdrivingtask/trafficconditions.Thetaskdrivingtaskwillbevariedbothin

termsofquantityandcomplexity.

13 https://www.sesarju.eu/sites/default/files/documents/sid/2018/papers/SIDs_2018_paper_33.pdf 14 https://human-factors-consult.de/en/projects/aviation/victor/ 15 http://safemodeproject.eu


TheprimarydependentmeasuresthatwillbeusedintheexperimentsarederivedfromtheTrafficConflictTechniquedevelopedatLundUniversity,seeforexampleLaureshyn&Várhelyi(2018).InterviewREDOproject:WP3Mikael Nybacka at KTH Vehicle Dynamics/ITRL ismanaging the REDOwork packageconcerningremotedrivingfeedbackandcontrolduringteleoperateddriving,focusingontechnical requirements andpossibilities associatedwith remotedriving.Theprestudyteam interviewed him 2020-02-19. In a wider sense the work package addressesquestions regarding the design of the ARDAS (Advanced Remote Driving AssistanceSystems)functionsofthefuture.Oneofthehumanfactorsresearchquestionsoftheworkpackageregardstheuseofmotionfeedbackfromthevehicle'smovementtothedriver,especially since future vehicles can be over-actuated in order to give them newcapabilities.Theworkinthisworkpackagewillconsistofresearchandexperimentson:

• Remotedrivingfeedback:o Studywhat vehicle behaviour and events that needs to be fed back and

perceivedbytheremotedriver.o Designing and analysing driver support models to support a better

performanceofremotedrivingprecision.o Study what level and type of feedback is needed, with regards to

kinaestheticorvestibularsenses.• Remotedrivingvehiclecontrol:

o Developmethodstoattenuatebaddrivingcommands,includingpacketlossorwrongandunexpectedinput.

o Developmethods to be able to drive the vehicle during fault operation,whichwillinterfacetoremotedrivingfeedbackanddriversupportmodels.

InterviewVoysysTheSAFERprestudyteamvisitedVoysysABinNorrköping2020-03-02andwereabletotryoutVoysysremotedrivingsolution,bothwiththelargervisualdomesystemandtheirsmallerdrivingstation.Inthedemoapplicationascale1:8remotecontrolcarwasdrivenaroundatesttruck,butVoysyssystemisoperationalatanumberofcustomerswithfullsizevehicles.TheirVRsolutionswasnottestedduringthevisit.Voysysprimarymission is to supply technical solutions for teleoperation thathas thenetworkcoverage,bandwidthandlatencythatisrequiredforsafeandeffectiveremotedriving.TheuniquesellingpointsofVoysyssolutionistheiralgorithmscomparinglatencytimesfromseveralconcurrentnetworkcarriers,securingastableconnection.Thisfightslatencypeaksandprovidesthe lowest4G/LTElatencyinanymoment.Theirsolutionsincludeadaptivevideopackagingandeasilycanaccommodatecamerasalreadylocatedonthevehicle,regardlessoftheirpositioning,whereassomeothersuppliers’solutionsonlyaccommodateonecamerawithaspecificplacement.


Voysysstatement is that4Gconnectivitywillbesatisfactory,at leastwith theiruseofseveral carriers and constant comparison and seamless switching between differentcarriersinordertogetalowandstablelatency.Voysyshasachievedlatencytimesdownto60millisecond“glasstoglass”overthe4Gnetworkwiththeirsolution.Voysyshasalsodevelopedanumbersupportingproductssuchasanetworksimulator,bandwidthandnetworklatencymeasurementtools,andacybersecurityprotocol.Throughtheiruseofapurpose built, latency optimised 3D engine, augmented reality (AR) features arestraightforwardtoimplementintheirsystem.Duringthevisit,theprestudyteamnoticedtheimportanceofvisualreferencestotheownvehicle,e.g.thewheels inFigure6, inordertoconductprecisionmanoeuvring.Voysysmentionedthattheircustomers,usingVoysys’openSDKhadimplementedanumberofAugmented Reality features that support the manoeuvring, e.g. virtual future wheeltracks, braking distance visualisation, and predictive displays of where the vehicleactuallyisorwillbelocated.

Figure11.Voysysdomeequippeddriverstation.


Figure12.Voysyssmallerdriverstation.

Workshop1TheprestudyteamorganisedahalfdayworkshopattheIntegratedTransportResearchLab (ITRL) atKTH2020-02-18. SevenPhDs, onePhD student andone seniorproductdevelopmentmanager,allwithexperienceofautonomousvehicleR&DfromKTH,ScaniaandRISEparticipated.During theworkshoptheparticipantsshared theirremotedrivingrelatedexperiencesfromanumberofprojects,e.g.AVTCT,Mergen,andDriverSense.Attheendoftheworkshopatotalof50researchquestions(pre-generatedandduring-the-workshopgenerated)wereprioritizedby theparticipants throughasimplevotingprocess,conductedafteraquickreviewofallthequestions.Theintentfromtheprestudyteam was to conduct an initial assessment of interest in different types of researchquestions.Thecriteriafortheparticipantsvoteforaspecificresearchquestionwasthat,giventhattheyhadaverylimitednumberofvotes,thattheypersonallywouldliketobeinvolved in a project studying that specific research question. The results from thisresearchquestionprioritisationarepresentedinsection8.2.Workshop2Anotherworkshopwithasimilarformatastheonedescribedabovewasorganisedbytheprestudy team to be held at RISE in Gothenburg 2020-04-01. Due to the Covid-2019situationtheworkshopwastransformedfromaphysicalmeetingintoateleconferencingformatwithintroduction,freediscussionandasetofvotesconcerningresearchquestionsadministered through the Mentimeter tool. 12 experienced researchers and productdevelopers participated in the workshop. One from academia, four from researchinstitutes, fivefromindustryandtwofromaregulatoryagency.Sixhadhumanfactorsrelated PhDs and the other six were senior product designers or research program


managers. Four of the participants were from the prestudy team and thus alsoparticipatedinworkshop1.Basedon theresearchquestions thatwereused inworkshop1,acuratedsetof theseresearchquestions draftwhere usedduringworkshop2. Between theworkshops theresearchquestiondraftsweredevelopedfurther, i.e.,rephrased,combined,somewereremoved, and new aspectswere added based on the literature review and continueddiscussionintheprestudyteam.Asetof21draftresearchquestionswerethenusedasdiscussionstimuliduringthesecondworkshop.Theresults fromtheMentimetervotesconcerningresearchquestionprioritisationarepresentedinsummarisedforminsection8.3.


8 ResearchgapsandchallengesInthissection,theinformationcollectedbytheprestudyteamregardingresearchgapsandresearchquestionsforfutureprojectsispresented.

8.1 LiteraturereviewresearchquestionsThe general view of relevant research questions from the literature review can besummarizedbyhighlightingphenomenasuchaspsychologicaldistancing,screendelays,network latency delays, inefficient interface designs, and human operator’s cognitivelimitations.

8.2 Workshop1researchquestionsToconcludetheresultsfromworkshop1,Table1presentstheresearchquestiondraftsthatreceivedthemostamountofvotesfromboththefirstandsecondvote.Inthefirstvoting50 researchquestionswereavailable, andall that received less than twovoteswereremoved.Inthesecondvote12researchquestionswereavailable.Table1.Researchquestionswithmostvotesduringworkshop1.

Draftresearchquestion 2ndvote 1stvoteWhatarethedifferencesbetweendifferentremotecontrolconceptsandcontrollevels,onoperative,tacticalandstrategicallevels,e.g.wheel,waypoints,start-destination,andwhenaretheysuitable?

4 4

HowdowedefineODD's(OperationalDesignDomains)forremotedrivingandremotecontrol?HowdowemethodologicallydescribethepropertiesandprerequisitesfordifferentODD’s,astheseODDdescriptionsformthefoundationforHMIdesigndecisions,e.g.whichinformationelementsareimportantwhen?

4 3

WhatarethemaximumnumberofvehiclesthatarecontrollablegivendifferentODDs,context,andtypesofdeviations?Canaremotedriversimultaneouslyberesponsibleformorethanonevehicle?

4 3

Whichtaskswillaremotedriveractuallymanage,givendifferentsolutionconcepts.Whichscenariosshouldwebasethedesignon?

2 3

Arethereeffectsofpsychologicaldetachmentwhenthedriverisnotphysicallypresentinthevehicle,e.g.itfeelslikeagame,andhowcanthisbeavoided?

1 3

Willskilldecayaffectdriverwhoonlyusetheremotedrivingcapabilitysometimesasdifferenttypesofskills(e.g.perceptual,psychomotor,procedural,cognitive)typicallyshowdifferentdecayeffects?

1 3

Whichinformationandfunctionsdoesanoperatorneedinordertoswiftlyinterveneandtakeoveravehiclewhenneeded?Forexample,regardingvisibility,fieldofview,temporaldevelopmentoftrafficsituation/history,classificationofotherroadusers,identificationofrootcauseforalerttooperator.

4 2

IsitpossibletodesignanHMIthatenablescontrolonanoperationallevelwhilestillprovidingrequiredpolicyawarenessforageographicalortemporalzone?

2 2

Taskanalysisoftheinformationthatcurrentdriversofclassicalvehiclesuseduringdriving.

2 2

CanweidentifyinformationsecuritychallengesthatarespecificforremotedrivingandwhichhasdirectHMIimplications?DoesremotedrivingcapabilityenhanceordecreaseAVmarketintroductionfromaninformationsecurityaspect?

0 2

Whichaspectsoflatency(Qualityofservice,bandwidth,packetloss,latencyvarianceetc)arethemostrelevanttoaddressforremotedriving?

0 2

ShouldtheHMIpresentrawdataorasensor-fusionedandinterpretedfeed?Whichimagetakesthelongesttimetointerpret?Isstreamingvideofromthevehicleneededorcantheoperatorcontrolthevehicleonanonboardfusionedandinterpretedimage?

0 2


Lookingforclusters,thetopfourrowsindicateaneedformorestructuredandmaturedescriptionsoftheusecasesandoperationaldesigndomains,aswellasmethodsforthedefinition of them. Understanding of the operational domain and the requirementsderived from the tasks that operators perform form the basis forHMI design, so thiscluster innotverysurprising.Therearealsoseveralquestions relating toHMIdesignamongthese“top12”questions.

8.3 Workshop2researchquestionsAt the beginning of the Mentimeter session of workshop 2, the participants had theopportunity to bring forward one or two research questions that they consideredimportant, before they saw the researchquestionsdefinedby theprestudy team.Theresponsesfromtheparticipantswere:

• Theinteractionbetweenroles/responsibilitysplit.• Training.• Whatprecision is required for control on anoperational level fordifferentuse

casesandwhy?• Whatisthemostimportantdefiningconditionforenablingremotedriveornot:

infrastructure/ODD,HMI,users’educationorsomethingelse?• How do we bring forward a systems perspective on remote control (Man-

Technology-Organisation),andwidenthescope?• Howandwhenshouldhand-overbedone?• Howgivetheremotedriverenoughinformationwithinformationoverload?• Howtoscaleup?• Howtobuildtrustwithrobotsinthecitytransportingpeople?• "Thehotpotato"andoutofloop...forsystems/operationsthatisnot"safeenough"

tobefullytrustedinautomated"modus".• Functionalitysupportingtheoperators’needtoswitchbetweencontrollevelsand

situations.• Howcanremotecontrolcontributetonewmobilityservicesandsolutions?• Security.• Teamplayerapproach:whentohandover?Howtobuildtrust?Certification?• Whatarethemethodologicalneedsintermsofresearchinfrastructure?• Systemssafetyofremotedriving.• Responsibilityandlegislations.• WhatwouldberequiredfromHMI,infrastructureandusereducationinorderto

safely allow for emergency evacuation of AV:s who have stopped in a publichighwaycorridor?

• The hitchhiker perspective, i.e. non-authorized persons hitchhikingwith futureautonomousvehicles.

• With the exponential technology development pace in mind, how do weunderstandthetechnologicalphaseandhowdowedesignsolutionsthatcanworkovertime.


The participants were then asked four general questions concerning their opinionsconcerningremotecontrolwithresultspresentedinFigure13,Figure14,Figure15andFigure16.

Figure13.Answertothequestion“WhichofthesecontrollevelsrequirethemostR&Dattentionthecomingfiveyears?”.

Figure14.Answertothequestion“WhichoftheseoperationalenvironmentsrequirethemostR&Dattentionthecoming

fiveyears?”.


Figure15.Answertothequestion“Doyouconsiderremotecontrolonanoperationallevel,i.e.remotedriving,tobean

importantcapabilitywithin5years?”

Figure16.Answertothequestion“Doyouconsiderremotecontrolonanoperationallevel,i.e.remotedriving,tobean

importantcapabilitywithin10years?”.

Theprestudyteamthenpresented21curatedresearchquestions,whichcouldbeusefulforscopingofnewresearchprojectproposals.Basedontheresultsofworkshop1,theliteraturereviewanddiscussionswithintheprestudyteam,21researchquestionshadbeenselected.Theworkshopparticipantsthenforeachresearchquestionwereaskedto


estimate and vote concerning the research questions suitability as a project researchquestion.Answeralternativeswere0= lowerand1=higher.TheyalsoestimatedandvoteconcerningthetimeframewithinwhichaprojectorR&Dresultswasneeded.Answeralternativeswere0=shortertimeframe,within5yearsand1=longertimeframe,within10years.The21researchquestionswerepresentedinthreesections,withsectionsrelatingtotheoperationaldomain,theoperatorandthehuman-machineinterface,withsevenresearchquestions in each section. Figure 17 below show all the ratings in one grid, afterpostprocessingofresults.DuetotechnicallimitationsinMentimeteronly20votescanbedone on the same slide, so for the Vigilance and driver in the loop question, thevisualisation,i.e.dot21hasbeenenteredmanuallyintothefigure.Resultsarepresentedinvisualformwithonlyshortdescriptions.

Figure17.Resultofvotesforall21researchquestionsmergedintoonegrid.

Notethatgreatcareshouldbeappliedwheninterpretingthesetypesofresults.Theintentwasprimarily to initiate discussionduring theworkshop and to get somevery roughindicationconcerningthemeritandurgencyofthepresentedresearchquestions.Buttheoverarchingobservationisthatalmostallresearchquestionsclusterinthelower,rightquadrant,whichindicatesthattheparticipantsconsiderthemvalidresearchquestionsthatcouldbeusedtoscopefutureprojects.Theyallwereconsideredurgentandshouldbeaddressedwithinthetimeframeoffiveyears.Given the curated nature of the of the research questions thatwere presented to theparticipants this clustering might not be very surprising. However, the ordering ofresearch questions based on the responses might provide some recommendationconcerning the prioritisation of research questions, see Table 2 below, although careshouldbetakenwheninterpretingtheseresults.


As an example, for the research question ranked in the top of Table 2, Handoverprocedures/Safestopsneeded,therewasfullconsensusthatitwasasuitableresearchquestionforaprojectandthatresultswereneededwithinashorttimeframe,i.e.withinfiveyears.Almostalloftheresearchquestionsreceiveda“neededwithinashorttimeframe”rating.Thereissomevariance,buthardlyusefulasanindicationofwhichresearchquestionstoprioritise. For the Suitable as research project there is more variance among theresponsesfromtheparticipants.For some questions one to three participants chose to skip answers because theyconsideredthemselvesnotbeingabletoassess it.Themeanpercentvalues inTable2represent the percent of voters that actually did vote for that specific question. Thequotient between votes and voters are therefore also presented in Table 2 for fulltransparency.


Table2.Researchquestiondiscussedduringworkshop2andvotingresults.

Researchquestion Suitabilityasresearchproject.Meansfromvotingparticipantsbelow,with1,00indicatingfullconsensusonsuitabilityasresearchproject

Votesof1,i.e.suitableasresearchprojectquestion/numberofvoters

Neededintimeframe.Meansfromvotingparticipantswith0,00indicationfullconsensusthatprojectisneededwithinfiveyears

Votesof0,i.e.projectsnecessarywithin5years/numberofvoters

Handoverprocedures/Safestopsneeded?

1,00 12/12 0,00 12/12

Differencesbetweenoperational,tactical,strategicalcontrol.Usecasesandmaximumnumberofvehiclescontrollable

1,00 12/12 0,18 2/11

Perceptualaspectsandfeedbackdesign(visual,auditive,haptic),e.g.FOV

0,92 11/12 0,00 0/11

Trust,responsibility,automationsurprises,human-centricautomation

0,92 11/12 0,08 1/12

Informationneedsforswiftintervention

0,91 10/11 0,00 0/11

ComparisonMD,RD,AD/Taskanalysisforremotedriving

0,90 9/10 0,10 1/10

Sensorandviewcontrolneeds

0,83 10/12 0,00 0/11

Informationsecuritychallenges,specificallyHMIimplications

0,83 10/12 0,08 1/12

Vigilanceanddriverintheloop

0,83 10/12 0,18 2/12

ODDdefinition 0,82 9/11 0,09 1/11Controlroomrequirements

0,80 8/10 0,00 0/11

Psychologicaldetachment

0,75 9/12 0,08 1/12

Precisionmanoeuvringrequirements

0,75 9/12 0,08 1/12

AugmentedRealitydesign

0,75 9/12 0,17 2/11

Trafficregulationsinmixedtraffic

0,73 8/11 0,18 2/11

Driver’slicense/certificationforremotedrivers

0,70 7/10 0,20 2/10

Presentrawdatavsinterpreteddata

0,70 7/10 0,20 2/10

Suppressorenhanceinfoduringdeviations

0,64 7/11 0,08 1/11

Latencyeffects/5Gor4Gneeded

0,55 6/11 0,09 1/11

HMIforlatencyeffectreduction

0,55 6/11 0,17 2/11

On-sitesupportfunctions

0,45 5/11 0,09 1/11


Attheendoftheworkshoptheparticipantswereaskedtodescribetheirmaintake-awayfrom the workshop. Their entries are presented below (a few positive commentsconcerningtheworkshopformathavebeenremoved):

• FurtherR&Disneeded!• Howdowecreatethefirststeptowardsalignmentofsolutionsforremote

controltotakeonsafetyandefficiencychallengesintraffic?• Donotlookseparatelyondifferentcontrollevels.• Similaroreventhesamechallengesinall"vehicle"domains.Sweden,could

benefitfromanational"range"ofautomation-dilemmaresearch,sineallindividualprojectseemstoosmalltorelygettothecoreoftheproblems...

• Betterunderstandingofremotecontrolandrelatedresearchchallenges.• Knowledgeandexperiencesfromrelatedcontrolroom"industries"shouldbe

furtherinvestigated.• Thereisaneedtolookatthisfromdifferentperspectives.Butthereisalsoalot

tolearnfromdifferentdomains.Weshouldprobablyaddressitfrommultipleangles.

• Howisthisaffectingworkopportunitiesintheindustry?• Agoodoverviewofresearchinterestsinremotecontrolandremotedriving

based.• Thereisaneedforopenarchitecturesupportingremotedriving.

Throughthetwoworkshops,eventhoughthevotinguseddifferentstimuliandprocedure,ninePhDsandeightseniorproductsdevelopersorresearchprogrammanagershavebeengiventheopportunitytoprioritisewithinthecuratedsetofdraftresearchquestions.Toconcludetheanalysisofresultsfromworkshop2,itprimarilybecameavalidationofthe research questions that had been defined before, along with some prioritizationinformation,eventhoughallthepresentedquestionswereconsideredquiteurgent.


9 Conclusions,LessonsLearntandNextStepsInthissection,anumberofconclusionsmadebytheprestudyteamispresented.

• Onagenerallevel,thequestionwhetheranadequate,oroptimal,remotedrivingstationisareplicaofthedriverenvironmentofatruckcaboftodayor“somethingelse”needstobeanswered.Itmightwellbethataremotedriverstationandit’sHMIcontainmoreotherandotherfunctionalitythatwhatadriveroftodayhasavailable,indicatingthatthequestionneedsR&Dattention.

• Alotofresearchandinnovationhasbeendoneregardingremoteoperation,bothinautomotiveandinotherdomains.However,ourratherextensivereviewshowsthere is a lack of understanding of design and human factors requirements.Consequently, there is a risk that HMI solutions developed are not properlydesigned.Indeed,ourimpressionisthatthesolutionsbeensuggestedaremainlydriven by technology developments, rather than user needs, preferences andexperiences.Thereare,ofcourse,examplesofstudieswheresuchaspectshavebeen considered, however, amore systematic and holistic approach is needed,especially to achieve large scale implementation of automated driving systemswiththesupportofremoteoperation.

• Thedesignspaceisquitelargegiventhemanydifferentconcepts,scenariosandODDswhereremotedrivingcanbeausefulcapability.Figure5describesremotecontroldependenciesthatinFigure18havebeentransformedintoamorelinearrecommendation on how to approach a project. We suggest starting withdefinitionoftheODDwherethevehicleswilloperate.Therequirementsemergingfrom the static anddynamic environmentwill guidewhenandwheredifferentcontrolmodesaresuitable.Further,therewillbeaneedtospecifyODDrelatedtasks(predictingandreactingtochangesinthedrivingenvironment)aswellascontrol mode related tasks (operational driving, tactical guidance or strategicmonitoringandtheshiftbetweenthesemodes).Whentaskshavebeendescribedtheoperators’rolesandresponsibilitiescanbedefined.Finally,theremotedrivingHMIcanbedesignedinaccordancewiththeoperator’sneedsinthecurrentworksystem.


Figure18.Arecommendationonhowtoapproachprojectsonremoteoperation(basedonthedimensionsand

interdependenciespresentedinSection6)

• This raises the need to point out that best practice forHMIdevelopment is aniterative approach with user needs analysis, concept generation, design andtesting in several cycles. Given the complexity of interactions between humancognition,systemdesign,andthetrafficenvironmentitisquiteraretobeabletodesign a safe and efficient HMI without these iterations. The design of R&Dprojects, with regards to expected results, timelines and funding mustaccommodateforthiscomplexity.

• Thedifferentcontrollevelsshouldnotbeobserved/designedforinisolation,allcontrollevelsarevalidandapplicableinmostapplications.Effortshouldbespenton HMI and system functionality that supports the operator when switchingbetweencontrol levelsandtask.AgreaterchallengeforvehicleOEM’s(OriginalEquipmentManufacturer)thanthedesignofamarketableremotedrivingsolutionoraremotecontrolsystemworkingonamorestrategicalcommandlevel,willbeto develop a vehicle and control system architecture that enables ”seamless”transition(orwithas littlecognitive loadon theoperatoraspossible)betweendifferentmodesofcontrol.Thesystem,withitsHMI,needstomakeitveryexplicitwhat different work tasks for the human operators means and who has theresponsibility.Explanationofcontextanditseffectondecisionsbytheautomation,boundaryconditionsandoverviewofmodestatesandtransitionsbetweenmodesalsorequirecarefuldesignoftheHMI.Experiencesfromotherdomainsshowsthatthese are features that often is lost during the automationof aprocesses,withoperatorsbeingoutsideofthecontrolloopasaresult.Thesafetyimplicationsofthiscannotbeignoredandsafetyassumptionsmustalsobeanalysedcarefully.Asan example, consider thatwhen a system error happens itmaywell be that it


affectsallthevehiclesinafleetatthesametime.Theassumptionthataremoteoperatorcanmonitorafleetofvehiclesandtakecontrolofthemoneatthetimetohelpthemthenfalls.Theoperatorthenbecomesforcedtogofromasupervisoryroleofmanyvehiclesintotheverychallengingroleofmanualcontrollerforeachvehicleinthefleet.

• Despite that fact that there are quite a few commercial actors aswell as R&Dprojects active in the field, there are many human factors related researchquestionsthatrequirefurtherattention.Theprestudyreportpresentsaquitewiderangeofdifferenttypesofresearchquestionsdraftsthatarerelevanttoremotecontrolandremotedriving.Theseresearchquestionshavebeendefined,reviewedandrefinedbyanumberofspecialistsandshouldprovideagoodstartforanyoneinterested in definition of a human factors related remote control or remotedrivingR&Dproject.Thislistofresearchquestionsfulfilstheprimarypurposeoftheprestudy.Beforesystemsbecomeoperational inrealtrafficonawidescale,many of these research questions will need to be addressed to some extent.Returning to the figure theoretical framework andmodel for remoteoperation(seeSection5and6),the21researchquestionshavebeenapproximatelymappedto the dimensions in Figure 19, indicating both that the model is useful fororganisationofresearchquestionsinthisdomainandthattheidentifiedresearchquestionsaddressdifferentdimensionsofremotedriving.

Figure19.Researchquestionsmappedtotheidentifieddimensionsofremoteoperation.



10 DisseminationandPublicationsIn addition to this report, a paper draft is under preparation and is aimed to besubmittedtoaconference.TheprojectresultshavealsobeenpresentedinanonlineseminartoSAFERpartners.Also,SAFERpartnershavetakenpartoftheresults(andcontributedtoresults)byparticipatinginourworkshopsandinterviews.TheresultshavealsobeenpresentedataninternalonlineseminaratScania.Howhavetheresultsbeen spreadorwill be spread?Theproject resultswill serve as abasis for anFFIprojectonthistopic.

11 AcknowledgementTheprestudyhasbeencarriedoutwithfinancialsupportfromSAFERandScania.Theauthorsarealsogratefultoallparticipantsintheworkshopsandinterviews.

12 ReferencesAlmeida,L.,Patrao,B.,Menezes,P.,&Dias,J.(2014).Betherobot:Human

embodimentintele-operationdrivingtasks.The23rdIEEEInternationalSymposiumonRobotandHumanInteractiveCommunication.doi:10.1109/roman.2014.6926298

Bhat,R.,Pandey,V.,Rao,A.K.,&Chandra,S.(2017).Anevaluationofcognitiveandneuralcorrelatesforindirectvisiondrivingandroverteleoperation.20172ndInternationalConferenceonManandMachineInterfacing(MAMI).doi:10.1109/mami.2017.8307885

Bidwell,J.,Holloway,A.,&Davidoff,S.(2014).Measuringoperatoranticipatoryinputsinresponsetotime-delayforteleoperatedhuman-robotinterfaces.Proceedingsofthe32ndAnnualACMConferenceonHumanFactorsinComputingSystems-CHI14.doi:10.1145/2556288.2557174

Bout,M.,Brenden,A.P.,Klingegård,M.,Habibovic,A.,&Böckle,M.-P.(2017).AHead-MountedDisplaytoSupportTeleoperationsofSharedAutomatedVehicles.Proceedingsofthe9thInternationalConferenceonAutomotiveUserInterfacesandInteractiveVehicularApplicationsAdjunct-AutomotiveUI17.doi:10.1145/3131726.3131758

Brannigan,J.C.,Veeningen,D.M.,Williamson,M.,&Gang,Z.(2008).HumanFactorPrinciplesinRemoteOperationCenters.IntelligentEnergyConferenceandExhibition.doi:10.2118/112219-ms

Cabrall,C.,Stapel,J.,Besemer,P.,Jongbloed,K.,Knipscheer,M.,Lottman,B.,…Rutten,N.(2019).PlausibilityofHumanRemoteDriving:Human-CenteredExperimentsfromthePointofViewofTeledriversandTelepassengers.ProceedingsoftheHumanFactorsandErgonomicsSocietyAnnualMeeting,63(1),2018–2023.doi:10.1177/1071181319631006

Chen,J.Y.C.,HaasE.C.,&Barnes,M.J.(2007).Humanperformanceissuesanduserinterfacedesignforteleoperatedrobots.IEEETransactionsonSystems,Man,and


Cybernetics—PartC:ApplicationsandReviews.2007;37:1231–1245.doi:10.1109/TSMCC.2007.905819

Chen,J.Y.C.,Barnes,M.J.,&Harper-Sciarini,M.(2011).SupervisoryControlofMultipleRobots:HumanPerformanceIssuesandUserInterfaceDesign.IEEETransactionsonSystem,Man,andCybernetics—PartC:ApplicationsandReviews,41(4),435–454.

Chen,J.Y.C.,Barnes,M.J.,&Harper-Sciarini,M.(2010).SupervisoryControlofUnmannedVehicles(ARLReportARL-TR-5136).AberdeenProvingGround,VA:USArmyResearchLab.

Cherpillod,A.,Floreano,D.,&Mintchev,S.(2019).EmbodiedFlightwithaDrone.2019ThirdIEEEInternationalConferenceonRoboticComputing(IRC).doi:10.1109/irc.2019.00070

Chucholowski,F.E.,Sauer,M.,&Lienkamp,M.(2016).EvaluationofDisplayMethodsforTeleoperationofRoadVehicles.JournalofUnmannedSystemTechnology,3(3),80–85.doi:10.21535/just.v3i3.38

Cummings,M.L.,&Guerlain,S.(2007).Developingoperatorcapacityestimatesforsupervisorycontrolofautonomousvehicles.HumanFactors,49(1),1-15.doi:10.1518/001872007779598109

Davis,J.,Smyth,C.,&Mcdowell,K.(2010).TheEffectsofTimeLagonDrivingPerformanceandaPossibleMitigation.IEEETransactionsonRobotics,26(3),590–593.doi:10.1109/tro.2010.2046695

DoD(2012).TheRoleofAutonomyinDoDSystems.USDepartmentofDefense.Availableat:https://fas.org/irp/agency/dod/dsb/autonomy.pdf

EuropeanRailwaysAgency(2016).CommonSafetyMethods.Availableat:https://www.era.europa.eu/activities/common-safety-methods_en

Galster,S.M.,Knott,B.A.,&Brown,R.D.(2006).ManagingmultipleUAVs:areweaskingtherighquestions.ProceedingsofHumanFactorsandErgonomicsSociety60thAnnualMeeting,SanFrancisco,CA,pp.545-549,October16-20,2006.

Gatsoulis,Y.,Virk,G.S.,&Dehghani-Sanij,A.A.(2010).OntheMeasurementofSituationAwarenessforEffectiveHuman-RobotInteractioninTeleoperatedSystems.JournalofCognitiveEngineeringandDecisionMaking,4(1),69–98.doi:10.1518/155534310x495591

Haans,A.,&Ijsselsteijn,W.A.(2012).Embodimentandtelepresence:Towardacomprehensivetheoreticalframework.InteractingwithComputers,24(4),211–218.doi:10.1016/j.intcom.2012.04.010

Haduch,T.W.,&Mitchell,B.T.(1995).AHumanFactorsApproachtoUnmannedGroundVehicleOperatorControlUnitInterfaceResearch.SAETechnicalPaperSeries.doi:10.4271/951482

Hedayati,H.,Walker,M.,&Szafir,D.(2018).ImprovingCollocatedRobotTeleoperationwithAugmentedReality.Proceedingsofthe2018ACM/IEEEInternationalConferenceonHuman-RobotInteraction-HRI18.doi:10.1145/3171221.3171251

Hollnagel,E.&Woods,D.D.(2005).Jointcognitivesystems:Foundationsofcognitivesystemsengineering.BocaRaton,FL,USA:Taylor&FrancisBooks,Inc.doi:10.1201/9781420038194


Hosseini,A.,&Lienkamp,M.(2016).EnhancingtelepresenceduringtheteleoperationofroadvehiclesusingHMD-basedmixedreality.2016IEEEIntelligentVehiclesSymposium(IV).doi:10.1109/ivs.2016.7535568

InternationalCivilAviationAuthority(2018).SafetyManagementManual4thed.Note:the3rdedfrom2013isfreelyavailableathttps://www.skybrary.aero/bookshelf/books/644.pdf.

Inam,R.,Schrammar,N.,Wang,K.,Karapantelakis,A.,Mokrushin,L.,Vulgarakis,F.,&Fersman,E.(2016).Feasibilityassessmenttorealisevehicleteleoperationusingcellularnetworks.Proceedingsof2016IEEE19thInternationalConferenceonIntelligentTransportationSystems(ITSC).

H.Karvonen,H.Koskinen,J.Haggrén,Enhancingtheuserexperienceofthecraneoperator:comparingworkdemandsintwooperationalsettings,InProceedingsofthe30thEuropeanConferenceonCognitiveErgonomics.ACM,2012,pp.37–44.

Laureshyn,A.,&Várhelyi,A.(2018).TheSwedishTrafficConflictTechnique-Observer’smanuallv.1.0(04/2018).LundUniversity.Availableat:http://www.tft.lth.se/english/research/traffic-safety-and-behaviour/the-swedish-conflict-technique

Liu,R.,Kwak,D.,Devarakonda,S.,Bekris,K.,&Iftode,L.(2017).InvestigatingRemoteDrivingovertheLTENetwork.Proceedingsofthe9thInternationalConferenceonAutomotiveUserInterfacesandInteractiveVehicularApplications-AutomotiveUI17.doi:10.1145/3122986.3123008

Lu,S.,Zhang,M.Y.,Ersal,T.,&Yang,X.J.(2019).WorkloadManagementinTeleoperationofUnmannedGroundVehicles:EffectsofaDelayCompensationAidonHumanOperators’WorkloadandTeleoperationPerformance.InternationalJournalofHuman–ComputerInteraction,35(19),1820–1830.doi:10.1080/10447318.2019.1574059

Lungaro,P.,Tollmar,K.,Saeik,F.,&MateuGisbert,C.(2018).DriverSense:Ahyper-realistictestbedforthedesignandevaluationofnoveluserinterfacesinself-drivingvehicles.AdjunctProceedings-10thInternationalACMConferenceonAutomotiveUserInterfacesandInteractiveVehicularApplications,AutomotiveUI2018,AssociationforComputingMachinery,Inc,2018,p.127-131.

Mahboob,Q.,&Zio,E.(eds)(2018).HandbookofRAMSinRailwaySystems:TheoryandPractice.CRCPress.doi:10.1201/b21983.

Neumeier,S.,Wintersberger,P.,Frison,A.-K.,Becher,A.,Facchi,C.,&Riener,A.(2019).Teleoperation:TheHolyGrailtoSolveProblemsofAutomatedDriving?Sure,butLatencyMatters.Proceedingsofthe11thInternationalConferenceonAutomotiveUserInterfacesandInteractiveVehicularApplications-AutomotiveUI19.doi:10.1145/3342197.3344534

Nie,H.,Lu,Y.,Wang,C.,Yang,G.,&Fu,S.(2019).TheRelationshipbetweenEmotionStateandHumanPerformanceintheTeleoperationTask*.2019IEEE15thInternationalConferenceonControlandAutomation(ICCA).doi:10.1109/icca.2019.8899948

Ringdahl,O.,Automationinforestry:developmentofunmannedforwarders.DepartmentofComputingScience,UmeåUniversity,2011.


USDepartmentofDefense(2013).UnmannedSystemsIntegratedRoadmap2013-2038.Availableat:https://digital.library.unt.edu/ark:/67531/metadc949794/m2/1/high_res_d/DOD-USRM-2013.pdf

USDepartmentofDefense(2017).UnmannedSystemsIntegratedRoadmap2017-2042.Availableat:https://apps.dtic.mil/dtic/tr/fulltext/u2/1059546.pdf

Vincente,K.(2003).TheHumanFactor:Revolutionizingthewaypeoplelivewithtechnology.A.A.Knopf,Toronto,Canada.

Wahlström,M.,Hakulinen,J.,Karvonen,H.,&Lindborg,I.(2015).HumanFactorsChallengesinUnmannedShipOperations–InsightsfromOtherDomains.ProcediaManufacturing,3,1038–1045.doi:10.1016/j.promfg.2015.07.167

Wintersberger,P.,Neumeier,S.,Frison,A.-K.,Becher,A.,Facchi,C.,&Riener,A.(2019).Teleoperation.Proceedingsofthe11thInternationalConferenceonAutomotiveUserInterfacesandInteractiveVehicularApplications-AutomotiveUI19.doi:10.1145/3342197.3344534

Wojtusch,J.,Taubert,D.,Graber,T.,&Nergaard,K.(2018).EvaluationofHumanFactorsforAssessingHuman-RobotInteractioninDelayedTeleoperation.2018IEEEInternationalConferenceonSystems,Man,andCybernetics(SMC).doi:10.1109/smc.2018.00641

Zunjic,A.(2015).PerformanceandWorkloadofOperatorsinaHuman-TelerebotSystem.InternationalJournalofBehavioralResearch&Psychology,01–03.doi:10.19070/2332-3000-150004e

coordinator project manager linda meiby … · 2020. 11. 20. · page 1 of 44 finalreport_safer...

Documents