S&AS:INT:AutonomousMulti-RobotVisualMonitoringforUrban,Agricultural,andNaturalResourceManagement

PI:A.Roy-ChowdhuryCo-PIs:K.Karydis,Q.Zhu,A.Mourikis,G.Jenerette

SeniorPersonnel:N.Abu-GhazalehUniversityofCalifornia,Riverside

Research Objective

Framework for multi-agent aerial imaging systems that can adapt autonomously toenvironmental conditions, technical constraints, and ethical considerations for dataanalysis, trajectory planning, and computation/communication resource management,with applications in monitoring vegetation health.

Overview:Thisworksynthesizesideasfromvisualsensing,autonomousrobotnavigation,decisionmaking andsystemoptimization todevelopageneralframeworkforadaptivemulti-agentvisualmonitoringappliedtonaturalresourcemanagement.

A. AdaptiveVisualAnalysis1. Adaptivealgorithmdesignandselection2. Resource-constrainedoptimaldataprocessing

B. AutonomousAerialRobotNavigation1. ReliableGPS-deniedposeestimationwithmulti-modalvisualinput

a. Useofmulti-modalvisualinputb. Useofsatellitemapsforbounded-uncertaintyvisuallocalization

2. Trajectoryplanningandcontrolunderconflictingconstraintsa. Aerodynamicsandenergeticsofroboticflightinrealisticoperationalconditionsb. Sensory-basedadaptiveplanningunderconflictingconstraints

C. System-levelDecisionMakingandAdaptation1. Sensingandnavigationco-adaptation2. Computationandcommunicationconstraintsanalysisandmitigation

Major Research Tasks

A. Online Video Summarization: Video Fast-Forwarding via Reinforcement Learning (CVPR 18)

Goal: Develop a method for fast-forwarding through a video that is computationallyefficient, causal, online and results in informative video segments

Approach:

• Formulate fast-forwarding as Markov Decision Process (MDP)- Solve using Q-learning• Online framework to deal with incremental observations without requiring to store and

process the entire video• Experiments on 2 standard summarization datasets – about 6%-20% improvement in

coverage with 80% reduction in number of frames processed

ReinforcementLearningFrameworkforFast-ForwardingVideos

Approach:

• Introducefaithfulmodelstocaptureandmitigatetheimpactofvariousconstraints,e.g.,energyconsumptionatvariousspeedsorincloseproximitytootherrobots.

• Introduceaccurateandreliablevisual-inertialodometrymethodstoclosefastaction-perceptionloopsforautonomousnavigation.

• Developnewmulti-robotdeploymentalgorithmsthatareawareofenergyconsumptiontooptimizerobotformationsandtaskallocation.

• Developresourcemanagementpoliciestotradeoffaccuracyandperformancetomeetcomputationalconstraintsandoperateundervariableconnectivity.

B/C. Autonomous Navigation under Energy, Computation and Communication Constraints

Goal: Develop and evaluate multi-robot motion planning and navigationalgorithms under energy, computation and communication constraints

Quadrotorflightisfoundmoreenergyefficientwhenflyingforwardat5-8m/s.

Evaluation: Imaging and Modeling Vegetation

Goal:Identifyandcharacterizevegetationcanopyforrapidevaluationofhealthandproductivity.

Approach:

• CombinefieldsurveyswithUAV-basedimageryinvisibleandthermalwavelengths• Rapidlyidentifyspeciesofdominantplants,andtheirhealth• Usestructure-through-motionapproachtoreconstruct3-dimensionalgeometryofvegetation

Initialcollectionofhighresolutionimageryfromacommunitywithtwodominantplantspecies.

Exampleof3-Dreconstructionoffieldsiteusingthestructure-throughmotionapproach.