UAS Research in Support of

ODOT Operations

Research Team• Profs: Arthur Helmicki, Victor Hunt, Kelly Cohen, Manish Kumar

• Staff: Mahdi Norouzi, Bryan Brown

• Students: Paul Guentert, Chan Ventkatesh, Niranjan Krishnan, Arjun Chiddarwar

Scope of Work• Acquire the sensors and hardware and develop supporting software in order to

integrate into prototype systems.

• Test and demonstrate prototype systems for select identified missions.

• Develop mission SOPs and training.

• Deliver all prototype equipment and supporting materials to ODOT.

Project Objectives• How can we improve ODOT core business functions by application of drone

technologies?

• What are these missions? What drone configurations and capabilities are necessary to perform these missions?

Market SurveyPlatform Test Location Application

Location

Subject Matter

Expert

Mission 1:

Aerial Photography/GIS

Mission 2:

Exterior Inspection

Mission 5:

Construction Site Inspection

Mission 6:

Remote Inspection

Mission 7b:

Accident Scene

Platform Development,

Sensor testing and Software

integration

Intel Aero,

FlamewheelUC Victory Pkwy

Labs, Summit Park

Wilmington Air

Park

Fred Judson

Mission 4:

Traffic MonitoringMission 5:

Construction Site Inspection

Mission 7a:

Quick Clear

Gro

up

3 Mission 3:

Interior Inspection

VGCS Prototype,

Intel Aero, Hybrid

UC Victory Pkwy

Facility

JM, FWW, Other Fred Judson, TBD

Gro

up

4 Mission 10:

LIDAR

TBD Wilmington Airpark TBD Fred Judson, TBD

Mission 8:

Search and Rescue

Mission 9:

Fire Monitoring &

Chemical Cloud Detection

Mission 11:

MARCS/Communications

Gro

up

1G

rou

p 2

Oth

er/

Seco

nd

ary

DJI Matrice

100/210, VGCS

Prototype

Sites TBD by

Subject Matter

Experts, candidate

sites under

discussion

Summit Park,

Wilmington

Airpark, MUTC, UC

Victory Pkwy Labs

Hoverfly Typhoon

Summit Park,

Wilmington

Airpark, MUTC, UC

Victory Pkwy Labs

Sites TBD by

Subject Matter

Experts, candidate

sites under

discussion

Fred Judson, TAC,

Mike Brokaw,

Brandon Collett,

Stephan Mather

Fred Judson, TAC,

Carl Merckle, John

McAdam, Bryan

Comer

Matrice, Hoverfly,

Aero, other

UC Victory Pkwy

Labs, Summit Park,

Wilmington Airpark

WVa, MUTC, Other Fred Judson, TBD

Focus

• Survey/Questionnaire• 60+ responses from across ODOT as

well as externally (ODNR, Ohio Patrol, Turnpike, Ohio National Guard, Cleveland Metroparks, etc)

• Review of UAS Market• 400+ vendors

• Market rapidly evolving

• Identified and rank ordered 11 potential mission scenarios

• Initial focus given to Groups 1 and 2

Results to date

• Drone Fleet Acquisition

• VGCS, Interior Inspection

• Jeremiah Morrow, Real-time Milestone Uplink, and MUTC

• Automated Traffic Monitoring

• Image Analysis/Processing

• Common Operating Platform

• Visualization techniques (Hololens AR and VR Goggles)

Drone Fleet • Vehicles selected to best meet ODOT’s application needs

• Used for mapping, inspection, aerial imagery

• Manual and waypoint navigation

• Yuneec Typhoon H with CGO3+ 4K Camera• Up to 25 minute flight time• Live HD Video Link

• DJI Matrice 100 with X5, Z3, and Z30 Cameras• Up to 25 minute flight time• Z3 offers 3x optical zoom • Z30 offers 30x optical zoom• X5 offers high resolution imagery for mapping• SDKs emerging for customization• Compatible with Pix4DCapture, DJI GSPro, Drone Deploy and others

Drone Fleet (cont.)• Intel Aero

• Developmental platform for multiple UAV control

• Allows user to operate multiple drones through single web-based ground station

• Equipped with ADS-B and Intel Realsense for Obstacle Avoidance of other drones, aircraft, and other objects

Drone Fleet (Tethered Platforms)• Vehicles were selected to provide constant

situational awareness by getting power through tether (e.g., traffic monitoring).

• This allows for hours of operation without need to switch out batteries.

• Payloads include cameras, lighting, MARCS repeaters, etc.

• Hoverfly Tethered System• Includes Yuneec Typhoon H with 4K/12MP camera• Max operational altitude of 150ft.

• Powerline Tethered Platform• Works with variety of drones including: Matrice 100,

Inspire 1, Inspire 2• Max operational altitude of 200ft.

Drone Fleet (Upcoming Platform) • DJI Matrice 210 RTK:

• Flight Autonomy

• IP43 Waterproofing

• ADS-B Receiver

• Dual Battery, up to 38 min flight time

• Multiple Payload Configurations

• Camera Payloads:• XT Thermal Camera

• Z30 30x Optical Zoom Camera

• X5 4K/16MP Camera

• Anticipated delivery in Sept/Oct 2017

Interior Inspection - VGCS

Concept Vehicles

Lessons Learned thru Engagement and Feedback

• Obstacle/Collision Avoidance and Stability Control

• Dust Cloud Issue

• Increased the Flight Time

• Simplified Manual Control

• Need to Perch for Extended Duration and for Inspection in Tight Spaces

Jeremiah Morrow Demolition

Video Encoder

Jeremiah Morrow mission from Position 1

Deployed quickly to provide HD video for use by ODOT, media, etc.

https://youtu.be/Q6Q4BcEHfwU

Milestone Uplink Box Objective• Capture Video from Drone • Transfer Video to ODOT Video Servers (Milestone) Real-time• Stream data for local officials

Video Feed

Control commands

Remote Users

SmartphoneLaptopGoggles

On-site Users

Situational Awareness, MUTC Emergency Preparedness Exercise

• Realtime video capture from drone and uplink to Milestone• Stream data for local and remote officials

Traffic Video ProcessingTraffic Analysis

• Off the shelf software doesn’t present drone video solution or is cost prohibitive

• Performed case studies of different drone video analysis solutions

• Currently implementing and testing robust detection process

Proposed Method of Video Analysis

Individual Lane Detection

Multi-Object Detection

Combination of Methods

Drone Video Captured along I75

https://youtu.be/R_N6L9FjX00

Image Analysis/Processing (Mapping, Surveying, 3D Modeling)

Capture Process AnalyzePlan

Pix4DCaptureDJI GSPro

Matrice 100Matrice 210 RTKFalcon 8+

Pix4DODM

GreyhoundEntwynePotree

Custom UCII website

Some sample modelsAccident scene recreation Construction site management Survey and Mapping

SR266 Construction Project, District 10

GSD: 1.63 cm/pixelNum of images: 256(256)No GCP

GSD: 2.2 cm/pixelNum of images: 251 (261)No GCP

Aug 24 Aug 26

Comparison – Measurements and AnalysisAug 24 Aug 26

Common Operating Platform Objective• Central system for archiving documents, aerial images and videos• Back-end for automated processing, transforming images to 3D

models, etc• Front-end to enable the user to interact with 3D models through the

website or through AR/VR tools (e.g., Hololens)

Website Authentication

Upload Raw Data

• 3D Model Generation • Model post-processing for fast

presentation on the website• Model post-processing for

presentation on Virtual Reality Goggles

• Realtime traffic analysis

Present the processed 3D model

to the user

Database

Processing Server

WebsiteUsers

Website Demo

https://youtu.be/F6g4pFztM90

Virtual/Augmented Reality objectiveProvide shared Interaction with 3D models (users at different physical locations)• Interaction with models looking at the Avatar of other users • Share notes with other users• Talk to each other using the VoIP technology

Virtual/Augmented Reality demo

https://youtu.be/2ESgngz2Ojc