![Page 1: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/1.jpg)
Drones At Work :: Capturing Data, Generating Insights,
Solving Real Problems
Ong Jiin Joo CTO, Garuda Robotics
[email protected] DSSG - 23/9/2015
![Page 2: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/2.jpg)
Drones at work gathering useful data
(1) Not flying for fun (2) Not flying and shoo5ng for aesthe5cs Drones at work: (1) Solve Customer’s Problems (2) Gather Useful Data:
Can be analyzed to produce intelligence / insights
![Page 3: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/3.jpg)
In the next 45 minutes
• Share our experience – Behind the scenes – Technology and processes
• Data capture workflow • Data analysis workflow
• Precision agriculture case study: Tree counting
![Page 4: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/4.jpg)
Case Study Background
Running example in this presenta1on: Precision Agriculture for Palm Oil Planta5ons Planta1on customers want to know: How many trees are there in my planta5on? This affects: (i) Manpower & equipment planning, (ii) fer1lizer purchase and dissemina1on
![Page 5: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/5.jpg)
In case you haven’t heard of Drones / UAVs …
![Page 6: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/6.jpg)
Our Workflow
Data Acquisi5on / Genera5on
Data Storage / Transporta5on
Data Analy5cs / Presenta5on
1 2 3
![Page 7: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/7.jpg)
Data Acquisition :: Planning
Project Planner -‐ Define objec1ves, targets, obstacles, deliverables
![Page 8: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/8.jpg)
Data Acquisition :: Before Flight
Prepara5on for deployment On-‐site equipment prepara5on
![Page 9: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/9.jpg)
Data Acquisition :: Before Flight
Onsite systems prepara5ons Mission planning and briefing
![Page 10: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/10.jpg)
Data Acquisition :: During Flight • Autonomous Flight – Monitor telemetry, video feed
. HUD (head up display)
Antenna
![Page 11: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/11.jpg)
Data Acquisition :: After Flight
1. Check Data Integrity
• Is the picture clear, focused?
2. Quick Process
• Low res img processing
![Page 12: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/12.jpg)
Data Generation :: Comparing pictures taken over a period of time
![Page 13: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/13.jpg)
Data Generation :: Combining various electromagnetic spectrum
![Page 14: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/14.jpg)
Data Transportation :: Live
![Page 15: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/15.jpg)
Urgency of analysis
• When do we need the deliverable – Real Time or within minutes / hours – Non Real Time (days / weeks)
• Some analysis require huge amount of compute – such as image recognition
![Page 16: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/16.jpg)
Tradeoff between using more bandwidth to transport data elsewhere vs. shipping more
compute power on site
In-‐country Telco Ground Sta1on
Drone Cloud Services
Wi-‐Fi
3G Dongle
Internet backbone
![Page 17: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/17.jpg)
Fallback plan – transport the old way
![Page 18: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/18.jpg)
Data Storage Size
Photogrammetry Example (simplified) • Fly at 100m, Camera FOV 90° both
sides, 1 picture covers 200x200m = 4 ha
• Suppose plantation 10,000 ha square (or 10km by 10km)
• 80% overlap required ~= shooting 5 times same area
• Total size: (10,000/4) * 5MB * 5 = 62.5GB – fits one 64GB SD card.
![Page 19: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/19.jpg)
Data Analytics
• More on this on part 2 of the presentation
![Page 20: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/20.jpg)
Data Presentation :: Image Stitching
• Combine everything or by blocks
• Highly repe11ve
• Lack control points
![Page 21: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/21.jpg)
Data Presentation :: Orthomosaics
• Geometrically corrected • Can be placed on map
Used by surveyors to measure true distance
![Page 22: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/22.jpg)
Data Presentation :: 3D Reconstruction
• Photogrammetry methods
Similarly, used by surveyors to measure length, area and volume of interest in 3D space
![Page 23: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/23.jpg)
DRONE DATA ANALYTICS
![Page 24: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/24.jpg)
Data Analytics Framework
Descrip1ve Analy1cs
Predic1ve Analy1cs
Prescrip1ve Analy1cs
+ +
![Page 25: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/25.jpg)
Descriptive Analytics
Example: Telco Tower Inspec5on • Is the antenna s1ll slanted at 2.8 degrees
from ver1cal? • Any disconnected wires, bird nest, damage
from harsh weather?
Example: Flare Stack Inspec5on • Is the structural integrity of the flare stack
holding up? • Is the flare stack opera1ng at normal
temperature?
![Page 26: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/26.jpg)
Predictive Analytics
What will happen next? Example: Solar Panel • What is wrong? • How many times
observed • Correlate with
electricity yield curve
![Page 27: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/27.jpg)
Back to our case study :: Plantation Management
Dry leaves, but next to river. Why?
Empty space, but no palm planted. Why?
Winding road, difficult to bring harvested palm out. Redo?
Palm of mixed age: high maintenance cost. Is it 1me to replant the en1re area? If so, should the river be shi]ed for water to drain be^er?
![Page 28: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/28.jpg)
Case Study :: Plantation Management
Great! Now I just have to keep it going for 25 years
How much fer1lizers do I need to get?
How should I distribute them so that my workers don’t just throw excess away?
How many trees do I have!?
![Page 29: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/29.jpg)
Tree Counting
• Conventional way(s) – Ground staff count them one by one! – Guesstimate (e.g. 143 trees / ha)
![Page 30: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/30.jpg)
Tree Counting The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
• More advanced ways
1. Satellite imagery
2. Drone imagery
3. Apply Computer vision
![Page 31: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/31.jpg)
Tree Counting
Posi1ve Features Histogram of • Colour • Intensity • Mean • Standard
Devia1on
Nega1ve Features (random sampling)
Our naïve model! RFC
![Page 32: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/32.jpg)
Tree Counting
• Didn’t work so well…
How can we do beZer?
![Page 33: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/33.jpg)
Ways to improve tree counting
• Non-CV techniques – Operations: capture trees at same size and
light intensity (vary altitude, time of flight etc.)
– Domain info: planting patterns, tree distance, max tree per block
– Past data: information from previous flights, manual count, last count
• How about CV techniques?
![Page 34: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/34.jpg)
Ways to improve tree counting
Source: Oil Palm Tree Detec2on with High Resolu2on Mul2-‐Spectral Satellite Imagery h?p://www.mdpi.com/2072-‐4292/6/10/9749?trendmd-‐shared=0 13 April 2014
![Page 35: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/35.jpg)
Ways to improve tree counting
Active research area • Some new proposals • Undergoing R&D and
trials with our corpus • Trials with customer
with existing data about their tree count
![Page 36: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/36.jpg)
Tree Counting :: Next Steps
• Impact from good tree count – Yield prediction and correction – Plantation ops – Prescriptive Analytics together with Arborists
• Next things to classify – Healthy trees vs. sick trees – Other trees / crops – Heterogeneous plantations
![Page 37: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/37.jpg)
Summary
• Drones are already at work delivering actionable insights
• We can capture the data with our drones, but the challenge is to go beyond the descriptive into the predictive and prescriptive analytics
• Lots of opportunities coming soon
![Page 38: Garuda Robotics x DataScience SG Meetup (Sep 2015)](https://reader033.vdocuments.site/reader033/viewer/2022042907/586fa61f1a28abcc238b82d3/html5/thumbnails/38.jpg)
Thank You