infusing gis with artificial intelligence · data-driven algorithms and techniques that automate...
TRANSCRIPT
Infusing GIS with
Artificial
Intelligence
Rohit Singh
Kyunam Kim
What is AI, machine learning and deep learning?
Machine Learning
• Intro to Machine Learning
• Machine Learning use cases
Deep Learning
• Intro to Deep Learning
• Deep Learning use cases
What’s next?
Session Overview
So what is AI?
Neural Networks
TensorFlow
CNTK
Natural Language
ProcessingCognitive
Computing
GeoAI
Computer Vision
Dimensionality Reduction
Object Detection
Support Vector Machines
Object Tracking
Keras
PyTorch scikit-learn
fast.ai
Random Forest Machine
Learning
Deep
Learning
Artificial Intelligence
Caffe
Machine
Learning
Deep
Learning
Artificial Intelligence
Machine
Learning
Deep
Learning
Artificial Intelligence
Machine
Learning
Deep
Learning
Artificial Intelligence
CNTK TensorFlowPyTorch
Video game
behavioral AI
Keras
Convolutional
Neural Networks
fast.ai scikit-learn
Computer
Vision
Natural
Language
Processing
Speech
Recognition
Machine Learning
Data-driven algorithms and techniques that automate
prediction, classification and clustering of data
Traditional Machine Learning• Useful to solve a wide range of spatial problems
• Geography often acts as the ‘key’ for disparate data
Spatial Machine Learning• Incorporate geography in their computation
• Shape, density, contiguity, spatial distribution, or proximity
What is Machine Learning?
Labelled Data Sets: Features + Output
Call
Drops
# Complains Subscribed
Package
Call Rate
Decline
Churned?
4 5 ABC 20% Yes
6 2 ABC 5% No
9 4 XYZ 12% Yes
Features Output (label)
Training Data
(Historical)
For Learning..
New Data Prediction
Learning
Models
Trained
Model
Problem: Predicting Churn
7 5 KLM 8% Yes (75%)
Labelled Data Sets: Features + Output
Segment TypeProximity to
Intersection
Time of
DayWeather Accident
Highway 0.1 M Morning Raining Injury
Tunnel 0.3 M Evening Sunny Property
Inner 0.2 M Noon Foggy Injury
Pipe
AgeDepth Temperature Pressure Break
20 3 m 95 F 20 P Yes
15 4 m 5 F 35 P Yes
6 3.5 m 2 F 17 P Yes
IS Indirect
Fire
IS Explosive
Attack
IS Vehicle
Attack
Coalition
Air Ops
ISIS
Camp
0.2 M 2 M 3 M 0.7 M Yes
4 M 9 M 6 M 1 M No
1.2 M 5 M 5 M 0.5 M Yes
# Females
35 - 50 F&B Sales
% Urban
Chic
#
CompetitorsSales
35,000 $400M 55% 15 $20M
14,000 $150M 15% 27 $5M
27,000 $210M 26% 9 $12M
Road Accidents Prediction Water Leakages Prediction
ISIS Location Prediction Retail Sales Prediction
ArcGIS has Machine Learning Tools
ArcGIS
Classification
Clustering
Prediction
Using the known to estimate the unknown
Use Case: Accurately predict impacts of climate change on local temperature using global
climate model data
Prediction
In ArcGIS: Empirical Bayesian Kriging, Areal Interpolation, EBK Regression Prediction, Ordinary Least Squares
Regression and Exploratory Regression, Geographically Weighted Regression
The process of deciding to which category an object should be assigned based on a
training dataset
Use Case: Classify impervious surfaces to help effectively prepare for storm and flood
events based on the latest high-resolution imagery
Classification
In ArcGIS: Maximum Likelihood Classification, Random Trees, Support Vector Machine
The grouping of observations based on similarities of values or locations
Use Case: Given the nearly 50,000 reports of traffic between 5pm and 6pm in Los Angeles
(from Traffic Alerts by Waze), where are traffic zones that can be used to elicit feedback
from current drivers in the area?
Clustering
In ArcGIS: Spatially Constrained Multivariate Clustering, Multivariate Clustering, Density-based Clustering,
Image Segmentation, Hot Spot Analysis, Cluster and Outlier Analysis, Space Time Pattern Mining
The grouping of observations based on similarities of values
Use Case: Use different economic indicators to define economic regions automatically
Spatially-Constrained Clustering
Similar In ArcGIS: Multivariate Clustering, Density-based Clustering, Image Segmentation, Hot Spot
Analysis, Cluster and Outlier Analysis, Space Time Pattern Mining
Machine learning is also used throughout the platform as a means of choosing smart,
data-driven defaults, automating workflows, and optimizing results
Examples:
• EBK Regression Prediction uses principal component analysis (PCA) as a means of
dimension reduction to improve predictions
• The OPTICS method within Density-based clustering uses ML techniques to choose a
cluster tolerance based on a given reachability plot
• The Spatially Constrained Multivariate Clustering tool uses an approach called evidence
accumulation to provide the user with probabilities related to clustering results
Behind the scenes…
Machine Learning Demos
Integrating ArcGIS with AI Platforms and Libraries
Apps
Desktop
APIs
Accidents PredictionMachine Learning to predict Accident Probability per Segment per Hour
What would Cause an Accident?
Temperature
Sun, Mon, Fri..
Visibility
High/Low
Wind Speed
Fast, Slow..
Snow Depth
High/Low
Time of the Day
12:45, 23:00
Day of the Week
Sun, Mon, Fri..
Month
Feb, Dec..
Road Alignment
Straight / Curved
Proximity to
Intersections
Speed Limit
120 km/h
Sun Direction
East, West
Daily Traffic
AADT
Proximity to
Billboards
…
7 Years of Data
400,000 Accidents
500,000 Segments
10s of
Variables
Road Width
20-30 M
Impossible
to Manually
Analyze
Train a
Machine to
do?
Data Prep > Machine Learning > Visualization
ArcGIS Pro ArcGIS OnlinePython Scikit Learn
Data Preparation Model Training Results Visualization
Smart Road
Digitization
Using
Machine Learning
Problem
Create optimal sequence of stops for a route using incompletely digitized road network.
Today’s Solution
Hand digitize all missing roads based on “traversable” conditions.
Very time consuming…
Laborious, and…Immediately
out of date !
Potential Source To Solution
There exists GPS targets from previous routes
Question…
1. Given massive GPS targets, can we use Machine Learning to detect missing
roads ?
2. Infer a set of connected polylines from the GPS targets to form a complex road
network ?
3. Can we then “augment” the existing road network with newly detected roads to
compute an optimal stop sequence ?
ML Solution
Use BigDataRuntime to Find Unsnapped targets and SmartAssemble targets into
clusters.
ML Solution
Use BigDataRuntime to form an Unsupervised Self Organized Topology for each cluster of
targets.
ML Solution
Self Organized Topology as connected set of Polylines
Applications
• Oil and Gas Remote Assets “connection”
• Local Municipalities
• Roads and Highways
• Logistics for the “last feet” in delivery
Deep Learning
Deep Learning broadly refers to
algorithms that break down
complex problems using layered
algorithm architectures known
as “neural networks”, leveraging
vast quantities of training data
and compute power
What is deep learning?
What is deep learning?
What is deep learning?
What is deep learning?
What is deep learning?
What is deep learning?
So how does it learn?
Feed-Forward Backpropagatio
n
Feed-Forward is test-taking
Feed-Forward is test-taking
Backpropagation is grading
(Your labeled data are the test answers)
The first results are bad…
But a few results are going in the right direction.
Unfortunately, this teacher is harsh…
Our poor-performing students are expelled!
The next round of testing is prepared…
And the testing begins again.
So what is deep learning
useful for?
ETA PredictionPredict Arrival Times accurately to Optimize Supply Chain & Boost Customer Satisfaction
What if we want to accurately
Predict Arrival Times per Fleet Car,
taking into consideration:
• Weekend VS Weekday
• Winter VS Summer
• Holidays VS Traditional Days
• Rush-Hour VS Traditional Hours
• Type of Vehicle
• Time of the Day
• Type of Products
• .. Other variables
Traditional Rout-Solvers wan’t
do the job, we need Machine
Learning to:
• Mine the deep patterns in historical
delivery data (Origin, Destination,
Total Time, Time o the Day, Vehicle
Type, Weather, Type of Products..etc)
• Build Predictive Models that take such
seasonality and patterns into
consideration, to predict arrival times
with high accuracy
ETA Prediction
Nvidia GPU: 2560 CUDA cores
300K Predicted Points/Second
(VS 4-5 points/sec NA)
Network Analyst (Blue) VS Predictions (Red)
Training Data: 320M Network Analyst OD Points
Deep Learning: Computer Vision Use Cases
Image
Classification
Object Detection Semantic
Segmentation
Instance
Segmentation
Dense Crowd
Image Classification
Assigning labels and metadata to provided images
Surface-to-Air Missile
(SAM) site detection
Advanced Object DetectionDetecting complex Objects from Satellite Imagery using a trained deep learning CNN TensorFlow Model.
The model is called on-the-fly via a Raster Function from Pro. Detected Sites are then converted to vector
points to apply further spatial analytics
4. ArcGIS Field Apps like Workforce + Collector and
Survery123 could be used to plan inspections for
detected sites. Operations Dashboard could be used to
monitor execution of assigned tasks in Real-Time
Inspections Results could then be analyzed in ArcGIS
1. ArcGIS “Export Training
Data For Deep Learning” GP
tool used to Prepare the a
labelled training data set from
feature class
2. CNTK or TensorFlow used
to train a CNN to detect objects
of interest using the labelled
training data set
3. ArcGIS Imagery tools used for imagery management and
analytics. A Raster Function is used to call the trained CNN and
generate the results directly at Pro, allowing for further vector and
raster analytics
Object Detection using Deep Learning + ArcGIS Pro1. Sample Training Data 2. Extracting Images 3. Bounding Boxes 4. Train TF CNN
5. Detect Objects 6. Call the model directly from Pro.. ..Via Python Raster Function
Access all GeoAI
Demos from:
https://github.com/A
rcGIS/geo-ai
Request Access by
sending an email to
github_admin@esri.
com
Imagery Tools for Deep Learning
Bounding Boxes
Labelled Pixels+
Export Training Data For Deep LearningAvailable with Spatial Analyst / Image Analyst license
GIS Feature Class Imagery Data
Link for tool here:
http://prodev.arcgis.com/en/pro-app/tool-reference/image-
analyst/export-training-data-for-deep-learning.htm
Object Detection
Extracting features and context location from imagery
Demonstration:
Pedestrian Activity
Classification
Image ClassificationASSIGNING LABELS TO IMAGES
Demonstration:
Finding Swimming
Pools in Parcels
ClassificationASSIGNING LABELS TO IMAGES
Object DetectionEXTRACTING FEATURES FROM IMAGERY
Demonstration:
Real-time Activity
Counting from
Video Camera
Feeds
Object DetectionEXTRACTING FEATURES FROM IMAGERY
Semantic Segmentation
Associating each pixel of an image with a class label
Demonstration:
Pixel-level
Classification
Semantic SegmentationLAND COVER CLASSIFICATION
Good agreement between
CNN and ground truth
CNN fails to contextually label
roads often times
Ground Truth labels roads that
are covered by forest
CNN detects the piers
Piers not in ground truth Roads completely
obscured by forest
Demonstration:
Road Detection
Semantic Segmentation
Instance Segmentation
Extracting features and context location from imagery
Demonstration:
Building
Footprints
Instance Segmentation
What’s next?
What’s Next
Image Translation Using Cycle GAN:
Generating Maps from Satellite Imagery
Generative Adversarial Network
Imagery
Discriminator
Map
Generator
Imagery
Generator
Image Discriminator Map Discriminator
What’s Next
Enhancing Imagery:
Super-resolution
Map Art Generation:
Neural Style Transfer
What’s Next
Cartographic Style
Transfer:
Neural Style Transfer
What’s Next
How can I get started?
These demonstrations can be found at:
• SunTrust Park Pedestrian Classification
• Swimming Pool Detection
• Traffic Camera Video Detection
• Pixel-level Land Cover Classification
• Road Detection Using U-Net
• Building Footprint Mask R-CNN
Resources to learn more:
• ArcGIS API for Python
• Image Classification
• Export Training Data for Deep Learning
• Spatial Statistics
• Medium Blog
Other useful links:
• IBM Watson
• CNTK
• TensorFlow
• fast.ai
• PyTorch
