![Page 1: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/1.jpg)
Rohit Singh (@geonumist)Cédric Despierre Corporon (@ceddc)
Geospatial Deep Learningwith arcgis.learn
![Page 2: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/2.jpg)
Session Overview
• What is AI, Machine Learning & Deep Learning• Machine Learning & Deep Learning
- What is Machine Learning?- How is it different from Deep Learning
• Deep Learning workflow in ArcGIS Pro• Geospatial Deep Learning with arcgis.learn• Types of models and their applications
- Training and deploying deep learning models- Scalable deep learning with Image Server
• Resources / Getting started
![Page 3: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/3.jpg)
What is AI?
![Page 4: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/4.jpg)
Neural Networks
TensorFlow
CNTK
Natural Language Processing
Cognitive Computing
GeoAI
Computer Vision
Dimensionality Reduction
Object Detection
Support Vector Machines
Object Tracking
Keras
PyTorch scikit-learn
fast.ai
Random Forest Machine Learning
Deep Learning
Artificial IntelligenceCaffe
![Page 5: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/5.jpg)
Machine Learning
Deep Learning
Artificial Intelligence
![Page 6: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/6.jpg)
Machine Learning
Deep Learning
Artificial Intelligence
![Page 7: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/7.jpg)
Machine Learning
Deep Learning
Artificial Intelligence
CNTK TensorFlowPyTorch
Video game behavioral AI
Keras
ConvolutionalNeural Networks
fast.ai scikit-learn
Computer Vision
Natural Language Processing
Speech Recognition
![Page 8: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/8.jpg)
Machine Learning
![Page 9: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/9.jpg)
Labelled Data Sets: Features + Output
Call Drops
# Complains Subscribed Package
Call Rate Decline
Churned?
4 5 ABC 20% Yes
6 2 ABC 5% No
9 4 XYZ 12% Yes
Features Output (label)
Training Data(Historical)
For Learning..
New Data Prediction
Learning Models
Trained Model
Problem: Predicting Churn
7 5 KLM 8% Yes (75%)
![Page 10: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/10.jpg)
Labelled Data Sets: Features + Output
Segment Type Proximity to Intersection
Time of Day Weather Accident
Highway 0.1 M Morning Raining Injury
Tunnel 0.3 M Evening Sunny Property
Inner 0.2 M Noon Foggy Injury
PipeAge Depth Temperature Pressure Break
20 3 m 95 F 20 P Yes
15 4 m 5 F 35 P Yes
6 3.5 m 2 F 17 P Yes
IS Indirect Fire
IS Explosive Attack
IS Vehicle Attack
Coalition Air Ops
ISIS Camp
0.2 M 2 M 3 M 0.7 M Yes
4 M 9 M 6 M 1 M No
1.2 M 5 M 5 M 0.5 M Yes
# Females 35 - 50 F&B Sales % Urban
Chic#
Competitors Sales
35,000 $400M 55% 15 $20M
14,000 $150M 15% 27 $5M
27,000 $210M 26% 9 $12M
Road Accidents Prediction Water Leakages Prediction
ISIS Location Prediction Retail Sales Prediction
![Page 11: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/11.jpg)
ArcGIS has Machine Learning Tools
ArcGIS
Classification
Clustering
Prediction
![Page 12: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/12.jpg)
![Page 13: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/13.jpg)
Deep Learning Workflowin ArcGIS
![Page 14: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/14.jpg)
Train DL Model
Detect Objects
Export Training Data For DL Model
Training Samples
ArcGIS Pro ArcGIS Proarcgis.learn
Classify Pixels
Collect Samples Export TrainingSamples
TrainPerform Inference
ArcGIS Enterprise ArcGIS Enterprise
ArcGIS Deep Learning Workflow
![Page 15: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/15.jpg)
Data Labeling: Training Samples Manager
• Add Labels • Quickly Collect Samples• Save Samples to a Feature Class
Collect Samples
Export TrainingSamples
Train Perform Inference
![Page 16: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/16.jpg)
Export Training Data for Deep Learning Tool
Collect Samples
Export TrainingSamples
Train Perform Inference
• Exports Samples to Training Images• Each Image has Labels• Supports various formats
![Page 17: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/17.jpg)
Train Model
• arcgis.learn module in ArcGIS API for Python- No installation (ArcGIS Notebooks)- Easy to use - 3-5 lines of code- Models:
- Object Detection- Pixel Classification- Feature Classification
• External Deep Learning Frameworks- Tensorflow- PyTorch…
Collect Samples
Export TrainingSamples
Train Perform Inference
![Page 18: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/18.jpg)
Consume Deep Learning ModelsPerform Inference
Model Definition
ArcGIS User
Inference results
Input Images
InferenceTools
• ArcGIS Image Analyst in Pro• ArcGIS Image Server on Enterprise
Inference Tools
• Classify Pixels Using Deep Learning
• Object Detection Using Deep Learning
Non Maximum Suppression
Collect Samples
Export TrainingSamples
Train Perform Inference
![Page 19: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/19.jpg)
Classify Pixels Using Deep LearningRuns the model on an input raster to product a classified raster, each valid pixel has an assigned class label.
• Mini-batch support• Processor type: CPU or GPU• Parallel processing in ArcGIS Pro and distributed
raster analysis on Enterprise
Inference Tools
• ArcGIS Image Analyst in Pro• ArcGIS Image Server on Enterprise
![Page 20: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/20.jpg)
Detect Objects Using Deep LearningRuns the model on an input raster to produce a feature classcontaining the objects it finds. • Batch processing
• Optional Non Maximum Suppression
• Processor type: CPU or GPU
• Parallel processing in Pro and distributed raster analysis
on Enterprise
Inference Tools
• ArcGIS Image Analyst in Pro • ArcGIS Image Server on Enterprise
![Page 21: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/21.jpg)
arcgis.learn moduleDeep Learning using ArcGIS Python API
![Page 22: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/22.jpg)
gis
geometry
network
schematics
features
realtime
widgets
mapping
env
geocoding
geoenrichment
geoprocessing
raster
geoanalytics
learn
![Page 23: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/23.jpg)
arcgis.learn
The arcgis.learn module in ArcGIS API for Python enables GIS analysts and data scientists to easily adopt and apply deep learning in their workflows. It enables training state-of-the-art deep learning models with a simple, intuitive API.
![Page 24: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/24.jpg)
Beyond Detections: End to End GeoAI Lifecycle enabled by ArcGIS Imagery AI Capabilities
ImageryAccess
Imagery Prep
Training Data Prep
Train & ConsumeModels
Run Inference at SCALE
Feedback Loop
TakeAction
Deploy Models to Production
Before After
• Installing External DL Frameworks
• Dozens of lines of Code
• HARD!
• No Installation (Notebooks)
• 3-5 lines• EASY
![Page 25: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/25.jpg)
Not just “Training”!
Exporting Training Data arcgis.learn.export_training_data
Training DL Modelsarcgis.learn.SingleShotDetectorarcgis.learn.UnetClassifierarcgis.learn.FeatureClassifierarcgis.learn.PSPNetClassifierarcgis.learn.RetinaNetarcgis.learn.MaskRCNNarcgis.learn.EntityRecognizer
Preparing Data (Augmentation)arcgis.learn.prepare_data
Model Managementarcgis.learn.list_modelsarcgis.learn.Model
Model.installModel.uninstallModel.query_info
Inference APIsarcgis.learn.detect_objectsarcgis.learn.classify_pixels
![Page 26: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/26.jpg)
Things you can do today with ArcGIS.LearnObject Detection using SSD, Pixel Classification using Unet, Feature Classification
Damaged Structures
Roads
Swimming Pools
Building Footprints
Oil Pads
Land Cover
Road CracksCars Palm Trees
Pipeline Encroachment
![Page 27: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/27.jpg)
Computer Vision TasksApplied to GIS
![Page 28: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/28.jpg)
Why use Deep Learning for Imagery
ImageNet Visual Recognition Challenge error rate
• Better than human accuracy at vision tasks
![Page 29: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/29.jpg)
Semantic SegmentationAssign a label to each pixel
Cat
Ground
Sky
Turf/Grass
Building
Water
Pixel Classification
Applications:- Land Cover Classification- Pervious/Impervious mapping…
Models:- UNetClassifier- PSPNetClassifier
![Page 30: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/30.jpg)
Image ClassificationAssign a label to a given image
Cat
Applications:- Damaged building classification- Clean or ‘green’ pools…
Undamaged Damaged
Feature ClassificationAssign a label to a given feature
Models (from torchvision):- Inception- ResNet- VGG…
![Page 31: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/31.jpg)
Object DetectionFind objects and their location (bounding boxes)
Applications:- Detect trees, cars, airplanes, …
Models:- SingleShotDetector- RetinaNet
![Page 32: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/32.jpg)
Instance SegmentationFind objects and their precise locations (masks or polygonal features)
Applications:- Building footprint extraction
Models:- MaskRCNN
![Page 33: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/33.jpg)
Scalable Deep Learning
![Page 34: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/34.jpg)
ArcGIS Enterprise for Scaling Deep Learning
• Leverage Raster Analytics to scale inference
• All desktop inferencing tools are accessible through enterprise
• Clients to invoke distributed inferencing – map viewer, Pro, notebooks
• Multi GPU support
• Requires the ArcGIS Image Server license
![Page 35: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/35.jpg)
ExportTrainingDataforDeepLearning Uses a remote sensing image to convert labeled vector or raster data into deep learning training datasets. The output is a folder of image chips and a folder of metadata files.
DetectObjectsUsingDeepLearning Runs a trained deep learning model on an input raster to produce a feature class containing the objects it finds. The features can be bounding boxes or polygons around the objects found, or points at the centers of the objects.
ClassifyPixelsUsingDeepLearning Runs a trained deep learning model on an input raster to produce a classified raster with each valid pixel having a class label assigned.
QueryDeepLearningModelInfo Extracts the model specific settings from the model package item or model definition file.
InstallDeepLearningModel Installs the model package item from portal to the Raster Analysis Image Server.
UninstallDeepLearningModel Uninstalls the model package from portal to the Raster Analysis Image Server
ListDeepLearningModels Lists all the installed model packages on the Raster Analysis Image Server
Pro
+ Se
rver
Too
lsSe
rver
Onl
y To
ols
ArcGIS Enterprise Deep Learning Tools / Services
![Page 36: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/36.jpg)
Image ServerImage Server
Image Server
ArcGIS Pro
Portal
Apps
DesktopAPIs
Web Apps, Maps, Dashboards
ArcGIS Server
ArcGIS Data Store
ArcGIS Services
Image Server
GDB
Content StoreDistributed Raster Data StoreImagery Storage
Notebook Server
Deep Learning Models on Portal
GPU per Server
ArcGIS Enterprise Deep Learning – System Architecture
![Page 37: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/37.jpg)
60,000 buildingsArcGIS Pro: 1 GP100 GPU (16 GB): 4.5 hoursArcGIS Enterprise: 4 nodes RA server with 3 x P40 GPUs (24GB): 20 minutes
Building Detection – Large Scale ProcessingObject Detection using SSD, Pixel Classification using Unet, Feature Classification
![Page 38: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/38.jpg)
Impervious Surface Classification
Coconut Tree Detection
Building Footprint Extraction
Damaged House Classification
Applications of Deep Learning to Imagery
Pixel Classification Object Detection Instance Segmentation Image Classification
End to End Deep Learning – Wide spectrum of deep learning models
![Page 39: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/39.jpg)
Text / NLP
• arcgis.learn – model for extracting location and other entities- EntityExtractor model
![Page 40: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/40.jpg)
Resources for Getting Started
arcgis.learn:• https://developers.arcgis.com/python• https://medium.com/geoai• https://github.com/Esri/arcgis-python-api
Practical deep learning for coders:• https://course.fast.ai/
![Page 41: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/41.jpg)
Complete answersand select “Submit”
Scroll down to find the feedback section
Select the session you attended
Download the Esri Events app and find your event
Please Take Our Survey on the App
![Page 42: Geospatial Deep Learning with arcgis · -Pixel Classification-Feature Classification ... It enables training state- of-the-art deep learning models with a simple, intuitive API](https://reader031.vdocuments.site/reader031/viewer/2022012918/5f0281a57e708231d40499a9/html5/thumbnails/42.jpg)