data interoperability extension: best practice · 2019. 6. 14. · data interoperability extension:...
TRANSCRIPT
Data Interoperability
Jason Schroeder
Solution Engineer – Local Government GA & East TN
Best Practices
Agenda
1. What Data Interoperability isA. How to license, install, openB. Why use?
2. Example use casesA. TransformersB. Settings
3. Automation – how to script with Task Scheduler
4. Resources
What is Data Interoperability? Why use it?
ETL – Extract, Transform, Load
1. Can read and write to hundreds of different formats (spatial and non-spatial)
2. Transform data (data not be in a useful format)A. No coding necessaryB. Validate / reject bad data
3. Validate data (Change detection)A. INSERTSB. UPDATESC. DELETES
Data Interoperability 101
• How to:- License – Data Interoperability is an extension for ArcGIS Pro
- Install – Download separate install file from My Esri
- Open – From ArcGIS Pro Analysis ribbon
3 basic ways to use DI demoDive in to Data Interoperability
1. Quick Import & Export2. Spatial ETL
1. User Parameters – pass to Model Builder
3. FME Standalone
Create Spatial ETL Tool:
• A new workbench file will be created or embedded (check ‘Import’) by default
• For existing workbench files:1. To embed the workbench (fmw format will be embedded in toolbox)
A. Browse to the existing workbench file (*.fmw), click Import
2. To use existing workbench independently A. Workbench file can be opened independently with FME Desktop / Data Interoperability
A. Can share with others if they have FME Desktop
B. Browse to the existing workbench file, leave Import unchecked
Use case – Overwrite or Append
• When to use Data Interop over Desktop/Online tools?
• Data is not acceptable in its current form1. Attribute scrubbing2. Spatial scrubbing
Material Date State
1 1232019 Georgia
2 20190124 Mississippi
2 1-24-2019 Texas
2 1-24-2019 Texas
Coded values1 = Plastic2 = Iron
Date formatting issuesFormat with DIRinse and Repeat
Change data Use State Acronym(GA, MS, TX, etc.)
Bad coordinates- 0, 0- NULL
DuplicatesDetect and Remove
Attribute handling demoDive in to Data Interoperability
Use Case – Enrich
1. We may want to enrich the dataA. Intersect with attributes from some other Geometry
Join Council Districts
Enrichment demoDive in to Data Interoperability
This is a simple demonstration- FME can be used for geoprocessing so sky is the limit- Based mostly on convenience
Use Case – Repeatable process
import arcpyarcpy.ImportToolbox("C:/Users/YourUser/Documents/ArcGIS/Projects/DataInterop/DataInterop.tbx")arcpy.OpenData_DataInterop("C:/Users/YourUser/Documents/RaleighCrimes.csv")
Python Script filename(In this case, it is the same
name as the tool)
Path to folder containing the python script
(OpenData.py)
Path to Python
Parameter(input file for Spatial tool)
Path to toolbox
OpenData.py
Resources- Esri Data Interoperability Training:
https://www.esri.com/training/catalog/5c4b6c7f2c10d337f906f885/arcgis-data-interoperability-in-action/
- Bruce Harold (Data Interoperabilty Product Manager) Blog:https://community.esri.com/people/bharold-esristaff
- FME training site:https://www.safe.com/training/recorded/
Please Take Our Survey on the AppDownload the Esri Events app and find your event
Select the session you attended
Scroll down to find the feedback section
Complete answersand select “Submit”