using fme to support open data initiatives and inspire
DESCRIPTION
Presentation from IMGS on Using FME to support open data initiatives and INSPIRE from FME World Tour 2012 in Dublin. 10/09/2012TRANSCRIPT
![Page 1: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/1.jpg)
Using FME to support Open Data initiatives and INSPIRECiaran KirkTechnical Director
6th September 2012
Tony KentGIS Technician
![Page 2: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/2.jpg)
![Page 3: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/3.jpg)
The Challenge
You want to meet Open data requirements, but your data is organized rather differently
?
![Page 4: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/4.jpg)
?What we do …
We build data bridges – to your SDI
![Page 5: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/5.jpg)
Harmonization Core Concepts
Harmonization: implied requirement for building an SDI
Disparate sources must be mapped to a common destination data model
Core to the harmonization workflow is a process called schema mapping.
Delivered by services based on open standards
![Page 6: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/6.jpg)
INSPIRE Harmonization Principles
Typical stages:1. Evaluation2. Assembly3. Transformation4. Validation5. Publication
Based on the Spatial ETL concept (Extract, Transform and Load), as applied to INSPIRE SDI’s
![Page 7: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/7.jpg)
Evaluation
Assess destination schema and data requirements Assess source datasets and schema Consider fundamental differences in representation,
resolution Closely inspect actual representative datasets
![Page 8: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/8.jpg)
Data Assembly
Assess the diversity of source data types: vector, raster, CAD, GIS, database, text, XML, web,
3D, sensor and non-spatial Review format and semantic translation needs Decide how to perform necessary joins
ID joins, spatial relates, nearest neighbor, one to many relationships
Goal is to build a data structure that corresponds with your published standard
![Page 9: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/9.jpg)
Metadata – Data about data
Describes data structures• tables• geometry types• data types• fields
Describes data content• coordinate system• extent• modification date• quality, ownership, etc.
![Page 10: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/10.jpg)
Metadata - Purpose
![Page 11: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/11.jpg)
Key FME Metadata Capabilities
• Reading
• Writing
• Updating
• Harvesting
• Validating
• Integration with web services
![Page 12: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/12.jpg)
CSW: Catalog Service for the Web
• Deegree
• GeoNetwork
• OpenGIS
• TerraCatalog
![Page 13: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/13.jpg)
Metadata Services: CSW
• Reading• Writing / Publishing• Updating
![Page 14: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/14.jpg)
Data Transformation - Schema
Reshape source data to match required destination schema
Schema mapping feature type attribute name new attribute creation code lists conditional value mappings
![Page 15: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/15.jpg)
Schema Mapping in FME
Feature Type Mapping in FME Workbench
Attribute Mapping in FME Workbench
Schema Mapping in FME
![Page 16: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/16.jpg)
Value Mapping
FME Data Model Restructuring: Attribute Names & Values
![Page 17: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/17.jpg)
FME SchemaMapper: INSPIRE geographic names
Name mapping
Name & value mapping
FME Workspace
![Page 18: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/18.jpg)
Transformation: Geometry
Non-spatial to spatial Geometry extraction (spatial to GML) Representation transform: CAD drawing lines with labels
to GIS polygonal features with attributes Coordinate System Reprojection (ED50 to ETRF89) Simple to complex geometry
Source point and polygon data to multiple geometric representations (city as point / area, river as line / area)
Generalization and interpolation Highly granular national and regional datasets often require
thinning to be usable on pan-European scales
![Page 19: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/19.jpg)
Validation
Schema validation i.e. INSPIRE (xsds) Data integrity
Unique IDs Geometric integrity (closed polygons) Null values (nullable?) Valid values: ranges and domain codes Data gaps Bounds Network integrity
Custom validity rules specific to domain Validation automation via FME Server upload
Ensure data quality throughout the data transformation process.
![Page 20: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/20.jpg)
Publication
Produce INSPIRE compliant GML Provide discovery, view or download services, for
WxS, GML and other desired formats Publish with FME Server or integrate with your
geo web server of choice Spatial Data Services
![Page 21: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/21.jpg)
Publish workspace to FME Server Store the workspace in a central repository
Make your FME workspaces available to others –over the web
Register the workspace with one or more services (Data Streaming, Data Download, etc.)
Publication with FME Server
21
![Page 22: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/22.jpg)
Open Data Demonstration
![Page 23: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/23.jpg)
Format translation Schema mapping String and list manipulation Data validation Database load and extract XML,GML,WFS: reading, validation,
publication Web services: WFS, WMS, integration
with others Metadata support Enterprise services with FME Server
FME Tools for INSPIRE
![Page 24: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/24.jpg)
FME can provide all the tools to help you support your data sharing needs:
Integrate your data sources Manage your meta data catalogues Transform your data to standard schemas Publish the data in the required formats
Summary
![Page 25: Using FME to support open data initiatives and INSPIRE](https://reader036.vdocuments.site/reader036/viewer/2022081512/5586dad7d8b42ab1678b474b/html5/thumbnails/25.jpg)
Thank You!
Questions?
For more information: Ciaran Kirk – [email protected] Tony Kent – [email protected]