integration of built and natural environmental datasets ... · pdf fileintegration of built...

International Seminar onLand Administration Trends and Issues in Asia and the Pacific Region

WG3 Data Integration Project

Integration of Built and Natural Integration of Built and Natural Environmental Datasets within National SDI Environmental Datasets within National SDI

InitiativesInitiatives

20th August 2008 – Kuala Lumpur, Malaysia

Data Integration ProjectData Integration Project

IntroductionIntroductionAbbas RajabifardProject Coordinator

Vice Chair – PCGIAP-Working Group 3

One of the major initiatives of WG3 over the last three years is work on the project of the “Integration of Built (cadastral) and Natrual (topographic) environmental datasets within National SDI initiatives.

Integration of Built (cadastral) and Natural (topographic) Integration of Built (cadastral) and Natural (topographic) environmental datasets within National environmental datasets within National SDIsSDIs

The aim of the project was to better understand and describe the technical, jurisdictional, institutional, legal and land policy perspectivesurrounding the two foundation datasets (cadastral and topographic) in a National SDI.

Integration of Built (cadastral) and Natural (topographic) Integration of Built (cadastral) and Natural (topographic) environmental datasets within National environmental datasets within National SDIsSDIs

ProjectProject AimAim

Integration of datasets to facilitate sustainable development

Sustainable Development

Built Data

Natural Data

The project has investigated the justification for integrating built and natural environmental datasets in support of sustainable development and developingan integration model, framework and associated tools capable of being used in diverse jurisdictions.

Project ObjectivesProject Objectives

• Investigate data integration of natural (topographic) and built (cadastral) environmental datasets in the context of establishing and maintaining national SDIs within countries in Asia and the Pacific region.

• Identify problems, issues, similarities and differences through case studies in: – institutional arrangements; – related SDI policies; – data integration methods and technologies; and – human resource and capacity building

Project ObjectivesProject Objectives

• Identify the technical and non-technical tools and requirements of an effective data integration (Integration toolbox):– Standards– Technical tools

• Integration data model• Spatial data integration validation tool• Metadata content

– Institutional, policy, legal and social guidelines• Design and implement the tools

Project SupportProject Support

• Support of PCGIAP• Research Funding and support of Australian

ARC-Linkage scheme• Support from /Case Study countries

– Australia (three Australian government organisations)

– Thailand– Japan– Malaysia– New Zealand– Brunei Darussalam– Indonesia

Project Case StudiesProject Case Studies

• Adopted case study approach to undertake research which enabled best practice solutions and improved capacity to evaluate the success of countries systems.

• Case studies were facilitated through PCGIAP-WG3 and 7 countries:– Australia– Thailand– Japan– Malaysia– New Zealand– Brunei Darussalam– Indonesia

Local/StateLocal/State

NationalNational

International Case study

International Case study

Case StudiesCase StudiesSp

atia

l dat

a in

tegr

atio

n pr

oces

s

Data Exploration

Data Assessment

Communication

Access/Acquisition

Geo-Processing

Data Collation

TechnicalFramework DataData QualityMetadataSpatial Data Dictionary(Single) Point of AccessData ModelsSpecifications and GuidelinesStandards Interoperability

Non-technicalCollaborationCapacity BuildingEffective Communication

ChannelsPrivacy PolicyRestrictions on UsePricingCustodianship

Case studiesCase studiesCase

StudiesCase

Studies

Stage 1 Current situation of

multi-source spatial data integration

Existing documents and resources

Stage 2The investigation of

technical and Non-technical issues

Questionnaires and country reports

Stage 3Detailed investigation of

technical and non-technical issues

Agency visits, meetings and data audit

WA, Australia

Bhutan

Botswana

Canada

China

India

Greece

Brunei Darussalam

Australia

Malaysia

Japan

Philippines

Singapore

Indonesia

Victoria, AUSTRALIA

NSW, AUSTRALIA

GeoScience, AUSTRALIA

Potential Integration Issues and ObstaclesPotential Integration Issues and Obstacles

Tools and Requirements of effective spatial data integration


Overall Plan and Project MethodologyOverall Plan and Project Methodology

Case study development

Review of current concepts and situation

Literature review

Project Start

Project Start

Compare & contrast case study

results

International workshop –UNRCC-AP

Develop integration framework, data

model & associated tools

Refinemodel &

framework

Project End

Project End

Pilot study

Integration template

June 2005

Oct 05 –Apr 06

Local-StateState-Nationalin Australia

Distributed to PCGIAP members June 06

Sept 2006

March 2007Mar-June 07

June 2007Aug 2008Dec 2008

Develop implementation

Strategy

LA Seminar – 14TH

PCGIAP

LA Seminar – 14TH

PCGIAP

Project Activities and OutcomesProject Activities and Outcomes

• Major Activities– International Case Studies– International Workshop on Data Integration as part of the 17th UNRCC-AP, 2006– Pilot study of Geo-web Service Integration– Development of a data integration framework– Development of a web-based system using the data integration framework to

facilitate data interogrability test and data integration. – Publication Strategy

• Spatial data integration validation tool• Spatial data integration tool• Spatial data integration guideline• Technical and policy reports• Publications

– 5 international papers– 2 book chapters– Project website

Spatial Data Integration Validation Tool

Specification and User Guide

PCGIAP Working Group 3Research Project

Integration of Built and Natural Environmental Datasetswithin National SDI Initiatives

August 2008

Centre for Spatial Data Infrastructures and Land AdministrationDepartment of Geomatics, University of Melbourne

http://www.geom.unimelb.edu.au/research/SDI_research/Integrated/http://www.geom.unimelb.edu.au/research/SDI_research/Integrated/

Technical DetailsTechnical DetailsHosseinHossein MohammadiMohammadi

PhD CandidatePhD CandidateCentre for Centre for SDIsSDIs and Land Administrationand Land Administration

Department of Department of GeomaticsGeomatics, The University of Melbourne, The University of Melbourne

Data Integration ProjectData Integration Project

Why data integration?Why data integration?

Band 1Band 1

Fundamental DataFundamental Data

CadastreRoads

ImageryTopography

CensusAdministrative Boundaries

Street DirectoryWaterwaysVegetation

Band 2Band 2

LocationalLocational

PoliceFire

AmbulanceSES

Public Schools

Private HospitalsAged care

Community CentersCCTVIcons

Key BuildingsPeople at Risk

Private SchoolsPublic Hospitals

Rural Property BuildingsHazardous Chemicals

Oil Spill Response AtlasFlora and Fauna Atlas

Points of Interest

Band 3Band 3

InfrastructureInfrastructure Socio-economicSocio-economic

DemographyEmploymentValuations

Public TransportSchedules

Pedestrians FlowFloor Plans

Hazard Models

ElectricitySub-stations

GasWater

HydrantsSewerage

Storm-waterTelecoms

Buildings DatabaseFerry RoutesBush Fire Prone Zones

Flood Planes3D Buildings

More publicly availableRich metadataConsistent data modelsLow priceClear policiesLow access and use restrictionInteroperable dataNational coverageSingle point of access

More publicly availableRich metadataConsistent data modelsLow priceClear policiesLow access and use restrictionInteroperable dataNational coverageSingle point of access

Less Publicly availablePoor metadataInconsistent data modelsHigh priceNot clear policiesAccess and use restrictionNot Interoperable dataNo Single point of access

Less Publicly availablePoor metadataInconsistent data modelsHigh priceNot clear policiesAccess and use restrictionNot Interoperable dataNo Single point of access

Rob Colles (EICU), 2005

Why data integration?Why data integration?• Hazard management, Tsunami

Utilities

Bathymetry

Buildings Data

Critical Infrastructures

Datasets:

Cadastre

Access Network

Underground Features

…

• Gaps of Data, No Vertical Topology, Currency, Collaboration, Data Exchange Policy,…

• Restrict Privacy• Different Custodians

• Not Completely Fits to Utility and Access Network

• Privacy• Poor Attribution

• Data Barely Exists

• Created and Owned by Users

• NO Nation-wide Data• Poor Metadata

UtilitiesCadastreBuilding Data

Bathymetry

Why data integration?Why data integration?

• PSMA’s Experience

Inconsistencies in Data Model, Metadata, Standards, Specifications, Collaboration Model, and IP issues in Victoria, NSW, GA, …

• G-NAF• CadLite• Transport and Topography• Admin Boundaries• Points of Interest

IDM

ExampleExample

• Data duplication

ExampleExample

• Completeness

ExampleExample

• Data model

11. Physiography 112. Production 113. RailTransport 514. RoadTransport 315. SeriesIndex 416. Relief 217. Surveymarks 118. Utility 219. Vegetation 420. Waterbodies 621. …

1. Admin 42. Aviation 13. Cartography

44. Culture 35. Drainage 76. Errors 17. Framework 48. Habitation 59. Industry 310. Marine 4

1. Address 112. Admin 33. Imagery4. …

1. Addresses 12. Admin 233. Elevation 84. Hydrography 85. Planning 56. Property 147. Transport 88. Vegetation 39. Imagery10. …

Theme

GANSWVictoriaData

ExampleExample

• Data model

DrainageTopoFDHydroCategory

Point_watertankTankPoint

No particular category for Tank points

(HY-WATER_STRUCT_POINT)Beside watering places and

swimming pools

File Name

GANSWVictoriaData

ExampleExample

• Feature Level and Theme Level Metadata

Feature Level Theme Level

Case Studies

Case Studies

Stage 1 Current situation of

multi-source spatial data integration

Existing documents and resources

Stage 2The investigation of

technical and Non-technical issues

Questionnaires and country reports

Stage 3Detailed investigation of

technical and non-technical issues

Agency visits, meetings and data audit

WA, Australia

Bhutan

Botswana

Canada

China

India

Greece

Brunei Darussalam

Australia

Malaysia

Japan

Philippines

Singapore

Indonesia

Victoria, AUSTRALIA

NSW, AUSTRALIA

GeoScience, AUSTRALIA

Potential Integration Issues and ObstaclesPotential Integration Issues and Obstacles



Potential integration issues and challengesPotential integration issues and challenges

DataIntegration

Technical Integration

Funding Model DifferencesUtilizing Inconsistent Collaboration ModelsLack of Awareness of Data

Weakness of Capacity Building ActivitiesCultural IssuesDifferent Background of Stakeholder

Lack of Supporting Legislations Inconsistency in Policy Drivers and Priorities Pricing

Definition of Rights, Restrictions and ResponsibilitiesInconsistency in Copyright and Intellectual Property Rights (IPR) ApproachesDifferent Data Access and Privacy Policies

Tools and requirements for Tools and requirements for effective spatial data integrationeffective spatial data integration

• Integration data model• Data validation tool• Guideline

– Custodianship program– Metadata requirements– Policy arrangements– Etc.

Integration data modelIntegration data model

Physical UniverseParcel as environmental feature

resources, habitat, etc

Envi

ronm

ent E

xper

t

Parcel as feature

aspatial + spatial area, buffer, point in

polygon, topology etc.

GIS Expert

Parcel as legal entity

rights, restrictions, legal boundaries,

interference of interests

Legal

Parcel as coordinates

(x, y) Surveyor

Coordinates

Integration data model: exampleIntegration data model: example

• Victoria– 12 road classes– Stored in a single

layer and singled out by attributes

Integration data model : exampleIntegration data model : example

• New South Wales– 7 road classes– Stored in different layers

Integration data model: exampleIntegration data model: example

• PSMA (Public Sector Mapping Agency)

Victoria

NSW

South Australia

…

Integration data modelIntegration data model• New definition and categorization

PSMA’s Definitions

GA’sDefinitions

Vic’s Definitions

NSW’sDefinitions

Federated Ontology

Spatial Entities (eg Roads)

Queries Data ModelIntegration …

Freeway (adopted from VicMap)

Hard surface formation, high volume, high speed roads declared as “Freeway”; comprising dual carriageway and full access control and grade separated intersections; ie no direct access from adjoining properties or side roads and all crossings are by means of overpass or underpass bridges with traffic entering or leaving carriageways by means of ramps. Single carriageway sections forming part of declared freeways may be included within this category.

Integration data modelIntegration data model• Freeway (adopted from VicMap)

– hard surface– high volume – high speed– dual carriageway– full access control– grade separated intersections– no direct access from adjoining properties or side roads– all crossings are by means of overpass or underpass bridges– traffic entering or leaving carriageways by means of ramps

Can be defined in attributes

and extracted by SQL queries

Can be extracted by topological relations with other spatial datasets

and extracted by spatial queries

Guideline: Metadata requirementsGuideline: Metadata requirements

• Measurable content– Attribute and spatial accuracy– Restrictions/rules on attributes– Scale– Logical consistency– Etc.

Guideline: Custodianship arrangementGuideline: Custodianship arrangement

• Agreements on– Accuracy, completeness, currency and logical

consistency– The provision of effective metadata– Effective collaboration– Adoption of a nationally consistent data model– Adoption of nationally consistent data content

(theme-based)– etc

Data validation toolData validation tool

Real World Applications

(land administration, emergency management,

resource management)

User

FITNESS-FOR-PURPOSEData manipulation

Spatial Data Integration

Data Validation


StandardsGuidelines

DatumFormat

Spatial ExtentAttributesQuality

Restrictionsetc

Integratability WebService

Web-Servers (WFS, WMS)

Local dBases

Remote dBases

Integratability Check

Define Inconsistency

Superimpose

Comprehensive Report

Revision Instruction

Modify

Inconsistency

No inconsistency

Res

tric

tion o

f use

Data validation toolData validation toolOther

BuiltData User

National dBase Coordinator

Data Supplier

Natural

National Database

Other

Built Natural


• Users– Define queries to retrieve datasets

• System coordinator– Define measures– Make configurations

• Data supplier– Define new session for data register– Resume previously commenced sessions– Evaluate datasets– Amend data based on instructions

Structure of the toolStructure of the tool

DatabaseManages authority access level, data register sessions, data records, measures and rules,

jurisdiction specifications

GIS environmentProvide display, analysis and amendment services

Development environment

Manages queries on database, link with GIS, data validation and manipulation

System architecture

System architecture: System architecture: Relational DatabaseRelational Database

• Entities– System user– Register sessions– Issue record– Data record– Jurisdiction– Jurisdiction entities: Custodian, rules, datum, metadata

(schema), bounding box, restrictions, scale and etc

System architecture: System architecture: DevelopmentDevelopment

• ESRI ArcGIS toolbar: developed by macro programming

System architecture: System architecture: DevelopmentDevelopmentUser

Database coordinator Data provider User/data retriever


• Jurisdiction (context) selection


• Data and metadata entry, additional rule definition


• Evaluation report


• Amendment services: attribute manipulation, format conversion, datum conversion


• Record and Display

Scenario: SettingScenario: Setting• Accepted Formats

– ESRI shapefile, ESRI coverage, WFS/WMS, ERMapper ECW• Metadata Standard and DTD/Schema

– ANZLIC’s profile, anzmeta1.3.dtd • Policy agreement

– Privacy policy– Restrictions– Licensing

• Geographical extent• Australia’s boundaries• Datum

– GDA_1994• Currency

– Not older than one month• Minimum and Maximum scale

– 1:1,000 to 1:1,000,000• Completeness

– complete• Attribute and positional accuracies• And user-defined rules:

– No “null” is accepted for attributes

Dataset: contour linesDataset: contour lines• Format

– MapInfo TAB• Metadata and DTD

– ANZLIC’s profile, anzmeta1.3.dtd • Geographical envelope

– Australia, Victoria• Coordinate system

– WGS86• Currency

– Two weeks old• Scale

– 1:25000• Complete• Some “null” attributes

EvaluationEvaluation processprocess• Information is extracted from data and metadata

– Bounding box, restrictions, accuracies, scale, attribute content, format, datum, completeness

• This information is compared against the information defined by the system/jurisdiction

• If any incompliancy occurs, it will be reported and guidelines and if applicable tools to overcome will be provided

• Upon amendment, comparison module evaluates the data and if there is no more issues a record of data is added to system

ReportReport

Amendment toolsAmendment tools

• Amendment services: attribute manipulation, format conversion, datum conversion

Record the dataset and displayRecord the dataset and display

System capabilitiesSystem capabilities

• Capability to adopt new jurisdiction-specific rules and standards

• Extendable to accept new data formats and metadata standards

• Capability to be utilized as a data validator• Capability to be a part of bigger spatial services• Capability to develop new metadata and metadata

schema

Data Integration Validation Tool Data Integration Validation Tool is available onlineis available online

• Software together with specification, manual and installation guide is available through the project website at:

Or• if you require further information or want to report a

feedback, please contact PCGIAP-Working Group 3

www.geom.unimelb.edu.au/research/SDI_research/Integrated

Spatial Data Integration Validation Tool

Specification and User Guide

PCGIAP Working Group 3Research Project

Integration of Built and Natural Environmental Datasetswithin National SDI Initiatives

August 2008

Centre for Spatial Data Infrastructures and Land AdministrationDepartment of Geomatics, University of Melbourne

Specification and GuidelineSpecification and Guideline

• Tools– Data validation– Datum conversion– Interoperability tool/data conversion– Attribute manipulation– Metadata entry tool

Tool FunctionsTool Functions

• System configuration


• System configuration• Data validation


• System configuration• Data validation• Data manipulation


• System configuration• Data validation• Data manipulation• Data registry


• System configuration• Data validation• Data manipulation• Data registry• Data display

Hardware/Software requirementsHardware/Software requirements

• ArcGIS 9.x• Internet connection (in case of using data or

metadata/metadata schema from internet)• Integration.zip (from project website)

Thank you for your attentionThank you for your attention

integration of built and natural environmental datasets ... · pdf fileintegration of built...

Documents