lesley wyborn and stuart girvan geoscience australia, pmd*crc achieving practical cross-...

Lesley Wyborn and Stuart Girvan

Geoscience Australia, pmd*CRC

ACHIEVING PRACTICAL CROSS-JURISDICTIONAL INTEROPERABILITY

FOR THE AUSTRALIAN MINING INDUSTRY

Australian Government

Geoscience Australia

PPDM Australian Data Management Conference 2004 Geoscience Australia

Outline of Presentation• This presentation is about the Australian AUSindustry

‘Interoperability for Geospatial Data Project’

• Our presentation will be based on the OpenGIS® Reference Model

• We will discuss: o what were the drivers for this project

o which pieces of the standard service orientated architecture were developed

o whether other communities and domains can use our results

• We will elaborate on what were the most important lessons learnt


Interoperability for Geospatial Data Project:AcknowledgementsPeople

o Simon Cox, Rob Woodcock, Joan Esterle – CSIROo Stuart Girvan – Predictive Mineral Discovery CRCo Tim Mackey, Aaron Sedgmen – Geoscience Australiao Rob Atkinson, Peter Barrs – Social Change Online

Funding Organisations (~$250K cash)o AUSIndustryo pmd*CRCo Minerals Council of Australiao Geoscience Australiao Every State and Territory Geoscience Agency


OpenGIS® Reference Model

Enterprise Viewpoint

Engineering Viewpoint

Information Viewpoint

Computational Viewpoint

Technology Viewpoint

Business drivers, policies Who else is doing it?

What technology do we use?

Whatlanguage are wespeaking?

What components & Interfaces?

Source http://www.opengeospatial.org/specs

Who does what?

http://www.opengeospatial.org/specs


Key Driver: Minerals Exploration Action Agenda –

June 2004Industry input highlighted• problems in gaining access to pre-competitive geoscience

information.

• described existing information as commonly incomplete and

fragmented across eight government agencies, each with its own

information management systems and structures.

• noted that the disparate systems lead to inefficiencies causing

higher costs, reduced effectiveness and increased risk incurred by

the industry and its service providers


Key Driver: Minerals Exploration Action Agenda –

June 2004Key Initiative

o Australian Government, State and Territory geoscience

agencies, professional associations and industry to

cooperatively develop and implement nation-wide protocols,

standards and systems that provide internet-based access to,

and effective storage and archiving of, industry and government

exploration-related DATA

Recommended actionso Implement web-based services for on-line access

o Develop and endorse a plan for implementation of an Australian

Earth Science Grid

Client has to download

and thenlocallymergeeach fileinto

single data set

The Geoscience Portal is an index of websites

To merge common data types from all surveys either 1. All data sets are downloaded from each source and then combined or

2. All data are combined into a single data set and hosted by one source: currency of data is a major issue

NSWdatabases

Victorian Databases

Queensland Databases

Tasmanian databases

NTDatabases

GADatabases

WADatabases

SADatabases


Global Driver: the exponentially increasing digital world

Therefore:• No long do data,

programs and computer grunt have to be local

• We need to switch to a distributed culture, ie an interoperable culture where data, programs and computer grunt are ‘operable’ regardless of where they are housed

Moore’s Law vs. storage improvements vs. optical improvements. Graph from Scientific American (Jan-2001) by Cleo Vilett, source Vined Khoslan, Kleiner, Caufield and Perkins.


Evolution of the Internet

Source: http://www.dstc.edu.au/Tech_Transfer/Events/Canberra/web_services_cnb02.pdf

People

People

People

Content

Application

Application


Multiple

overlaid maps

One GetMap request:

Web Map Service (WMS) can access multiple maps

Borders

Elevation

Cloud cover

Cities

Source: Reed, C., 2004. Data Integration and Interoperability: ISO/OGC Standards for Geo-Information http://www.directionsmag.com/article.php?article_id=687

http://www.directionsmag.com/article.php?article_id=687


WMS GetMap returns a server’s “dumb” JPEG, GIF or PNG representation of the data on the server. It does NOT return the actual data, only a bitmap of the data.

WMS can’t “give data away.”

Roma




Web FeatureServer

GetFeaturerequest:

Feature &attributedata

Web Feature Service (WFS) returns data.

I-95

I-295

I-87




Multiplethematic

data layers

GetFeaturerequest:

Web Feature Service (WFS) gets operable feature data from multiple servers

Cities

BordersElevation

Each layer is data, not merely a view:Country is:• Name: Italy• Population: 57,500,000• Area: 301,325 sq km. . .




Why seek funding from the AUSIndustry Innovation Access

Program (IAccP)• AUSIndustry is part of the Department of

Industry, Tourism and Resources

• The goal of the IAccP is to promote innovation and competitiveness by improving Australian access to global, leading-edge research and technologies and facilitate their uptake by Australian firms, particularly by SME’s and researchers.

• The IAccP is a technology diffusion program


External links – who else is doing it?

This project provides a mechanism to understand and share practice with other domainsAustralian:

o ASIBA Spatial Interoperability Demonstration Project (SIDP)o Australian Spatial Data Infrastructure (ASDI)o Australian Earth and Ocean Network (AEON)o Australia Partnership for Advanced Computing (APAC)o National Oceans Office (NOO)o Australian Antarctic Division (AAD)

Internationalo GEON (Geoscience Network) - USo NERC Datagrid (UK National Environmental Research Council)o IUGS CGI (International Union of Geological Sciences)


Policy Issues for our Project

• Pricing and Chargingo For both Commonwealth, State and Territory

Geoscience Agencies, all data are free over the interneto Thus we did not have to build charging into our project

• Managing Intellectual Propertyo Easy to put data source and request for client to

acknowledge into data set o Acknowledgement of the data source

at the destination is difficult to enforce

• Copyright and licensingo Source Agency still maintains copyrighto They still need to know who is using ito How this is done is a global issueo Geoscience Australia is participating in the OGC project

on Digital Rights Management (DRM)








Who does what?








Computational Viewpoint: Standard Web Services

Architecture

Services

Registry

ClientApplications

publish

bind

find

Data Access Services

ModelManagement

ServicesProcessing

Services

Computational Viewpoint: Notional Architecture

Services


Registry

ClientApplications

ModelManagement

Services

FeaturesCoverages Models

Real-timedata

publish

bind

find

OntologyService

Metadata

PersistentBindings

FeatureType

Catalog

DatasetMetadata

ProcessingServices


Computational Viewpoint: our simplified architecture of

components and interfaces

Services

Registry

ClientApplications

publish

bind

find


For our demonstrator we are1. not using registries to find or publish services2. focusing on delivering data as operable ‘features’3. hard wiring the applications to the data services

ModelManagement

Services

ProcessingServices


Computational Viewpoint: registries & catalogues

Services

Registry

ClientApplications

publish

bind

find


NOO, QLD NRE, WA DLI, SEEGRID These people are developing/thinking about developing registries and catalogues in Australia


Services

WMS

Coord.Transf.Service

CoveragePortrayalService

Data Services

Portrayal ServicesProcessing Services

ArchivesArchives

WFSWFS WCSWCS SCS Geocoding Chaining

Gazetteer

This Project/ SEEGrid

ASIBA CANRI WA DLI

QLD NRE GA etc

WA DLI GA


Who else is doing what in OZ in the Computational Viewpoint

PortrayalServices Processing

Services


Our Computational Viewpoint of Components and Interfaces

ServicesClient

Applications

bind Data Access Services

This project is focusing on Data Access Services

and on binding these to client applications


Main International Standards used in Project

• OGC Services o Web Feature Service (WFS)o Web Map Service (WMS)

• OGC Languages o Geography Markup Language (GML)o eXploration and Mining Markup Language (XMML)

• W3C Languageso eXtensible Markup Language (XML)

• W3C Web serviceso Simple Object Access Protocol (SOAP)













Who does what?



Who is doing what in Australia in language standards to enable

data binding

Encodings

GML XIMA SLD

ServiceCapabilities

SensorML Obs&Meas

Image Metadata

CSIRO SEEGRID OGC

ServicesClient

Applicationsbind



Traditional GIS data models for Geosceince do not allow the same data to be viewed seamlessly in

2D, 3D, 4D• Data are in points, lines and polygons

• Geometry-centric abstraction relates to the implementation, not the business object

• One shape per feature does not allow multiple spatial properties,

• scale-dependent versions …

Tenement


Information Viewpoint in the OZ Geosciences

Borehole collar location shape collar diameter length operator logs related observations …

Fault shape surface trace displacement age …

Ore-body commodity deposit type host formation shape (point, polygon, 3D

shell) resource estimate …

Observation location subject/specimen/station property/theme/measurand method operator date/time result (+ type/reference

system/scale/classification) …

• We use ‘feature’ models which:o are based on geological conceptso allow for multiple geometries in

2D, 3D, and 4D

Basin formations shape – time

dependent resource estimate …








Who does what?








SEEGrid Community Engineering Viewpoint

Our Project’s Focus

SEEGrid is helping to coordinate the building of an interconnected distributed modelling and computational services, whereby the models as well as the input data and modelling programs are preserved in a model library


Our project’s Engineering Viewpoint

ServicesClient

Applications



Bind

Our projects Engineering Viewpoint in Detail

(who engineered what?)

WA GeochemistryFeature

Data Source

SAWeb Feature Service (WFS)

ClientApplications

GA GeochemistryFeature

Data Source

SA GeochemistryFeature

Data Source

WAWeb Feature Service (WFS)

GAWeb Feature Service (WFS)

South Australia

Western Australia

Gescience Australia

pmd*CRC, GA, CSIRO, Social Change On-line













Who does what?



Our Project’s Technology Viewpoint:

what technology do we use?

ServicesClientApplications



Bind

XMML, GML

Our projects Technology Viewpoint in Detail

ClientApplications

Geoserver (Open Source) PostGIS (OpenSource)

Web Map Composer(Proprietary)

GA ReportingApplication

GA PLOT-ITApplication

South Australia

Western Australia

Geoscience Australia

WAWeb Feature Service (WFS)

SAWeb Feature Service (WFS)

GAWeb Feature Service (WFS)

GA GeochemistryFeature

Data Source

Oracle (Proprietary)

PostGIS (Open Source)

SA GeochemistryFeature

Data Source

WA GeochemistryFeature

Data Source


Our Technology for distributing data

• We used Geoserver – open source

• Needed modification to be able too Source data from complex database (many tables of data) o Produce complex application schemas (ie more complex

than simple GML such as XMML)

• Other WFS software*:o Degree (open source), Cadcorp SIS Map Server, CARIS

Spatial Fusion Enterprise, RedSpider Web 3.0, Map Manager, GenaWare, SclFeatureServer, JCarnacGIS, GeoMedia WebMap, MapXtreme (MapInfo), MapServer(UMN) 4.2 + more

o We do not know if any of these can do complex mapping

* Source: OGC Registered Products

http://www.opengeospatial.org/resources/?page=products


Clienthas todown load andthenlocallymergeeachfileinto asingledata set

The Geoscience Portal is an index of websites

To merge common data types from all surveys either 1. All data sets are downloaded from each source and then combined or

2. All data are combined into a single data set and hosted by one source: currency of data is a major issue

NSWdatabases

Victorian Databases


Tasmania databases

NTDatabases

GADatabases

WADatabases

SADatabases

GADatabases

WADatabases

SADatabases

Many other data types besides geochem!!!

Project results: the Geoscience Portal will now have these wfs

components to enable• access of SA, WA, GA geochem data in real time via common interfaces• each participant to develop their database to suit their business needs

Chem

Chem

ChemWA

GeochemInterface

SAGeochemInterface

GAGeochemInterface

NSWdatabases

Victorian Databases


Tasmania databases

Northern TerritoryDatabases

wfs interfaces still needed for these Surveys


Information Viewpoint requires the most work











Who does what?



XMML schema’s developed by CSIRO

• Borehole• Observation (OGC SensorWeb) • Gravity measurement• Geochemistry/Assay result • Geological material • Geological timescale • Mineral occurrence • Procedure, Project, Station, Specimen, Tenement• Point, Curve, Surface, Solid volume with properties• Structural geology• Time-series • Finite element model (FLAC, FastFlo)• Simulation/model state

implemented but only for WA, SA and GA


Who is the XMML Community?

• The XMML community led by CSIRO is one of the most advanced globally – why?

• The XMML community has been going since 2000• Data types within each domain have different

communities • For XMML

o For locations community is global – this led to GML3o For Geochemistry metadata, community is the Australian

Mining industry• This illustrates the complexity of developing

community schema• There is no national or international coordinating

agency

Introducing the Solid Earth and

Environmental Grid

(SEEGrid)

Community languages

http:www.seegrid.csiro.au


Community pages


Lesson’s learnt: Business Issues

• Interoperability represents perhaps the most significant paradigm shift in how data and information are managed and utilised since the emergence of the Internet

• The Interoperable world is a Markedly Different Business Paradigm: change is from ‘supply’ to ‘demand’o In the supply or ‘push’ paradigm, suppliers push their content to

clients, how they think their clients want to utilise that contento In the demand or user ‘pull’ paradigm clients access and use

what data and information they require, from any data supplier that can supply authoritative data in the appropriate standardised formats.

• For the Interoperable paradigm, data suppliers must be able to map their content word for word to international standard interfaces.

Source: Williams, Neil, 2004. Interoperability – responding to National Drivers. http://www.osdm.gov.au/osdm/docs/resources/osdm_interoperability_con_03112004/neilwilliams1.pdf

http://www.osdm.gov.au/osdm/docs/resources/osdm_interoperability_con_03112004/neilwilliams1.pdf


Lesson’s learnt: Impacts on Business

• Business processo Workflow and versioning control are criticalo Archiving dynamically changing data is an issue

• As clients get very large data sets, statistically valid data mining is a realityo Much more detailed metadata will be required for

machine to machine transfer (SensorML in particular)o Errors are more likely to be picked upo Need for, and administration of, organisation-wide

QA/QC will be a major impact


Key Take Home Messages• Cross jurisdictional interoperability has been achieved in the

geoscience community (almost) by standards based interfaces

• The final project release will be in March at the SEEGrid 2 workshop in Canberra – please come

• The greatest amount of work still needs to be done on the XML data models (application schemas) used in any data system within and between communities

• The ability of technology and specifications to deal with data models that realistically reflect the complexities of real world geoscience from the technical perspective

• Effective global governance for dividing the complexities of the real world into manageable pieces for effective data modelling is required and will help mitigate the above

• Track our progress (or better still join us) on http://www.seegrid.csiro.auhttp://www.pmdcrc.com.au/http://geoserver.sourceforge.net

http://www.seegrid.csiro.au/

http://www.pmdcrc.com.au/



http://geoserver.sourceforge.net/

lesley wyborn and stuart girvan geoscience australia, pmd*crc achieving practical cross-...

Documents

currency of data

data sets

geospatial data project

single data set

territory geoscience

government agencies

source http

information management