www.csiro.au using the assay data exchange standard with wfs to build a complete minerals...

www.csiro.au

Using the Assay Data Exchange standard with WFS to build a complete minerals exploration data-transfer chain

Simon Cox A.Dent, S.Girvan, R.Atkinson

CSIRO Exploration and Mining I. Whitehouse, C. Legg

5 September 2006 R.M.Woodcock

ADX – IAMG 2006

Outline

ADX history

OGC “Observations and Measurements”

ADX3 issues

Interoperability

ADX – IAMG 2006

History

ADX 1

Aug-Sept 2002

Sponsors: Metech, Anglo-American, Newmont, Placer-Dome, WMC

Contractor: Pindan Software

LIMS-focussed

XML document matched workflow, not information model

Iffy use of XML and XML Schema

ADX – IAMG 2006

History (2)

ADX 2

May-Oct 2003 (report: Feb 2004)

Sponsors: Metech, Anglo-American, Newmont, Placer-Dome, WMC, BHP-Billiton, ALS Chemex

Contractor: CSIRO

Introduced normalised Procedure/Specimen/Analysis model

Inherited some limitations from ADX1

ADX – IAMG 2006

ADX3 Goals and experiments

1.Harmonize ADX with generic standards (O&M) + GeoSciML

2.Refine Specimen + Procedure model

• tracking specimen processing history is key to QA/QC procedures

• tracking chain-of-custody is key to audit trail

3.Testing, feedback, documentation, examples, training

Collaborators: Metech, Newmont, ALS Chemex

ADX – IAMG 2006

Standard transfer format allows multiple data sources

WFSClient

WFSServer

WFSServer

B

WFSServer

C

ADX – IAMG 2006

User

GML for assay data:

XMML and OGC O&M

XMML

XMML

XMML

Demonstrated: Same format and standards

ADX – IAMG 2006

Completing the loop?

“From field to desktop”

Regulator

Assay LabADX

Assay-WFS

Assay-WFS/T ADX-request?

ADX – IAMG 2006

Design methodology: model in UML, serialize in XML

<sa:Specimen gml:id="sp1" … >

<gml:description>Specimen test instance</gml:description>

<gml:name>Specimen test 1</gml:name>

<sa:location> … </sa:location>

<sa:sampledOn xlink:href="http://my.big.org/station/st13"/>

<sa:sampleTime ></gml:TimeInstant … /></sa:sampleTime >

<sa:material codeSpace=“urn:x-seegrid:lexicon:specimenType">chips</sa:material>

<sa:mass uom=“kg">0.35</sa:mass>

<sa:reportingSpecimen xlink:href="http://my.big.org/specimen/sp34"/>

<sa:processingStep> … </sa:processingStep>

<sa:processingStep xlink:href="http://my.big.org/pst/p98"/>

<sa:relatedObservation xlink:href="http://my.big.org/obs/m45"/>

</sa:Specimen>

GML Application Schema

ADX – IAMG 2006

«FeatureType»Observ ation

+ quality: DQ_ThematicAccuracy [0..1]+ responsible: CI_ResponsibleParty [0..1]+ result: Any

«FeatureType»Event

+ eventParameter: TypedValue [0..*]+ time: TM_Object

«DataType»TypedValue

+ property: ScopedName+ value: Any

«Union»Procedure

+ procedureType: ProcedureSystem+ procedureUse: ProcedureEvent

AnyIdentifiableObject

«FeatureType»AnyIdentifiableFeature

AnyDefinition

«ObjectType»Phenomenon

+followingEvent 0..*+precedingEvent 0..*

+propertyValueProvider

0..*

+featureOfInterest

1

+generatedObservation

0..*

+procedure 1

+observedProperty1{Definition must be of aphenomenon that is a propertyof the featureOfInterest}

Generic “Observation” model

An Observation is an Event whose result is an estimate of the value of some Property of the Feature-of-interest, obtained using a specified Procedure

- “metadata for a property estimate”

ADX – IAMG 2006


+ quality: DQ_ThematicAccuracy [0..1]+ responsible: CI_ResponsibleParty [0..1]+ result: Any





«Union»Procedure


AnyIdentifiableObject

«FeatureType»AnyIdentifiableFeature

AnyDefinition


+followingEvent 0..*+precedingEvent 0..*


0..*

+featureOfInterest

1


0..*

+procedure 1

+observedProperty1{Definition must be of aphenomenon that is a propertyof the featureOfInterest}

Assay measurements – specialized observation model

“Rock specimen”

“Concentration[analyte]”

“Instrument/technique”

Domain-specialization of observed property, feature of interest, procedure

ADX – IAMG 2006

Assay data issues

Many measurements per (delivered) specimen:

• different analytes

• same analyte QA/QC - “checks”, “splits”, “duplicates”, “replicates”

1. different instruments/labs

2. same instrument, different time

3. different specimen preparation (“splits”)

Observ ation

Procedure

AnyIdentifiableFeature

PhenomenonEvent


0..*

+procedure 1


0..*

+featureOfInterest1

+observedProperty

ADX – IAMG 2006

How to model measurements on split specimens?

customer viewpoint: one specimen, complex procedure

compound “procedure” composed of

preparation phase (as above)

followed by analytical phase

lab viewpoint: many specimens, standard analytical procedure

derived from incoming specimen, with different preparation histories

multi-step preparation procedure

splitting may occur at various points

“biased” vs “unbiased” splits

need to record exact splitting point …

pulp may then go into several analytical systems

ADX – IAMG 2006

Generic “Specimen” model

«FeatureType»Specimen

+ currentLocation: Location [0..1]+ currentSize: Measure [0..1]+ materialClass: GenericName

«FeatureType»SamplingFeature

+ property: TypedValue [0..*]+ responsible: CI_ResponsibleParty [0..1]

«FeatureType»LocatedSpecimen

+ samplingLocation: GM_Object+ samplingMethod: ProcedureSystem [0..1]+ samplingTime: TM_GeometricPrimitive

«Union»Location

+ geometryLocation: GM_Object+ nameLocation: EX_GeographicDescription

«Union»Procedure


+preparationDetails

0..1

ADX – IAMG 2006

Procedure systems, events, histories

AnyDefinition

«ObjectType»ProcedureSystem

+ instanceIdentifier: ScopedName [0..1]

Event

«FeatureType»ProcedureEv ent

::Event+ eventParameter: TypedValue [0..*]+ time: TM_Object

«FeatureType»ProcedureHistory

«Union»Procedure


+procedure

1

Compose

+step 1..* {ordered}

ADX – IAMG 2006

Standard procedure?

preparation and analytical phases

based on a standard design sequence (“path”),

but a specific instance (“history”)

1. has parameters recorded for each step

% passing a given sieve, weight-in/weight-out, instrument settings, … etc

2. may vary from the prototype

How much information can be “inherited”, how much should be explicit …?

N.B. This information does matter:

analysis of results

billing!

ADX – IAMG 2006

Raw XML Zipped XML

Explicit 1470 kB 39 kB

Hand-optimised 650 kB 36 kB

Different encoding strategies

1. fully explicit “UML→GML” – direct from UML model to an XML document

• apparently verbose - 40-60 lines of XML per split

• much duplication between splits

2. manual optimization – inherit parameters from standard procedures

• less duplication

• indirect mapping to information model– XML must be maintained separately from model, processing requires more conditions and loops

Concerned about file-size?

650 kBHand-optimised

1470 kBExplicit

Raw XML

ADX – IAMG 2006

Summary

ADX can potentially harmonize with GeoSciML, O&M

challenge in modeling prep/analytical procedures

let Shannon take care of document size issue

standards support interoperability, lead to round-tripping

www.csiro.au

Thank You

CSIRO Exploration and Mining

Name Simon Cox

Title Research Scientist

Phone +61 8 6436 8639

Email [email protected]

Web www.seegrid.csiro.au

Contact CSIRO

Phone 1300 363 400

+61 3 9545 2176

Email [email protected]

Web www.csiro.au

ADX – IAMG 2006

Conceptual object model: features

Digital objects correspond with identifiable, typed, objects in the real world

mountain, road, specimen, event, tract, catchment, wetland, farm, bore, reach, property, license-area, station

Feature-type is characterised by a specific set of properties

Specimen

ID (name)

description

mass

processing details

sampling location

sampling time

related observation

material

…

ADX – IAMG 2006

Why is a standard methdology useful?

GML-based encoding is supported by standard web data access protocols from Open Geospatial Consortium

ADX – IAMG 2006

Is any of this for real?

SEEGrid interoperability demonstrator 2004-5

ADX – IAMG 2006

PIRSAWeb Feature Service (WFS)

Common Interface Binding – GML/XMML

GA Geochemistry

Feature Data Source

DOIRGeochemistry

FeatureData Source

DOIRWeb Feature Service (WFS)

GAWeb Feature Service (WFS)

Geoserver (Open Source)

PostGIS (Open Source)

OraclePostGIS (Open Source)

CLIENT APPLICATIONS

DATA ACCESS SERVICES

DATA SOURCES

Mapping Reports

PIRSA Geochemistry

FeatureData Source

Little or no change required here

Translation to standards here

ADX – IAMG 2006

Completing the loop

“From field to desktop”

Regulator

Assay LabADX

Assay-WFS

Assay-WFS/T ADX-request?

ADX – IAMG 2006

Assignment of property values

Each property value is either

i. asserted

name, owner, price, boundary (cadastral feature types)

ii. estimated

colour, mass, shape (natural feature types)

i.e. error in the value is of interest

ADX – IAMG 2006

Observations and Features

An estimated value is determined through observation

i.e. by application of an observation procedure

ADX – IAMG 2006

Feature of interest

may be any feature type from any domain-model …

observations provide values for properties whose values are not asserted

i.e. the application-domain supplies the feature types

ADX – IAMG 2006

Some feature types only exist to support observations

Station

+ elevation: DirectPosition [0..1]+ position: GM_Point

SamplingFeature

+ responsible: CI_ResponsibleParty [0..1]

Trav erse

Flightline

Profile

+ begin: GM_Point+ end: GM_Point+ length: RelativeMeasure [0..1]

Shape3D

SurfaceOfInterest

+ area: RelativeMeasure [0..1]

Interv al

Shape2D

SolidOfInterest

+ volume: RelativeMeasure [0..1]

Shape1D

SamplingFeatureCollection

constraints{count(member)>=1}

Swath

Section

Surv eyProcedure

Sounding

LidarCloud

Specimen

+ currentLocation: Location [0..1]+ mass: Measure+ material: CV_Coverage

+shape 1+shape 1+shape 1

+member 0..*

+surveyDetails

0..1

ADX – IAMG 2006

Observation model

Generic Observation has dynamically typed result


+ quality: DQ_DataQuality [0..1]+ responsible: CI_ResponsibleParty [0..1]+ result: Any+ resultDefinition: RecordType [0..1]


«FeatureType»AbstractFeature





«Union»ObsProcedure

+ generalProcedure: ObservationProcedure+ sensorProcedure: SensorML


0..*

+featureOfInterest

1


0..*

+procedure1

+observedProperty

1

ADX – IAMG 2006

Observation specializations

Override result type

ADX – IAMG 2006

SamplingFeature

Specimen


SamplingFeature

Specimen


Observation

Measurement

+ result: RelativeMeasure

Observation

Cov erageObserv ation

+ result: CV_DiscreteCoverage

Mass :Phenomenon

Material :Phenomenon

+observedProperty


+featureOfInterest

+observedProperty


+featureOfInterest

Observations support property assignment

These must match if the observation is coherent with the feature property

N.B. Each “Phenomenon” definition reifies a (Feature) property definition

ADX – IAMG 2006

SamplingFeature

Specimen


Material :Phenomenon

Mass :Phenomenon

Observation

Cov erageObserv ation

+ result: CV_DiscreteCoverage

Observation

Measurement

+ result: RelativeMeasure

Scales :Observ ationProcedure

MicroProbe :Observ ationProcedure


+featureOfInterest

+observedProperty


+featureOfInterest

+observedProperty

+procedure

+procedure

SamplingFeature

Specimen


Observations support property assignment

ADX – IAMG 2006

premises:

O&M is the high-level information model

SOS is the primary information-access interface

SOS can serve:

an Observation (Feature)

getObservation == “getFeature” (WFS/Obs) operation

a feature of interest (Feature)

getFeatureOfInterest == getFeature (WFS) operation

or Observation/result (often a time-series == discrete Coverage)

getResult == “getCoverage” (WCS) operation

or Sensor == Observation/procedure (SensorML document)

describeSensor == “getFeature” (WFS) or “getRecord” (CSW) operation

Sensor service

optional – probably required for dynamic sensor use-cases

ADX – IAMG 2006

SOS vs WFS, WCS, CS/W?

WFS/Obs

getFeature, type=Observation

WCS

getCoverage

getCoverage(result)

Sensor Registry

getRecord

SOS

getObservation

getResult

describeSensor

getFeatureOfInterest

WFSgetFeature

SOS interface is effectively a composition of (specialised) WFS+WCS+CS/W operations

e.g. SOS::getResult == “convenience” interface for WCS

ADX – IAMG 2006

Non-map use of the service

ADX – IAMG 2006

Tabular Reports by Source

Three servers, one report

ADX – IAMG 2006

GML-based data can be …. Rendered into a

queryable map …

… formatted into a

report or ….

… read and used by any

WFS/GML enabled application

Condition: community-standard data model/encoding

One service – many uses

ADX – IAMG 2006 PIRSA DOIR GA

http://cgsrv3.arrc.csiro.au/seegrid/savedapps/filter

Three servers, one map

Mapping client

ADX – IAMG 2006

Bounding Box

Layers

Initial Map Interface – HTML & SVG

Click a Map

Feature

ADX – IAMG 2006

…and see all Analytes on that sample

ADX – IAMG 2006

Desktop Visualisation (Fractal Technologies)

ADX – IAMG 2006

Data Structures

Proprietary Software

Versions of Software

Client

Now – Different formats and standards

www.csiro.au using the assay data exchange standard with wfs to build a complete minerals...

Documents

adx iamg

history adx

procedure slide

generic specimen model

histories slide

adx1 slide

woodcock slide

xml schema slide