leeds university geospatial metadata workshop 20110617

Geospatial Metadata and Spatial Data WorkshopGeospatial Metadata and Spatial Data Workshop

School of ComputingSchool of Computing

University of LeedsUniversity of Leeds

Workshop ProgrammeWorkshop ProgrammePRESENTATION SESSION:PRESENTATION SESSION:

Background information

Metadata, standards and application profiles

REFRESHMENTS (30 minute break)REFRESHMENTS (30 minute break)

UK Academic Geospatial Metadata Application Profile, Version 2.1 (UK AGMAP 2.1) and guidelines

Geodoc Metadata Editor tool, GoGeo portal and other resources

ShareGeo Open spatial data repository

Current and future activities

DEMONSTRATION/HANDS-ON SESSION: DEMONSTRATION/HANDS-ON SESSION:

Geodoc Metadata Editor tool, GoGeo portal and ShareGeo Open spatial data repository

BackgroundBackground three decades of GIS and spatial

data capture technology

an eclectic range of academic disciplines using GIS as a research and teaching tool

considerable cost and time invested in spatial data creation

*2006 spatial data audit at 4 universities: +500 dataset files, 100s of orphan datasets

Requires a spatial data managementspatial data management, discoverydiscovery and sharing solutionsharing solution delivered through portal technology and metadata metadata.

So what is So what is METADATAMETADATA?The word appears to be of Greek and Latin origin……

but metadatametadata represents something completely different……

Photographic Images copyright: Jupiter Images 2006

Photographic Images copyright: Jupiter Images 2006 and it’s not sun and holiday

Represents a documented and ordered summary of information that describes something, in this case, spatial data.

Provides the What, Where, When What, Where, When and WhyWhy for a spatial dataset.

Includes its Ownership Ownership and and Contact (Who)Contact (Who) details and Access Access andand Use Use conditions.

Metadata (data describing data)Metadata (data describing data)

WhatWhat are the ingredients?

WhereWhere wereingredientsproduced?

WhoWho sells theingredients?

WhatWhat are thebrewing steps?

WhenWhen does the fermentation process end?

Photographic Images copyright: Jupiter Images 2006

Think of metadata as a recipe for making beer.Think of metadata as a recipe for making beer.Think of metadata as a recipe for making beer.Think of metadata as a recipe for making beer.

metadatametadatametadatametadata

Photographic Images copyright: Jupiter Images 2006

Think of metadata as food product labelling.Think of metadata as food product labelling.Think of metadata as food product labelling.Think of metadata as food product labelling.

WhatWhat are the ingredients?

WhatWhat is the nutritional value?

HowHow many calories and howhow much fat?

WhenWhen is this product’s expiry date?

WhereWhere was it produced?

WhoWho produced it?

WhatWhat are the ingredients?

WhatWhat is the nutritional value?

HowHow many calories and howhow much fat?

WhenWhen is this product’s expiry date?

WhereWhere was it produced?

WhoWho produced it?

metadatametadatametadatametadata

WhereWhere are these datasets’ study areas?

WhenWhen were the data collected?

WhyWhy were these datasets created? 

type of application?spatial reference system?

spatial accuracy?processes or algorithms used?

WhoWho created these datasets?

Can you tell me from any of these files…..Can you tell me from any of these files…..

Now think of metadata as spatial data labelling.Now think of metadata as spatial data labelling.

WhatWhat do these polygons represent?

WhatWhat attribute information is associated with these polygons?

WhatWhat do these polygons represent?

WhatWhat attribute information is associated with these polygons?

WhatWhat do these SOILCLASS

values mean?

WhatWhat does this attribute mean?

metadatametadatametadatametadata

The importance of geospatial metadataThe importance of geospatial metadata

Local Data Management

MetadataMetadata RecordRecord

Spatial Dataset Spatial Dataset Directory or Directory or RepositoryRepository

SpatialSpatialDatasetsDatasets

Private MetadataPrivate MetadataDirectoryDirectory

GeoportalGeoportal: an interface to run searches to discover metadata records representing spatial data and geo-services on the internet.

GeoportalGeoportal Metadata Metadata RecordRecord

Spatial DatasetsSpatial Datasetsand Geo-servicesand Geo-services

Metadata: to discover spatial data via a Geoportal Metadata: to discover spatial data via a Geoportal

Geoportaleoportal:: free textfree text, resource typeresource type, geographic locationgeographic location (co-ordinate and placename) and date date searches.Geoportaleoportal:: free textfree text, resource typeresource type, geographic locationgeographic location (co-ordinate and placename) and date date searches.

Discovering spatial data through metadata offers the Discovering spatial data through metadata offers the prospect of developing new applicationsprospect of developing new applications

Contours

Raster Map

Draped 3D Model

and creating new datasetsand creating new datasets

© Crown Copyright/database right 2008

© Crown Copyright/database right 2008© Crown Copyright/database right 2008

© Crown Copyright/database right 2008© Crown Copyright/database right 2008

Other benefits: data protectionOther benefits: data protection

Photographic Images copyright: Jupiter Images 2006

protects investments of time and cost dedicated to dataset creation and development;

maintains an inventory of datasets to reduce time required to re-assess existing datasets for new and future applications;

ensures integrity of existing and new datasets using metadata as a tracking mechanism to monitor changes and edits to datasets;

reduces and minimises the disruptive effects of staff taking annual leave or departing for new careers;

eliminates or reduces the risk of redundancy in dataset collection;

saves against accidental deletion of dataset files.

Metadata creationMetadata creation

1. Intellectual Property Rights (IPR)2. residual licensed data rights for derived data 3. concerns over data quality (data creator and user)4. liability fears5. privacy and security6. time and cost for data delivery7. data transformation and harmonisation (scale, positional accuracy

projections, formats)8. legacy data9. time and cost to create anonymised data for release10. time and cost for metadata record (descriptive level) updates11. infrastructure performance, maintenance & enhancement costs 12. data and software archiving and warehousing issues13. long-term commitment and investment in the infrastructure14. revisions to changes in standards15. confusion about standards compliance and which standard to use

1. Intellectual Property Rights (IPR)2. residual licensed data rights for derived data 3. concerns over data quality (data creator and user)4. liability fears5. privacy and security6. time and cost for data delivery7. data transformation and harmonisation (scale, positional accuracy

projections, formats)8. legacy data9. time and cost to create anonymised data for release10. time and cost for metadata record (descriptive level) updates11. infrastructure performance, maintenance & enhancement costs 12. data and software archiving and warehousing issues13. long-term commitment and investment in the infrastructure14. revisions to changes in standards15. confusion about standards compliance and which standard to use

Many concerns remain (metadata and data)Many concerns remain (metadata and data)Many concerns remain (metadata and data)Many concerns remain (metadata and data)

Metadata standardMetadata standard Provides precise specifications to enforce and ensure consistency

and interoperability.

Defines and describes metadata entities and elements and, classify and group relevant metadata elements with entities.

Assigns structure and conditions (obligations, data type, domain).

Dublin Core (ISO 15836)Dublin Core (ISO 15836)Dublin Core: 15 elements to facilitate simple resource discovery in anetworked environment (e.g. internet or library).

T

Photographic Images copyright: Jupiter Images 2006

- Contributor- Contributor

- Title- Title

- Date- Date

- Description- Description

- Format- Format

- Identifier- Identifier

- Language- Language

- Publisher- Publisher

- Rights- Rights

- Source- Source

- Subject- Subject

- Relation- Relation

- Coverage- Coverage

- Creator- Creator

- Type- Type

Geospatial metadata standardsGeospatial metadata standards

Federal Geographic Data Committee (FGDC) Content Standard Federal Geographic Data Committee (FGDC) Content Standard for Digital Geospatial Metadata (CSDGM)for Digital Geospatial Metadata (CSDGM) introduced in mid 1990s for documenting spatial datasets.

ISO 19115 Metadata Standard for Geographic InformationISO 19115 Metadata Standard for Geographic Information was ratified in 2003 and supersedes FGDC.

Geospatial metadata standards are critical for supporting Geospatial metadata standards are critical for supporting metadata creation and………………………… metadata creation and…………………………

OtherContent Providers

UK Location Programme

Network

Geo-data Gateway

Local GoGeodatabase

Geoportal

portal interoperability and search capabilities across the internet.

User

Centre for Ecology and Hydrology (CEH)Centre for Ecology

and Hydrology (CEH)

National Environment Research Council (NERC)

National Environment Research Council (NERC)

National Soil Resources Institute (NSRI)

National Soil Resources Institute (NSRI)

•Derived from a geo standard and represents a reduction of the number of entities and elements.

* It should include the core (mandatory) element set of a standard to support interoperability across the wider geospatial community (DiscoveryDiscovery level level metadata).

* Perhaps include other elements for DescriptiveDescriptive level metadata?

* A profile can be extended to include elements which are best suited for a working group’s specific applications.

Example: The Biological Data Profile (BDP)The Biological Data Profile (BDP)

An approved profile with additional elements to document biological information such as taxonomytaxonomy, , methodologymethodology and analytical toolsanalytical tools.

•Derived from a geo standard and represents a reduction of the number of entities and elements.

* It should include the core (mandatory) element set of a standard to support interoperability across the wider geospatial community (DiscoveryDiscovery level level metadata).

* Perhaps include other elements for DescriptiveDescriptive level metadata?

* A profile can be extended to include elements which are best suited for a working group’s specific applications.

Example: The Biological Data Profile (BDP)The Biological Data Profile (BDP)

An approved profile with additional elements to document biological information such as taxonomytaxonomy, , methodologymethodology and analytical toolsanalytical tools.

http://www.flickr.com/photos/f10n4/186861991/http://www.flickr.com/photos/f10n4/186861991/

Geospatial Metadata Application ProfilesGeospatial Metadata Application Profiles

Creating application profiles from ISO 19115Creating application profiles from ISO 19115

ISO 19115Metadata Standard

ISO 19115 Core Element Set

Application

Profiles

Academia (43 + 47=90)

Public Sector(43 + 62=105)

Private Sector(43 + 12=55)300+

elements

43 elements

EnvironmentalSciences

EnvironmentalSciences

SpecialisedAPs

* INSPIRE Directive Metadata Guidelines* INSPIRE Directive Metadata Guidelines

* UK GEMINI 2.1* UK GEMINI 2.1, an INSPIRE compliant geospatial metadata standard for the UK

* ANZLIC Metadata Profile

* North American Application Profile (NAP), Canada and the US

* INSPIRE Directive Metadata Guidelines* INSPIRE Directive Metadata Guidelines

* UK GEMINI 2.1* UK GEMINI 2.1, an INSPIRE compliant geospatial metadata standard for the UK

* ANZLIC Metadata Profile

* North American Application Profile (NAP), Canada and the US

ArchaeologyArchaeology

BiologicalSciences

BiologicalSciences

GeoSciences

GeoSciences

HistoryHistory

HealthInformatics

HealthInformatics

Infrastructure for Spatial Information

in the European Community (INSPIRE)

Infrastructure for Spatial Information

in the European Community (INSPIRE)

*European Commission (EC)

*European Environment Agency (EEA)

*Representatives from Member States (Mapping/GIS)

*European Commission (EC)

*European Environment Agency (EEA)

*Representatives from Member States (Mapping/GIS)

INSPIRE Directive Metadata GuidelinesINSPIRE Directive Metadata Guidelines

INSPIRE Directive [2007 /2/ EC]INSPIRE Directive [2007 /2/ EC]INSPIRE Directive [2007 /2/ EC]INSPIRE Directive [2007 /2/ EC]

Targets electronic spatial data and services for environmental information.

Aims to create a European Spatial Data Infrastructure (SDI) based on Member States’ infrastructures, to improve interoperability of spatial information.

Data and services to be delivered through European initiatives.

INSPIRE Regulations came into force on 31 December 2009 and applies to England, Northern Ireland and Wales.

Scotland’s Parliament enacted a complementary regulation on the same date.

Public authorities obliged to produce and keep ‘metadata’ current for describing datasets, dataset series and geo-services.

Includes UK academia (Freedom of Information Act, 2000).

Targets electronic spatial data and services for environmental information.

Aims to create a European Spatial Data Infrastructure (SDI) based on Member States’ infrastructures, to improve interoperability of spatial information.

Data and services to be delivered through European initiatives.

INSPIRE Regulations came into force on 31 December 2009 and applies to England, Northern Ireland and Wales.

Scotland’s Parliament enacted a complementary regulation on the same date.

Public authorities obliged to produce and keep ‘metadata’ current for describing datasets, dataset series and geo-services.

Includes UK academia (Freedom of Information Act, 2000).

• Provide metadata catalogues to reveal what information is available.

• Provide online data discoverydiscovery, viewview, downloaddownload and transformationtransformation (interoperability) services.

• Licensing arrangements to allow for information sharing, access and use in accordance with each State’s regulations.

• Set up e-commerce arrangements where charging is applicable.

• Introduce monitoring mechanisms to show that information is being made available.

• Introduce co-ordination mechanisms to ensure effective operation of the infrastructure.

• Comply with the 34 spatial data specifications in three annexes (reference geographies, environmental datasets).

• Provide metadata catalogues to reveal what information is available.

• Provide online data discoverydiscovery, viewview, downloaddownload and transformationtransformation (interoperability) services.

• Licensing arrangements to allow for information sharing, access and use in accordance with each State’s regulations.

• Set up e-commerce arrangements where charging is applicable.

• Introduce monitoring mechanisms to show that information is being made available.

• Introduce co-ordination mechanisms to ensure effective operation of the infrastructure.

• Comply with the 34 spatial data specifications in three annexes (reference geographies, environmental datasets).

INSPIRE Regulations for member statesINSPIRE Regulations for member statesINSPIRE Regulations for member statesINSPIRE Regulations for member states

INSPIRE spatial data themes and deadlines for creating metadataINSPIRE spatial data themes and deadlines for creating metadata

INSPIRE Metadata Editor tool and GeoportalINSPIRE Metadata Editor tool and Geoportal

* First released in 2004 to support creation of ISO 19115 and e-GMS compliant metadata - supersedes the National Geospatial Data Framework (NGDF)

* Targeting the UK public sector

* 2010: UK Location Programme (UKLP) revised UK GEMINI 2.1 to meet the requirements of the EU INSPIRE Directive.

* First released in 2004 to support creation of ISO 19115 and e-GMS compliant metadata - supersedes the National Geospatial Data Framework (NGDF)

* Targeting the UK public sector

* 2010: UK Location Programme (UKLP) revised UK GEMINI 2.1 to meet the requirements of the EU INSPIRE Directive.

UK GEMINIUK GEMINI

UK Location Programme (UKLP) is a pan-government

collaborative initiative with theresponsibility to develop and implement

the INSPIRE Directive 2007/2/EC and the UK Location Strategy.

UK Location Programme (UKLP) is a pan-government

collaborative initiative with theresponsibility to develop and implement

the INSPIRE Directive 2007/2/EC and the UK Location Strategy.

UK Location Metadata Editor and Catalogue built with UK GEMINI 2.1

UK Location Metadata Editor and Catalogue built with UK GEMINI 2.1

http://www.flickr.com/photos/musebrarian/3289649684/#/http://www.flickr.com/photos/musebrarian/3289649684/#/

Metadata Metadata Madness!Madness!Metadata Metadata Madness!Madness!

Metadata and spatial data resources forMetadata and spatial data resources forUK academiaUK academia

UK Academic Geospatial Metadata Application UK Academic Geospatial Metadata Application Profile, Version 2.1 (UK AGMAP 2.1)Profile, Version 2.1 (UK AGMAP 2.1)

UK AGMAP 2.1 created to support the specific needs of the UK H&FE communities.

Comprises elements from ISO 19115, UK GEMINI 2.1 and INSPIRE.

Supports documentation of a dataset, dataset series or geo-service for the purpose of discovery and description.

Mapped to Dublin Core, FGDC, INSPIRE, UK GEMINI 2.1 and DDI.

UK AGMAP 2.1 profile for datasets and dataset series

UK AGMAP 2.1 profile for datasets and dataset series

Contact details = 7

Drop down lists = 9

Contact details = 7

Drop down lists = 9

Red: MandatoryRed: Mandatory

Green: ConditionalGreen: Conditional

29 mandatory29 mandatory

90 elements 90 elements

UK AGMAP 2.1 profile for

Geo-services

UK AGMAP 2.1 profile for

Geo-services

Contact details: 7

Drop down lists: 6

Contact details: 7

Drop down lists: 622 mandatory22 mandatory

39 elements 39 elements

Contain descriptions and examples to assist metadata creators and GoGeo portal users from eclectic range of academic disciplines.

UK AGMAP 2.1 GuidelinesUK AGMAP 2.1 Guidelines

Most spatial data information is stored in our heads.

We need to move it from there to an electronic file.

Metadata CreationMetadata Creation

Photographic Images copyright: Jupiter Images 2006

Mind reading aliens?

Solutions?Solutions?Solutions?Solutions?

Photographic Images copyright: Planet Vulcan Images 2006

Metadata extraction device?

Geodoc Metadata Editor ToolGeodoc Metadata Editor ToolGeodoc Metadata Editor ToolGeodoc Metadata Editor Tool

Geodoc design and functionality• Java-built online tool

• authentication required

• text fields and drop-down lists for entering information

• validation (red fields)

• access to metadata guidelines and Geodoc reference guide

Direct links to relevant UK AGMAP 2.1 guideline pages Direct links to relevant UK AGMAP 2.1 guideline pages

MyGoGeo! automation and customisation tool

Geodoc coordinate extent toolGeodoc coordinate extent toolMap tool captures co-ordinate values for bounding box elements used to define the extent of a dataset’s study area.

Extents for Nations atthe click of the mouseExtents for Nations atthe click of the mouse

Personal and secure directory for storing, editing and exporting metadata records.

Geodoc exports UK AGMAP 2.1metadata records in

PDF format

Geodoc exports UK AGMAP 2.1metadata records in

PDF format

Publication on open or private (Group) nodesPublication on open or private (Group) nodes

1) create and validate record;2) submit record for review; 3) metadata creator is contacted; and4) record is published on the GoGeo portal.

11 22

33

Easy steps to the publication Easy steps to the publication of a geospatial metadata recordof a geospatial metadata record

Photographic Images copyright: Jupiter Images 2006

44

A simple interface designed for UK academia to run queries to discover metadata for spatial datasets, and to locate geographic resources.

The portal enables searching by the use of various options including

- free text - date - resource type - geographic location

GoGeo Portal: www.gogeo.ac.ukGoGeo Portal: www.gogeo.ac.ukGoGeo Portal: www.gogeo.ac.ukGoGeo Portal: www.gogeo.ac.uk

GoGeo rebuilt and redesigned.

Launched on 17 May 2011

GoGeo rebuilt and redesigned.

Launched on 17 May 2011

GoGeo portal rebuilt with GeoNetwork GoGeo portal rebuilt with GeoNetwork ((http://geonetwork-opensource.org/)http://geonetwork-opensource.org/)

GeoNetwork: A standard’s (ISO 19115) based, free and open source catalogue application to manage spatially referenced resources through the web.

Provides metadata editing and search functions as well as an embedded interactive web map viewer.

GoGeo portal rebuilt with GeoNetwork GoGeo portal rebuilt with GeoNetwork ((http://geonetwork-opensource.org/)http://geonetwork-opensource.org/)

GeoNetwork: A standard’s (ISO 19115) based, free and open source catalogue application to manage spatially referenced resources through the web.

Provides metadata editing and search functions as well as an embedded interactive web map viewer.

GoGeo simple search Placename search using Unlock middleware gazetteer

Placename search using Unlock middleware gazetteer

Search results page and extents display

Metadata record published on GoGeo

Advanced search- Map Type- Location- Text- Date Ranges- Catalogue- Category- INSPIRE

- Map Type- Location- Text- Date Ranges- Catalogue- Category- INSPIRE

Access to metadata records published on a Group nodeAccess to metadata records published on a Group node

Why publish metadata on the GoGeo portal?Why publish metadata on the GoGeo portal?

a repository for you to store and manage your metadata thus savings in cost and time;

use metadata to announce your data and applications;

advertise (and sell?) your spatial datasets to other interested parties in academia and in the private and public sectors;

metadata in the portal can be referenced and cited for project proposals;

could be configured as an internal resource to access and share datasets.

* Cambridge, Edinburgh, Kingston, Leeds, Leicester, Sheffield and Southampton universities have created and published +200 metadata records on GoGeo portal (global and private).

GoGeo resource channels 3,000+ resources with daily updates

MyGoGeo!: bookmark resources

Other geospatial metadata resources

Geospatial Metadata Workshops July 2007 Aberdeen University Marine/ECO-IMAGINE

Geospatial Technologies Course

2008 Sheffield University

Southampton University

May 2010Plymouth University

2003 Durham University

City University

2004University of Essex

York University Strathclyde University

Bradford University Oxford Brookes

University Coventry University

Edinburgh UniversityOxford University

Kingston UniversitySheffield University

2006 Aberdeen UniversityLeeds University

Cambridge UniversityOxford UniversityStirling University

Glasgow University Heriot-Watt University Strathclyde University

Kingston UniversityGISRUK Conference

(Nottingham)Edinburgh University

Autumn 2010 Cardiff University

Aberdeen UniversityLancaster UniversityGlasgow University

Civil Engineering

Biological Sciences

Planning

Environmental Sciences

Architecture

Information Technology

Retailing

Climate

Sociology

Library Science Landscape Planning

Archaeology

Mathematics

Health

Ecology

Geosciences

Computing Spring 2011

Oxford University Northumbria University Cambridge University Strathclyde University

Leeds University

ShareGeo OpenShareGeo Open A repository for deposit and extraction of spatial data.

Supports access to and sharing of spatial data.

Holds national and international spatial datasets (raster, vector and tabular).

Key to delivering a Spatial Data Infrastructure (SDI) for the UK academic GI community.

ArcGIS plugin to create metadata to deposit with data.

Spatial dataset extractionSpatial dataset extraction

Search ShareGeo via the GoGeo PortalSearch ShareGeo via the GoGeo Portal

Spatial dataset submission Spatial dataset submission

Basic metadata form to complete and extents extracted from dataset.Basic metadata form to complete and extents extracted from dataset.

Ordnance Survey (OS OpenData)

ShareGeo DigimapShareGeo Digimap Solution for spatial data derived from licensed data.

Solution for spatial data derived from licensed data.

Current and future activitiesCurrent and future activities

-UK AGMAP 2.1-Guidelines

-Geodoc metadata tool-GoGeo portal nodes

-Workshops-eLearning objects

Geography

Archaeology

Geological Sciences

Biological Sciences

Research

Offer GoGeo resources for local spatial data management

UK AGMAP 2Guidelines

Geodoc tadata toolCustomised GoGeo Portal

NodesTraining

Geography

Archaeology

Geological Sciences

Biological Sciences

University A

UK AGMAP 2 Guidelines

Geodoc metadata toolCustomised GoGeo Portal

NodesTraining

Geography

Archaeology

Geological Sciences

Biological Sciences

University B

UK AGMAP 2Guidelines

Geodoc metadata toolCustomised GoGeo Portal

NodesTraining

Geography

Archaeology

Geological Sciences

Biological Sciences

University C

UK AGMAP 2Guidelines

Geodoc metadata tool Customised GoGeo Portal

NodesTraining

Geography

Archaeology

Geological Sciences

Biological Sciences

University D

Open Spatial Data Infrastructure (SDI) Open Spatial Data Infrastructure (SDI) for UK academiafor UK academia

Otherresources and portals

Spatial Data Spatial Data RepositoryRepository

Spatial data

Metadata

Search

Datauser

Metadata

Discover

Locate

Access

Use

Publish

Fit for purpose?

Preserve

Steps to data immortality

Questions?

Using the Geodoc Metadata Editor Tool to create a valid geospatial metadata record

Practical 1

Access MyGoGeo! to enter contact details.

Read description of CORONA satellite image dataset.

Access Geodoc Metadata Editor tool and open reference documents on home page.

Create and save a valid metadata record referencing the information provided for the CORONA satellite image dataset.

Export the metadata record into UK AGMAP 2.1 format using XML and PDF output.

Publish metadata record.

Using the GoGeo portal to access a ShareGeo Open spatial dataset

Practical 2

Conduct a search using the GoGeo portal.

Open and read the metadata record.

Access the ShareGeo Open spatial dataset via link provided in metadata record.