nerc data grid helen snaith and the ndg consortium …
Post on 21-Dec-2015
222 views
TRANSCRIPT
What is NDG?
• NDG provides the infrastructure which allows users to◦ Find data◦ Explore what is known about datasets
(including information about the observing or simulating tools)
◦ Access, manipulate and visualise data!
NDG Services
• The “NDG Discovery Service”◦ A database of discovery information◦ A web-site, to provide a portal to that database◦ A set of web-services, allowing NDG
consumers to exploit that database
• The “NDG Vocabulary Service”◦ Databases of environmental thesauri and
ontology tools to map between terms◦ Supports “machine assisted” browsing: eg,
search for rainfall and find datasets with precipitation
What Can NERC Data Grid Do For You?
• Depends on who you are!◦Data User - eg scientist looking for
data◦Data Provider - eg data centre, large
research group◦Developer - providing access tools or
using data sources
As a Data User?
• Discovery leads directly to data◦ Simple search page or “google-like” toolbar
gives direct access to data, not just the metadata
• Comprehensive coverage• Seamless access• Hides data format differences and
provides generic visualisation tools• Enhances interdisciplinary research
As a Data Provider?
• Provides a standards based metadata hierarchy◦ Exploit the NDG metadata structures to design a
metadata system from the ground up, or use them as an interoperability tool
• No need to run multiple security systems• Increases visibility and usage of datasets
◦ Inclusion in NDG does not prevent inclusion in other data discovery or access initiatives
• Minimise user support load
NDG Security
As a Developer?
• Exploit NDG web services• Exploit NDG modular
development• Exploit NDG metadata standards• NDG developments are non-
proprietary• NDG standards compliance
implies less future development
• Discovery◦ ISO standard
• MOLES - Metadata Objects for Links in Environmental Science◦Detailed metadata for browsing
• CSML – Climate Science Markup Language◦ For accessing “features” within data
MetaData Framework
What Can’t the NDG Do?
• NDG can’t generate the metadata!◦ Automated manipulation and searching relies on
comprehensive metadata, which require quality data management to create and maintain
◦ NDG does provide a clear framework for metadata requirements
• NDG doesn’t provide information services:- it provides data services!◦ NDG does provide a framework within which it is
possible to link data to derived information services:- research publications, reports, policy etc..
Competitors!How is NDG different from the
rest? • e2edm, End-to-End Data Management:
• Similar aims but using different technologies• NDG is more standards compliant
• Thredds:• Provides only catalogue services• Thredds is significantly more mature than NDG• Suitable for data providers with hierarchical data /
metadata
• OPeNDAP:• A protocol for accessing data• Limited support for access control• Requires the user to know exactly what the data are
NDG Status
• Approaching Beta◦ Discovery Gateway◦ Discovery Service◦ CSML V2◦ MOLES◦ Security Infrastructure◦ Vocabulary Service◦ MOLES browser
• Population (MOLES & CSML) by NDG partners
• Implementing & testing gateways, browsers & interfaces ndg.nerc.ac.uk
Trac wiki site