publishing hkh biodiversity data globally technical session ii

International Centre for Integrated Mountain Development

Kathmandu, Nepal

Technical session IIDay 1

Publishing HKH Biodiversity Data Globally

Deependra Tandukar, Suman Jaiswal, Bandana Shakya23-24 August 2012

Outline

• Using HKH-BIF• Processes for publishing the data

– Preparing the data– Uploading data– Filling Metadata– Mapping – Publishing

Platform

DataHKH-BIF GBIF

• for easy and efficient access and sharing of regional biodiversity information

• provisions for partners to publish and share their data

• for collaborative work e.g. data paper publication

Processes for publishing data

• Prepare your data– Data resource– Templates– Processing with spreadsheet processor

• Upload your data• Fill in metadata• Map the fields with Darwin core• Publish

– Make it public

Step 1

Data Preparation

Collection of data records Information about the dataset

Step 1: Data preparation

Data resource

Data resource

Dataset Metadata+=

Occurrence data Metadata+

Checklist Metadata+

Step 1: Data preparation

Templates

• Standardized templates (based on Darwin core)

• Glossary (location, scientific name, family, etc.)

• Metadata• Occurrence data• Checklists

– Checklist 1– Checklist 2– Checklist 3

Step 1: Data preparation

Template types

Download from: http://tools.gbif.org/spreadsheet-processor

• Readme sheet

Step 1: Data preparation

Templates: Metadata

Readme sheet gives the general instruction on what and how to fill the metadata sheet

• Metadata sheet

Step 1: Data preparation

Templates: MetadataMetadata sheet has number of fields for information to feed in, and some of the fields marked by * are mandatory

• Readme sheet• Metadata sheet• Occurrence sheet

Step 1: Data preparation

Templates: Occurrence

45 fields/columns in total

This sheet is used to record species occurrence data and is suitable for recording basic natural history collections specimen records or species observations.

• Readme sheet• Metadata sheet• Classification sheet

Step 1: Data preparation

Templates: Checklist1

This sheet is used to record the species lists where you are not sure of higher classification

• Readme sheet• Metadata sheet• Classification sheet

Step 1: Data preparation

Templates: Checklist2

This sheet is used to record the species lists where you are not limited to strict Linnaean classification

• Readme sheet• Metadata sheet• Classification sheet

Step 1: Data preparation

Templates: Checklist3

This template is used to record species checklists in a simple columnar format that stores the basic taxonomic data in the columns (Kingdom, Phylum, Subphylum, Class, Order, Family, Genus, Species, etc.).

• Use checklist 3 for data preparation• Enter mandatory fields in metadata• Fill in classification sheet

Step 1: Data preparation

Hands-on session

• Enter fields in metadata

Step 1: Data preparation

Metadata

• Classification field

Step 1: Data preparation

Checklist data fields

Kingdom Animalia

Phylum Vertebrata

Class Amphibia

Order Anura

Family Ranidae

Genus Amolops

Scientific name Amolops ricketti

Countries China

Step 1: Data preparation

Process the template

Spreadsheet processor

Archive file

• Go to spreadsheet processor(http://tools.gbif.org/spreadsheet-processor/)

Step 1: Data preparation

Process the template

• Click on ‘Choose File’ and select your prepared checklist template, it gets uploaded

• Click on ‘Process File’

Step 1: Data preparation

Process checklist template

• Download the processed archive by clicking on ‘Click to Download Archive’

• Save on your desktop (filename will be something like 503117a7e582a.zip>

Step 1: Data preparation

Process checklist template

Step 2

Upload your data

• http://192.168.10.19/hkh-bif • Enter your username and password• Click on ‘login’

Step 2: Upload your data

Go to HKH-BIF

• Click on ‘Manage Resources’• Give a short name for your data/resource• Click on ‘Choose file’ and select the ‘Archive’ that was

downloaded earlier • Press on ‘Create’

Step 2: Upload your data

Create New Resource

Step 2: Upload your data

New resource created

4

Step 2: Upload your data

Source data

Step 2: Upload your data

Source data

Step 3

Metadata

Step 3: Metadata

Over view page

Step 3: Metadata

Basic metadata

Step 3: Metadata

Geographic coverage

• Geographic area covered by the resource

Step 3: Metadata

Taxonomic coverage

• Information about one of more groups of taxa covered by the resource

Step 3: Metadata

Keywords

• One or more sets of keywords about the resource.

Step 3: Metadata

Associated parties

• Information about one or more people or organizations associated with the resource those not already covered on the Basic Metadata page

Step 3: Metadata

Project data

• Information about a project under which the data in the resource were produced/generated

Step 3: Metadata

Sampling methods

• Information about sampling methods in general, and about specific sampling steps

Step 3: Metadata

Citations

• Information about how to cite the resource as well as a bibliography of citations related to the data set, such as publications that were used in or resulted from the production of the data

Step 3: Metadata

Collection data

• Information about the physical natural history collection associated with the resource (if any) as well as lists of types objects in the collection, called Curatorial Units

Step 3: Metadata

External links

• Links to the home page for the resource, links to the resource in alternate forms (database, spreadsheets, etc.)

Step 3: Metadata

Additional metadata

• Extra information e.g. logo, copyright information, etc.

Step 4

Mapping

Step 4: Mapping

Mapping

Step 4: Mapping

MappingD

arw

in c

ore

term

s

Source data

fields

Step 5

Publishing

Step 5: Publishing your data

Publish

Step 5: Publishing your data

Publish

• Click on publish to publish your data

• generates a new dwca.zip file containing:• RTF file (draft of data paper)• eml.xml file describing the metadata of this resource• meta.xml file describing the mappings of DwC terms and

source data columns• Source data file

Step 5: Publishing your data

Visible

• Make it public for sharing

Step 5: Publishing your data

Visibility

Step 5: Publishing your data

Visibility

Technical session IIDay 2 (Hands-on)

Publishing non-template data

Xls to csv format

• Open your excel file

• Save as ‘comma delimited csv’ format in your desktop, give a name ‘my_first_data’

• http://192.168.10.19:8080/hkh-bif• Enter your username and password• Click on ‘login’

Go to HKH-BIF

• Click on ‘Manage Resources’• Give a short name for your data/resource• Press on ‘Create’

Create New Resource

my_first_data

• Click on ‘choose file’ under source file and select your .csv file

Create new resource

Create new resource

New resource

Mapping

Mapping

• Select ‘Darwin Core Mappings’ type from the drop down – in our case it is ‘Darwin Core Taxon’

• Then click on ‘Add’

Mapping

• Auto mapped 1 column

Mapping

• At the bottom of the page, a list of the unmapped fields are displayed

Mapping

• Let’s start mapping now• Auto mapped Kingdom = Kingdom• Map ScientificName = Sname, class = C according to our data

Dar

win

cor

e te

rms

Source data

fields

Filter

Exclude the records that do not match the criteria using filter

Mapping

• Once all the known fields are mapped, click on save button

• On the top of the page, click on ‘back to resource over view page’

Metadata

• Now enter the metadata

Metadata

Publish

• Click on publish to publish your data

Make it visible

• Make it public for sharing

Useful resources• Getting Started: Overview of Data Publishing through GBIF network

http://links.gbif.org/getting_started_publishing_en_v1 • Integrated Publishing Toolkit

http://links.gbif.org/ipt_user_manual • GBIF Spreadsheet Templates User Guide

http://links.gbif.org/dwca_spreadsheet_processor_guide • GBIF Metadata Profile How-To Guide

http://links.gbif.org/gbif_metadata_profile_how-to_en_v1 • GBIF Metadata Profile Reference Guide

http://links.gbif.org/gbif_metadata_profile_guide_en_v1 • Darwin Core Archive How-to Guide

http://links.gbif.org/gbif_dwc-a_how_to_guide_en_v1 • Publishing and Registering Your Data with GBIF

http://links.gbif.org/dwc-a_publishing_guide_en_v1 • Spreadsheet Processor and Excel Temp

http://tools.gbif.org/spreadsheet-processor/• Darwin Core Terms

http://code.google.com/p/darwincore/ • IPT Download link

http://code.google.com/p/gbif-providertoolkit/

Thank you