akn and node data architecture

AKN and Node Data Architecture

Draft 7June 8, 2009

The Avian Knowledge Network

USER

Data Entry

Data Entry

Data Processing

Data Processing

Data Visualization

Data Visualization

Data Access

Data Access

Data Analysis

Data Analysis

Data Services (DiGIR)

Data Services (DiGIR)Node Data

Warehouses

Covariates

Metadata

Data PublishingData Publishing

Events Geography

Observations Taxonomies

Node Data Store (NodeDb)

Access C

on

trol

Access C

on

trol

= dataflow

= process or application

= data store

AKN Node Architecture

PARTNER

PROVIDER

Node

Node Architecture and NodeDbN

od

eD

b

CountsCounts HabitatHabitatBandingBanding

Data components

ProjectSpecificProjectSpecific

EcologySpecificEcologySpecific

Lists, counts, surveys banding records,

nest data, etc…

Avian Ecology Observations

segregated by source(A, B, C..)

Published Interfaces, Filters & TransformationsPublished Interfaces, Filters & Transformations

Data RetrievalData Retrieval Data AnalysisData Analysis

SamplingUnits

SamplingUnits TaxonomyTaxonomyProtocolsProtocols Project

MetadataProject

Metadata DictionariesDictionaries

NodeDbInfrasstructure UsersUsers Roles /

GroupsRoles /GroupsRightsRights LogsLogs Other

PlumbingOther

Plumbing

ListsLists BehaviorBehaviorNestsNests


EcologySpecificEcologySpecific

BandingBanding CountsCountsNestsNests


EcologySpecificEcologySpecificA B C

warehouse1 warehouse3warehouse2Node Data

Warehouses warehouse4

Data PublishingData Publishing

. . .

Data EntryData Entry Bulk Data ProcessBulk Data Process

Node Web SiteNode Web Site Node ApplicationsNode Applications

USER PARTNER

Node

Node

USERPARTNERPROVIDER

How NodeDb Codifies Observations

Sampling EventsSampling EventsWhen?When?

How?How?

What recorded?What recorded? Sampling ObservationsSampling Observations

Sampling UnitsSampling UnitsWhere?Where?

Who?Who?

has m

an

y

Project /Protocol/ResearcherProject /Protocol/Researcher

Why?Why?

has m

an

yh

as m

an

y

defined in

NodeDb Major Data Components

TaxonomyTaxonomy

Sampling EventsSampling Events

Sampling EventObservations


ProtocolsProtocols

ResearchersResearchers

ProjectsProjects Sampling UnitsSampling Units

participates in

collects

is located at contains

consisting of

definesmethodologyfor

uses

NodeDb: Projects Focus

ProjectsProjectsProject

ProjectId

ProjectName (AK1)ProjectDescription ProjectStatusCd ProjectStartYr ProjectEndYr RightsRestrictions CopyRightDt CopyRightExpirationDt OriginalProjectId (IE1)ProjectDatabase

ProjectProtocol

ProjectId (FK,FK)ProtocolId (FK,IE1)

has many

ProjectResearcher

ProjectId (FK)ResearcherId (FK,IE1)

ProjectLeaderInd ProjectResearcherStatusCd

has many

Protocol

ProtocolId

ProtocolName ProtocolDescriptionURL ProtocolTypeCd

defines research for

Researcher

ResearcherId

ResearcherStatusCd DoNotContactInd UnknownResearcherInd PRBOdbAdminInd ResearcherFirstName (IE1)ResearcherLastName ResearcherInitials EmailAddress (variousResearcherContactInfo)

participates in

SamplingUnitProject

SamplingUnitID (FK)ProjectID (FK,IE1)

has many

SamplingUnit

SamplingUnitID

FullName (IE1,AK1)ShortName (IE1,AK1)PointGeometry CoordinatesCollection ParentSamplingUnitID Description EstablishingProjectID (IE1,AK1)PublicInd

is site of

NodeDb: Researchers Focus

ResearchersResearchers

Researcher

ResearcherId

ResearcherStatusCd DoNotContactInd UnknownResearcherInd PRBOdbAdminInd ResearcherFirstName (IE1)ResearcherLastName ResearcherInitials EmailAddress (variousResearcherContactInfo)

ProjectResearcher

ProjectId (FK)ResearcherId (FK,IE1)

ProjectLeaderInd ProjectResearcherStatusCd

participates in

SamplingEventObservation

ProjectId SamplingUnitId SamplingEventTypeCd ProtocolId SamplingEventCounter ObservationCounter

ResearcherId (FK)(variousObservationDetails)

observes

Users

UserName (FK)

Authorization OpenId GroupNames UserState SkinId Comment

authenticated as

GroupPermissions

GroupName (FK)Application Permission

PermissionType

Groups

GroupName (FK)

Description

has rights to

has

NodeDb: Sampling Units Focus

Sampling UnitsSampling Units

SamplingUnit

SamplingUnitID

FullName (IE1,AK1)ShortName (IE1,AK1)PointGeometry CoordinatesCollection ParentSamplingUnitID Description EstablishingProjectID (IE1,AK1)PublicInd

SamplingUnitType

SamplingUnitTypeID

Name ParentRequired Description SamplingUnitTypeGeometryID (IE1)

SamplingUnitParentSamplingUnitType

SamplingUnitTypeID (FK)ParentSamplingUnitTypeID

SamplingUnitSamplingUnitType

SamplingUnitID (FK)SamplingUnitTypeID (FK,IE1)

has many

categorizes encompasses

encompasses

SamplingEvent

ProjectId SamplingUnitId (FK)SamplingEventTypeCd ProtocolId (FK)SamplingEventCounter

is site of

Nest Monitoring

Banding/Tagging

Bird Counts

Study Area

Point CountTransect

Banding NetCollection

Nest Plot

Point CountPoint Net

A

P

A

PL

TrapCollection

Trap PA

AA

Nest SubplotA

OtherNest P

Territory A

Nest Box

Burrow

P

P

Crevice P

NestP

Other BirdObservationPoint P

Index /Census Plot

SeabirdObservationBlind

At-seaTransect

P

P

LA

Area SearchPlot

A

Sea WatchObservationPoint

Shark WatchObservationPoint P

P

Sample Collection

Other Banding LocationAP

Seabird Diet Collection P

Blood Collection P

Egg Collection P

Specimens P

Predatory FishDiet Collection Point

P

Zooplankton Trawl L

Water Sample P

Weather Station P

Remote Sensing

Hydroacoustic Data L

Biologging

TDR PL

GLS PL

Satellite PLLeg

en

d= required to be part of

= optionally part of

PL

A

= point

= line

= area

NodeDb: Sampling Unit Type Hierarchy

NodeDb: Protocols Focus

ProtocolsProtocols

Protocol

ProtocolId

ProtocolName ProtocolDescriptionURL ProtocolTypeCd

ProjectProtocol

ProjectId (FK,FK)ProtocolId (FK,IE1)

defines research for

SamplingEvent


describes research design for

Project

ProjectId

has

ProtocolSamplingEventType

ProtocolId (FK,FK)SamplingEventTypeCd (FK,IE1)

is a

is constrained to

NodeDb: Sampling Events Focus


SamplingEvent


SamplingEvent_AreaSearch

ProjectId (FK)SamplingUnitId (FK,IE1)SamplingEventTypeCd (FK,IE2)ProtocolId (FK,IE3)SamplingEventCounter (FK)

SamplingEventGroupId SamplingEventDataStatusCd SamplingEventDt SamplingEventStartTm SamplingEventEndTm SamplingEventYear SamplingEventMonth SamplingEventDay TotalBirdsCounted SamplingEventRemarks

SamplingEvent_Banding



SamplingEvent_Count



SamplingEvent_Encounter



SamplingEvent_Nest



SamplingEvent_Behavior



SamplingEvent_DeployedNet

ProjectId (FK,IE4)SamplingUnitId (FK,IE1,IE4)SamplingEventTypeCd (FK,IE2,IE4)ProtocolId (FK,IE3,IE4)SamplingEventCounter (FK,IE4)DeployedNetCounter

DeploymentDt DeploymentStartTm DeploymentEndTm NetNumber TotalNumberNets NetHours DeploymentRemarks

SamplingEvent_Habitat


PrimaryHabitatCd PercentPrimaryHabitat SecondaryHabitatCd SurveyUnitAspect SurveyUnitSlope SurveyUnitAspectMeasurementCd WaterInd SnagsGT10cmCount SnagsLT10cmCount LogCount RiparianZoneWidth RiparianZoneLength ChannelWidth AdjacentLandUseCd

SamplingEvent_Diet



SamplingEvent_Weather

ProjectId (FK)SamplingUnitId (FK,IE1)SamplingEventTypeCd (FK)ProtocolId (FK)SamplingEventCounter (FK)

SamplingEventGroupId SamplingEventDataStatusCd SamplingEventDt SamplingEventStartTm SamplingEventEndTm SamplingEventYear SamplingEventMonth SamplingEventDay DiurnalLowTemperature DiurnalHighTemperature SamplingEventRemarks

SamplingEventTypeCd

NodeDb: Sampling Observation Focus – Point Counts




SamplingEvent_Count



SamplingEventObservation_Count

ProjectId (FK,FK)SamplingUnitId (FK,IE5)SamplingEventTypeCd (FK)ProtocolId (FK,FK,FK,FK,FK,IE1,IE3,IE2)SamplingEventCounter (FK)ObservationCounter

BirdCount DetectionCueId (FK,IE1)DistanceBinId (FK,IE2)TimeBinId (FK,IE3)BreedingStatusCd SingingInd CompassBearing ClusterInd ObservationTm ResearcherId (FK)FieldBirdId (IE4)ObservationRemarks

produces

Project

ProjectId

Protocol

ProtocolId

conducts

Researcher

ResearcherId

has

defines

ProtocolDetectionCue

ProtocolId (FK)DetectionCueId

generates

has

ProtocolTimeBin

ProtocolId (FK)TimeBinId

has

containsProtocolDistanceBin

ProtocolId (FK)DistanceBinId

has

is observer distance from

NodeDb: Taxonomy Focus

TaxonomyTaxonomy

Taxon

TaxonId (FK,FK,FK,IE1)

TaxonName TaxonKingdomId TaxonRankId EncompassingTaxonId (IE1)ITIS_TaxonSerialNumber AOU_BirdSpeciesId AOU_PhylogeneticSequenceNo USDA_PlantSpeciesId BandSizeNote ReclassifiedTaxonId ReclassificationDt ScientificAuthority IdentificationQualifier CommonName (FK)Language (FK)

TaxonCommonName

TaxonId CommonName Language

TaxonDimension

TaxonId

EncompassingTaxonId TaxonRankId Kingdom Phylum Subphylum Class Order Suborder Family Subfamily Genus Species Subspecies CommonName

encompasses

has

has

NodeDb: Data Dictionary

DictionaryTable

TableId

SamplingEventTypeCd (IE1)TableName (IE2)TablePurpose ViewInd

DictionaryTableColumn

TableId (FK)ColumnId (FK,IE1)

ColumnCounter RequiredInd AutoIncrementInd PrimaryKeyInd ForeignKeyInd VirtualInd

contains

DictionaryColumn

ColumnId

ColumnName ColumnDefinition ColumnDataTypeCd ColumnUnitsCd ColumnLength DecimalPrecision RoleNameInd BaseColumnId (IE1)CodeColumnInd CodeSetId (IE2)

DictionaryCode

CodeSetId

CodeSetName CodeSetDescription ColumnId (FK)

DictionaryCodeValue

CodeSetId (FK)CodeValue

CodeValueDescription

has

is in

has

AnyNodeDbTable

SomeDataCd

table defined in

column(s) defined in

column inventory in

set of allowed values in

allowed values in

NodeDb: Data Dictionary ExampleSamplingEvent_AreaSearch



DictionaryColumn

ColumnId TableName CodeSetId

221 SamplingEventDataStatusCd 23

DictionaryCode

CodeSetID CodeSetName 23 SamplingEventDataStatus

DictionaryCodeValueCodeSetId CodeSetValue Description

23 RAW Raw Input - Unreviewed23 CLEAN Biologist Reviewed

23 APPROVED Available for analysis - internal PRBO/contributing institution use only

23 AVAILABLE Level 1 Stored in the AKN’s primary data warehouse, with the option that their data are not exposed to the public at all.

23 AVAILABLE Level 2 Data can be used in certain publicly available, predefined visualizations (i.e. maps and graphs), but direct access to the data is restricted.

23 AVAILABLE Level 3Data can be used in publicly available, predefined visualizations (i.e. maps and graphs). Additionally, data is made available to existing bioinformatic efforts (GBIF and ORNIS).

23 AVAILABLE Level 4

Data are used in publicly available, predefined visualizations (i.e. maps and graphs), and are available to existing bioinformatic efforts. Additionally, the complete BMDE data set is available upon request from the original data provider.

23 AVAILABLE Level 5Data are used in publicly available, predefined visualizations (i.e. maps and graphs), and are available to existing bioinformatic efforts. Additionally, the complete BMDE data set is available for download directly via the Internet.

23 RESTRICTED Restricted access - clean data but must contact PRBO or contributing institution project leader or division director before using

akn and node data architecture

Documents

data dictionarynodedb

nest data

data of record

node data architecturedraft

node data warehouses

data provider roles

key bodies of data

node partner