accurator: acquisition of qualitative annotations from niche communities
DESCRIPTION
MCN2014 Panel presentation on Accurator: Acquisition of Qualitative Annotations from Niche Communities.TRANSCRIPT
Henrike Hövelmann
AccuratorAcquisition of Qualitative Annotations
from Niche Communities
Chris Dijkshoorn
Collaboration between museum and research institutes
Nichesourcing of annotations
Accurator system sneak preview
Datasets used in Accurator
Evaluation
Future work
Accurator
Collection
‣ Collection of ~1,000,000 objects ‣ Print collection of ~550.000 prints ‣ Artworks on display ~8.000
Online Collection
‣ Accessible through API ‣ Published as Linked Data ‣ 548,785 object records ‣ 424,098 prints ‣ 212,517 works are digitised
Rijksmuseum Amsterdam
Make artworks accessible online
Professional catalogers and photographers
Process
‣ Register artwork ‣ Provide annotations ‣ Digitise artwork ‣ Publish online
~40,000 new object records a year
Print Room Online Project
Focus on art-historical information
Occasional lack of expertise regarding subject matter annotations
This print is described as:
‣ “Bird with blue head” ‣ “Branch with red leaves”
Annotating Artworks
Socially-Enriched Access to Linked Cultural Media Diverse collaboration
‣ 4 Research institutes ‣ 5 Cultural heritage institutions ‣ 5 Industry partners
4 PhDs are working on Accurator
‣ Crowdsourcing the annotation process
‣ Finding niche groups with required expertise
research project
State of the art
‣ Steve.museum ‣ YourPaintings
Rijksmuseum allowed tagging on website
Tagging
Use nichesourcing to gather annotations
‣ Find niche groups with required expertise
‣ Use the knowledge of expert laymen
‣ Cluster tasks based on topic
Beyond crowdsourcing
TrustClusterSearch
User Modelling
Crowd
Cultural Heritage
Collection
Annotations
NicheExperts
NicheExperts
NicheExperts
Accurator System
NicheExpert
Japanese
Redstart
Nichesourcing Annotations
Accurator screencast
Base choice on gaps in collection
Started with two topics
‣ Castles ‣ Flowers
Contacted multiple groups, but low response
Change the focus
Which niche group to address?
Focus on birds
Specimen Collection
‣ Songbirds 3.210 specimen
Print Collection
‣ Koekkoek 406 prints
Thesaurus
‣ IOC World Bird List 34,197 concepts
Naturalis biodiversity data
Open Annotation Data Model for storing information
Semantic support for annotators
‣ Autocompletion from structured vocabularies
‣ IOC world bird list in Accurator
Semantic Annotation
Semantic Annotation - Data model
rdf:value
dc:title skos:prefLabel
hdl:504055 ico: 25F34
"Eagle owl in magnolia"@en
"owls"@en
edm:aggregated
CHOagg:COLLEC
T.504055
edm:isShownBy
dc:subject
ann:id_6111fedf002e9
oa:hasTarget
birds:species-bubo_bubo
oa:hasBody
genid:target_44478b9e
oa:hasTarget
oa:hasSource
#xywh=percent:19,40,39,27
ann:h"0.27"^^xsd:float
ann:w "0.38"^^xsd:float
ann:x "0.18"^^xsd:floatann:y
"0.40"^^xsd:float
Compare provided annotations to correct answers
‣ Define correct answer ‣ Collect “ground truth”
Experts tend to disagree
Obtained a set of Naturalis bird prints with scientific names
Evaluation results
“The frog stands for the rejected lover”
“Sexy ladies”
“Women”
“The frog indicates the profession of the women”
Run experiment with ground truth
Incorporate automatic trust assessment in Accurator
Show effects of annotating by leveraging it for collection integration
Future work
Possibilities for integration - Rijksmuseum Website
PrintKono Bairei
Possibilities for integration - Dutch Species
Chris Dijkshoorn [email protected] http://chrisdijkshoorn.nl
Accurator Acquisition of Qualitative Annotations from Niche Communities
Henrike Hövelmann [email protected]