The State of Image Annotations

Robert Casties Max Planck Institute for the History of Science, Berlin

What we do• Work with historical sources

• scanned books, manuscripts, notebooks

• digital photos

• scanned drawings and paintings

• digital texts

• data

Galileo Galileis MS72 "Notes on motion" (1998) http://www.mpiwg-berlin.mpg.de/Galileo_Prototype/INDEX.HTM

Musawwarat Graffiti Archive (2011) http://musawwaratgraffiti.mpiwg-berlin.mpg.de

digilib client-side point annotation (since 2002)

HyperImage annotated presentation (2009?) of James Gillray: „Political-Dreamings!” (1801)

Status quo

• lots of digital editions and projects

• nice and diverse features and tools

• re-inventing lots of wheels

• data silos

• solution: annotations and standards

Our vision• "Weaving a web of knowledge" (Jürgen Renn, 2003?)

• close and distant reading of all kinds of sources (works best with Open Access)

• create comments, relations, narratives

• share the results during the process or the result with collaborators or everybody

• bi-directional links (Xanadu) - collect research results related to sources creating a "semantic network" of sources

references to networks

Annotations as bi-directional links

• standardized (open, stable) access to source documents (image, text,…)

• standardized annotation data linking comment to source

• relevant segment needs to be referenced

• annotation server needs to be shared

• relation can be reversed

Annotation standards• Open Annotation Data Model (http://openannotation.org)

• general annotation data model

• last spec 1.0 of 2013

• W3C Web Annotation Working Group (since 2014)

• Abstract Annotation Data Model (spec)

• Data Model Vocabulary

• Data Model Serializations

• HTTP API (spec)

• Client-side API (FindText API spec)

Image annotation in W3C web annotation

• Target of annotation can be an image

• Segment of image can be specified by

• FragmentSelector with W3C media fragment (rectangle specified in pixel or percent)

• SVGSelector with SVG

Annotation standards• SharedCanvas (http://shared-canvas.org)

• data model for ”virtual books“

• uses Open Annotation

• last spec 1.0 in 2013

• IIIF (http://iiif.io)

• Image API standard

• Metadata API standard

• spec 2.1 released May 12, 2016

IIIF (International Image Interoperability Framework)

• ARTstor • Bayerische

Staatsbibliothek (Bavarian State Library)

• La Bibliothèque nationale de France

• Biblissima • British Library • British Museum • Brown University • Centre de

Recherche et de Restauration des Musées de France (C2RMF)

• Cogapp • Columbia

University • e-codices – Virtual

Manuscript Library of Switzerland

• Cornell University • DPLA • Digital Image

Archive of Medieval Music (DIAMM)

• Digirati Ltd • Europeana • The J. Paul Getty

Trust • Ghent University • Gottingen State

and University Library

• Harvard University • Indiana University • Internet Archive • Johns Hopkins

University • Klokan

Technologies • Leiden University

• MIT Libraries • National Gallery of

Art • National Library of

Austria • Nasjonalbiblioteke

t (National Library of Norway)

• National Library of Denmark

• National Library of Egypt

• National Library of Israel

• National Library of New Zealand

• National Library of Poland

• National Library of Scotland

• National Library of Serbia

• National Library of Wales

• New York University Libraries

• Ohio State University

• Oxford University (Bodleian Library)

• Princeton University Library

• Qatar National Library

• Sirma Group • St. Louis

University • Stanford University • TextGrid • text & bytes • University College

Dublin • University of

Edinburgh

• University of Hong Kong

• University of Illinois at Urbana-Champaign

• University of Notre Dame

• University of Pennsylvania

• University of Toronto

• Vatican Library • The Walters Art

Museum • Wellcome Trust • Wikipedia

(Wikimedia Foundation)

• World Digital Library

• Yale Center for British Art

• Yale University

{scheme}://{server}{/prefix}/{identifier}/{region}/{size}/{rotation}/{quality}.{format}

Image API Presentation API

SoftwareIIIF Shapes

Annotorious - (+)

HyperImage / Yenda - / + + / +

digilib + / - +

SemToNotes - +

Mirador + (+)

Diva + -

T-PEN (+) -

AnnoTate - -

Annotorious

Annotorious• http://annotorious.github.io

• Javascript client image annotation toolkit

• annotates images in HTML pages

• can integrate with Annotator.js

• can use scaling image server (OpenSeadragon)

• can do polygons

HyperImage

HyperImage / Yenda• http://hyperimage.ws

• “like hypertext for images”

• Java editor and server, Flash/HTML5 client

• free shapes in layers, linking between images, texts and layers

• used in a number of digital humanities projects

• Yenda (http://yenda.tools)

• new fully web architecture using IIIF

• under development

digilib

digilib• http://digilib.sourceforge.net

• scaling image server (Java)

• HTML + Javascript client

• view state and client-side annotations encoded in URL (since 2002)

• server-side annotations using Annotator.js (since 2012)

• polygon shapes (since 2015)

• IIIF API server

digilib

SemToNotes

SemToNotes

• https://hkikoeln.github.io/SemToNotes

• JavaScript API to develop browser-based image annotation tools

• topological queries on eXist and EXPath Geo Module backend

• under development

Mirador

Mirador• http://projectmirador.org/

• IIIF reference viewer

• Javascript client, can use any IIIF Presentation API feed

• deep zoom, metadata display, book reading, bookmarking, comparison view

• shape annotations (under development)

Diva.js

Diva.js• http://ddmal.github.io/diva.js

• Javascript client

• uses IIIF Image API

• deep zoom, fast scrolling through documents, synchronized information views

• annotation overlays

T-PEN

• http://t-pen.org

• Transcription for Paleographical and Editorial Notation

• Java server, online version free to use

• new version 3 with IIIF support under development

AnnoTate

AnnoTate

• https://anno.tate.org.uk , https://github.com/zooniverse/AnnoTate

• Zooniverse crowdsourcing project for Tate Archive holdings

• transcription of artist’s personal papers

• zoomable view, simple annotations

where are we now• IIIF image server standard incredibly successful

• Annotation use cases growing

• Annotation standards on the way…

• Annotation software development scattered

• Discovering and sharing annotations not yet there

ceterum censeo…

Standards for annotating image areas

• something between very simple (W3C Media Fragments)

"selector": { "type": „FragmentSelector", "conformsTo": "http://www.w3.org/TR/media-frags/", "value": "xywh=135,16,25,53" }

• and very complex (SVG)

"selector": { "type": ["SvgSelector", "Content"], "text": "<svg:svg> … <polygon xmlns="http://www.w3.org/2000/svg" points="5315,5639 5444,5529 5505,5593 5346,5691 5315,5639" /> … </svg:svg>“ }

Standards for annotating image areas

• Proposition

• use resolution independent coordinates (e.g. 0≤x≤1, fraction of image width)

• use GeoJSON / WKT

• points, lines, polygons

• only point coordinates

Annotations as "micro-publications"

• stable URLs for annotations

• provenance information

• copyright and license information

• versioning?

• authority / reputation?