Collaborative Design of an Image Annotation Tool for Oceanographic Imaging SystemsAuthors: Joe Futrelle, Amber York @ Woods Hole Oceanographic Institution

Imaging FlowCytobot(Heidi Sosik et al)

HabCam(Scott Gallager et al)

SeaBED(Hanumat Singh et al)

Phytoplankton cytometry / imaging(e.g., estuarine sites)

Towed benthic surveys (e.g., NA continental shelf)

Autonomous benthic exploration (e.g., sub-Artic ice)

Imaging instrumentation and science teams

Web-based prototype annotation tool

Overview:Analyzing underwater imagery often requires labor-intensive manual annotation of large image sets, but few general-purpose tools exist to support this work. Typically, science teams build specialized “one-off” annotation tools unsuitable for reuse in other projects.

Approach:Build collaborative interdisciplinary teams with expertise in oceanographic imaging and informatics. Develop formal use cases representing consensus on needed capabilities and develop and test prototype systems.

5 15 25 35 45 55 65 75 85 95 105

115

125

135

145

155

165

175

185

195

205

215

225

235

0

100

200

300

400

500

600

700

800

Scallop Shell Heights (Northern Edge of Georges Bank)

Shell Height (mm)

Obs

erva

tions

Use case:• The ability to accurately determine the size, abundance and distribution of scallops is required for

responsible management practices that regulate the scalloping industry. Optical imaging systems have proven effective in maximizing survey effort and population estimate accuracy.

• The HabCam team needed the ability to identify and measure sea scallops and groundfish using a web-based annotation tool. The users also needed to specify the dominant and subdominant substrate in the image. The annotation tool also had to support to multiple users annotating images at the same time.

Tetherless World Constellation’s iterative development process:Focus on science outcomes allows exploratory design without being constrained by technical considerations; prototyping keeps technical development focused on science outcomes

Information model:In development workshops, all three teams, in collaboration with information scientists, reached consensus on an information model identifying conceptual relationships that the system needs to support. Implementation considerations are not included at this stage.

Subsequent prototypes include more concepts as neededFull information model contains all relevant concepts Initial prototype focuses on key subset

Remove concepts,Retain relationships

Add back concepts,Retain relationships

Activity diagram:Shows interaction patterns during system use

• HabCam imagery from the US Continental Shelf was annotated with scallop measurements using the web-based annotation tool prototype. Results of the scallop shell measurements are shown below.

• A total of 16 users made a total of 275,000 image annotatoins of scallops, groundfish, and substrate.

Use and science outcomes:

Peter Fox, RPI