r&d at sound and vision
TRANSCRIPT
R&D AT THE
NETHERLANDS INSTITUTE
FOR SOUND AND VISION
Bouke Huurnink
Lotte Belice Baltussen
5 April 2013
THE ARCHIVE
Nederlands Instituut voor Beeld en Geluid
• Foto loopbrug / canyon
• Foto restaurant
• Detail foto’s
• Opening bea
HISTORY BEELD EN GELUID
• Founded in 1996
• Archives from 4 organisations:
• Public Broadcasting Archive
• Netherlands Government Information Service
• Foundation for Film and Science
• Broadcaster Museum
MISSION STATEMENT
• The best audiovisual archive in the digital domain
• 5 strategic priorities– enriching, connecting, and giving meaning to its
collections
– bringing people into contact with audiovisual heritage
– collecting material from the past, present, and future
– digital storage and access, with manual and automatic annotation
– actively contributing to durability and accessibility of the NL Audiovisual Collection
USER GROUPS
• Education and research
• Creative industry
• Heritage institutes
• Consumers
maart 2012
• 70% of Dutch AV heritage
• 800 000 hrs, including
• 250 000 hrs television
• 150 000 hrs radio
• 300 000 hrs music
• 50 000 hrs documentary, film, commercials etc
• 2 million pictures
• 20 000 objects
maart 2012
SIZE OF THE COLLECTION
maart 2012
• film as of 1898
• commercials as of 1920
• polygoon (cinema journals)1922 – 1980
• radio as of 1934
• television as of 1951
• news and sports programs
• Royal house
• national music collection (cd)
• amateur film
• corporate films
EXAMPLE COLLECTION CONTENT
RESEARCH &
DEVELOPMENT
FOCUS AREAS
• Access– Annotation
– Retrieval
– Contextualization
• Exchange– Interoperability between collections
– Linking collection to semantic web
• Knowledge about end-users– Interviews
– Experiments
– Log studies
FOCUS AREAS
• Access– Annotation
– Retrieval
– Contextualization
• Exchange– Interoperability between collections
– Linking collection to semantic web
• Knowledge about end-users– Interviews
– Experiments
– Log studies
TODAY
ANNOTATION
ANNOTATION TODAY
• INSERT PICTURE OF IMMIX
– +-15 archivists, with changing role
– Speech recognition and radio streaming in
ANNOTATION RESEARCH
• Automatic annotation– Computer vision
– Audio analysis
• Help from the crowds– Crowd-sourcing
– Interactive games
• Secondary sources of descriptions– Subtitles
• Contextual material
ANNOTATION RESEARCH
• Automatic annotation– Computer vision
– Audio analysis
• Help from the crowds– Crowd-sourcing
– Interactive games
• Secondary sources of descriptions– Subtitles
• Contextual material
RETRIEVAL AND
ACCESS
RETRIEVAL REALITY
RETRIEVAL RESEARCH
• Search with all sorts of annotations
– Small scale, diverse
– Hard to integrate with production
• New forms of access
– AVResearcher (MeRDES)
– AXES
– TROVE
– SocialZap
– LinkedTV
– …
CONTEXT
CONTEXT IS...
CONTEXT IS....EVERYTHING?
All information that can be used to interpret and understand
the production, publication and reception of audiovisual
heritage from multiple standpoints.
metadata ≠ context source
CONTEXTUALISATION:
AN AV-ARCHIVE DEFINITION
5-4-2013
• What do you want to contextualise?
• What can be used as context?
• Which categories of context do you have?
• Which target users need which type(s) of context?
• How can you connect and represent context sources?
CONTEXTUALISATION:
IMPORTANT QUESTIONS
FROM SEPARATE SILOS...
5-4-2013
...TO CONTEXT: MUTUALLY
CONNECTED COLLECTIONS...
5-4-2013
Connecting collections:
topics, people, genres, etc
Ca
talo
gu
e
Ph
oto
s
B&
G W
iki
Pro
gra
mm
e
gu
ide
s
...AND NEW POSSIBLITIIES FOR
INTERPRETING AV-HERITAGE
NL-HaNA, ANEFO / neg. stroken, 1945-1989, 2.24.01.05, bestandeelnummer 929.8771
CC BY-SA
EXAMPLE OF WHAT DO WE WANT TO
DO AND PREVENT
EXAMPLE: COMERDA
5-4-2013
• BRIDGE project: ISLA, Centre for Television in Transition (UU), S&V
• Goal: linking diverse, relevant collections to aid tv-researchers
VIDEO HYPERLINKING
Method to link a video to other multimedia sources
Applications:
Search: connections between sources based on a query
(clustering, storytelling)
Detail-on-demand: zooming in on specific elements
through linked information sources (contextualisering)
12 februari 2013
VIDEO HYPERLINKING:
AXES VS LINKEDTV
12 februari 2013
12 februari 2013
VIDEO HYPERLINKING:
AXES VS LINKEDTV
Concept: Jan Sluijters (Painter)
In DBpedia & VIAF
Related items
Further links
• Styles
(Expressionism, Cu
bism, Fauvism)
• Period
(contemporaries)
Concept relations:
VIDEO HYPERLINKING:
LINKEDTV ENRICHMENT PROCESS
THANKS!
@lottebelice | [email protected]
Special thanks
• Marc Bron, Anders Bouwer, Maarten de Rijke (ILPS, UvA)
• Jasmijn van Gorp, Sonja de Leeuw (UU)
• Johan Oomen, Roeland Ordelman, Jaap Blom (B&G)
WORKING TOGETHER
ACCESS
• Sound and Vision Labs
– Place to put prototypes and
software
– Public/private access
– Data exchange
– Under construction
• We can put you in touch with
users and broadcasters
DATA
• What do we have available?
– Catalog collection (complete)
– Keyframes
– MPG1
– Query logs (searches, clicks, purchases)
– Other collections
EXAMPLE RESEARCH IDEAS
• Effective analysis of tweets about TV
• Large-scale video analysis
• Search over heterogeneous noisy metadata
• Online learning
• Incorporating thesaurus in search (suggestion)
• Binary concept detection (tagging)
WANT TO KNOW MORE?
• Running research
• New ideas
• Data access
• Potential collaboration
• …?
Get in touch!
Find this presentation @ http://www.slideshare.net/boukehuurnink1
NL-HaNA, ANEFO / neg. stroken, 1945-
1989, 2.24.01.05, bestandeelnummer 933-2365 CC BY-SA