![Page 1: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/1.jpg)
Lyndon Nixon, MODUL University
Annotating TV programming and
linking to related content on the Web
18 November 2014, Potsdam, Germany
http:/www.linkedtv.eu
![Page 2: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/2.jpg)
Structure
Introducing Linked Television
Challenges for TV Annotation
Challenges for TV Enrichment
LinkedTV Solutions
Summary
![Page 3: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/3.jpg)
What is Linked Television?
„... a true TV ecosystem must functionally integrate the apps with the
television programming service and content being offered to the consumer.“TVs can't be smart. Stop trying to make it happen. - WIRED Opinion, Gary Myer, Oct 2013
![Page 4: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/4.jpg)
What is Linked Television?
„Television won't last because people will soon get tired of staring at a plywood
box every night.“
- Darryl Zanuck, movie producer, 20th Century Fox, 1946.
![Page 5: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/5.jpg)
What is Linked Television?
Source: Tomorrow FOCUS Media „Smart-TV Effects 2014-1“
Source: Tomorrow FOCUS Media „Smart-TV Effects 2014-1“
![Page 6: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/6.jpg)
What is Linked Television?
ARD/ZDF Online Study 2014 „Second Screen: Parallelnutzung von
Fernsehen und Internet“
![Page 7: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/7.jpg)
08.11.2012
What is Linked Television?
37-52% of TV viewers are using a
companion device alongside the TV
program.
http://www.linkedtv.eu
Ever saw something
on TV and wanted to
know more about it,
but didn‘t even know
how to search for it?
![Page 8: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/8.jpg)
What is Linked Television?
Scenario 1:LinkedNews app
Easy access to
background to the news
Focus on key concepts of
interest in each item
hbbTV single screen and
dual screen apps
Scenario 2: LinkedCulture app
Easy access to
information on the art
objects discussed in the
program
Secondary screen-
orientation, cast TV
program to main screen
![Page 9: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/9.jpg)
Television annotations?
What we get?
![Page 10: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/10.jpg)
Television annotations?
What we need?
![Page 11: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/11.jpg)
Television enrichments?
What we get? What we want?
![Page 12: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/12.jpg)
dbpedia.org/resource/Jan_Sluyters
LinkedTV Solution
linkedtv.eu
Examples of Paintings by Jan Sluijters
“...schilderij van Jan Sluijters....”
Video object and
text detection
Personalisation
engine
Expansion of
related concepts
Linking to
related content
Connection to
concepts
… and their digital images
![Page 13: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/13.jpg)
Concept Detection
linkedtv.eu
“...schilderij van Jan Sluijters....”
Video object and
text detection
Scenario content specific video chapter detection (News Items, Art Objects)
Aggregated REST service for all analysis approaches in LinkedTV, integrated with the LinkedTV Platform
![Page 14: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/14.jpg)
Keywords and text fragments can be connected to uniquely identified, distinct named entities using online entity extraction services (NERD, http://nerd.eurecom.fr)
Concept Connections
linkedtv.eu
Connection to
concepts
dbpedia.org/resource/Jan_Sluyters
Introduction Silver Tea Jar
Nelleke van der Krogt Friesland
Hessel van Martena Silver (zilver)
Museum Martena Tea jar (theebus)
Franeker Emiel Aardewerk
Tin coffee decanter (art object)
Meesterteken
Zilversmid
Sondersitzung Blackout
BER Berlin Brandenburgairport
Berlin Fire Brigade
Hartmut Mehdorn Frank Henkel
Jochen Großmann Vattenfall Europe
![Page 15: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/15.jpg)
Linking to related content
linkedtv.eu
Digital images of Jan
Sluijters‘ paintings
Linking to
related content
• Information cards display relevant properties of annotated entities by type
• Links to related content are organised by dimensions
• Access to related video chapters on the LinkedTV Platform
• Specific enrichment services for • News articles• Cultural heritage objects
![Page 16: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/16.jpg)
Linked Culture
Tussen Kunst en Kitsch Video
Chapter segmentation + tagging „Art Objects“
Entity extraction incl. Art Object annotation
Related
TKK
chapters
Related
art objects
(Europeana)
Related
White List
media
TVEnricher (TKK configuration)
Editor Tool LinkedTV Player
Entity Proxy
![Page 17: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/17.jpg)
Art objects in TKK episodes
![Page 18: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/18.jpg)
Art object: semantic model
http://data.linkedtv.eu/object/avro/8a8187f2-3fc8-cb54-0140-7dccd76f0001/2138
A silver tea jar
RDF Is-a Container http://vocab.getty.edu/aat/300045611
CRM Consists-of Silver http://vocab.getty.edu/aat/300010975
VRA locationCreationSite Friesland http://www.geonames.org/2755812
DCT temporal Start: 1690, End: 1742
![Page 19: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/19.jpg)
Mapping to Europeana API
http://data.linkedtv.eu/object/avro/8a8187f2-3fc8-cb54-0140-7dccd76f0001/2138
A silver tea jar
RDF Is-a Container
CRM Consists-of Silver
VRA locationCreationSite Friesland
DCT temporal Start: 1690, End: 1742
what:(container+OR+houder+OR+b
ak+OR+tank+OR+blik)
proxy_dc_format:zilver
where:Friesland
YEAR:[1690+TO+1742]
Leeuwarden
![Page 20: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/20.jpg)
Enrichment results
![Page 21: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/21.jpg)
Linked Culture
https://vimeo.com/108891238
![Page 22: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/22.jpg)
Linked News
RBB Video
Chapter segmentation + tagging as „News Item“
Named Entity Expansion over news items
Related
news
items
Related
news articles
Related
White List
media
TVEnricher (RBB configuration)
Editor Tool LinkedTV Player
Entity Proxy
![Page 23: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/23.jpg)
Linked News
Asparagus
Harvest
Poland Beelitz
![Page 24: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/24.jpg)
Linked News
http://showcase.linkedtv.eu/showcase3.html
![Page 25: Annotating TV programming and linking to related content on the Web](https://reader033.vdocuments.site/reader033/viewer/2022060121/559452d91a28abc84f8b46f6/html5/thumbnails/25.jpg)
Summary
Rich annotation of AV content can enable added value services
but
• Consider the purpose of the annotation – this changes the
requirements on the annotation model and approach
• Consider the domain of the content to be annotated – every
domain has different requirements on how to annotate it
Enrichments are an example of added value on top of
annotations but
• Enrichment sources also need to be annotated
• Relevancy depends on the right choice of annotation entities
No perfect, automated approach: room for ongoing research!