embedded metadata in mxf - ebucore
DESCRIPTION
Presentation given by Limecraft at the 2012 EDM Metadata Hands-on Workshop organised by the EBU. Summary: The Material Exchange Format (MXF) has by now become the de facto standard for file-based media exchanges. Even though MXF is incredibly rich in features and flexibility, handling MXF files, and in particular, the metadata that describes the essence within them is still a complex matter as there are many ways in which metadata can be associated with MXF (e.g., as side-car information, as dark metadata or as properly encoded and standardized metadata sets, …). In this presentation we will discuss and demonstrate a recent initiative, supported by EBU, that aims to make the association and processing of MXF metadata easier and more accessible for application developers. In particular, an open source MXF SDK is demonstrated that is able to serialize and deserialize EBUCore metadata natively in MXF audiovisual containers.TRANSCRIPT
Embedded metadata in MXFDieter Van Rijsselbergen
2012 EDM Metadata Workshop
EBU, Geneve – 20/06/2012
Who are we?
Contents
• Metadata exchanges?• Metadata in MXF• How to put metadata in MXF?• Putting EBUCore into MXF• EBU MXF SDK • EBU MXF SDK – Demo• EBU MXF SDK – Improvements & Community
Metadata exchanges?
• When exchanging audiovisual essence…– We must also exchange its identity,
its context, its creators,….– i.e., the semantics
behind the essence– in the form of
metadata
Metadata
Metadata exchanges?
• 2 ways to exchange metadata during the production and broadcasting process
• In-band /w essence– Tightly coupled and
doesn’t get lost– Often optimized for
essence storage
• Out-of-band– Centrally stored– Optimized for
metadata (search, augmentation, …)
Metadata exchanges?
• Out-of-band is interesting within the facility– Metadata is quickly accessible– Metadata can be indexed, annotated, …
• Embedded metadata is interesting in B2B– Side-car metadata doesn’t get lost,
doesn’t need a dedicated channel, …– E.g., news exchange scenarios• No more need for NewsML XML,
just serialize it directly into the MXF file
Contents
• Metadata exchanges?• Metadata in MXF• How to put metadata in MXF?• Putting EBUCore into MXF• EBU MXF SDK • EBU MXF SDK – Demo• EBU MXF SDK – Improvements & Community
Essence Exchange: MXF
• The Material Exchange Format is here to stay• Standardized by SMPTE and EBU• One of the most versatile, flexible and
powerful container formats defined– Not sure there’s anything it can’t do…– But makes it complex and harder to use
Metadata in MXF
• MXF has become the de-facto standard for audiovisual material production and exchange
• Fortunately, it has significant support for embedded metadata
Metadata in MXF
• 2 kinds of metadata in MXF– Structural metadata• Defines the structure of the file
– Track layout– Describes the essence,
which tracks, which codecs and parameters per track
– References to other/older files
Metadata in MXF
• 2 kinds of metadata in MXF– Descriptive metadata• Exchange of essence semantics
– e.g., a continuity report
• At its simplest, a means of identification– “let the descriptive metadata reference
identifiers I use internally in my organisation”
• This is the kinds we’re interested in!
Descriptive Metadata in MXF
• Standardized MXF descriptive metadata exists– Descriptive Metadata Scheme-1 (DMS-1)• Standardized as SMPTE 380M• Production characteristics, rights information,
annotations, person and entity identification, …• Aligned with MPEG-7, TV-Anytime
– Is DMS-1 being used for real?• Perhaps due to the fact that there is no DMS-1 notion
outside of the MXF ecosystem?– Hard to obtain a viable and complete mapping
Descriptive Metadata in MXF
• Which other descriptive metadata are interesting to consider?– General identification and description: EBUCore– Format-specific descriptions: IPTC NewsML-G2– Acquisition metadata (state of the recording
device): EBU Tech 3349– Feature detection results: AVDP profile of MPEG-7
Contents
• Metadata exchanges?• Metadata in MXF• Putting EBUCore into MXF• How to put metadata in MXF?• EBU MXF SDK • EBU MXF SDK – Demo• EBU MXF SDK – Improvements & Community
How to put metadata in MXF?
• Structure of MXF files is KLV elements– Key to identify the kind of element– Length of the value that follows– Value of the element
– KLV is for both essence and metadata• Typically, for essence a single frame per KLV
Key Length Value
16 bytes16 bytes BER-encoded length
BER-encoded length Length lengthLength length
......
How to put metadata in MXF?
How to put metadata in MXF?
• Use KLV to embed metadata<?xml version="1.0" encoding="UTF-8"?><ebuCoreMain xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:dc=http://purl.org/dc/elements/1.1/ xmlns="urn:ebu:metadata-schema:ebuCore_2011" xmlns:default=http://iptc.org/std/nar/2006-10-01/ xmlns:ebu=http://ebu.org/nar-extensions/ version="1.3" dateLastModified="2011-01-31"> <coreMetadata> <title> <dc:title xml:lang="fr-can">Berlin Wall 50th</dc:title> </title> <alternativeTitle typeLabel="headline"> <dc:title>Berlin Wall 50th</dc:title> </alternativeTitle> <creator entityId="EBUNMSSRVCodeSource:TBA"/> <subject> <dc:subject/> <subjectCode>EBUNMSCategoryCode:EXCH</subjectCode> </subject> <subject> <dc:subject>In or about Europe and EU</dc:subject> <subjectCode>EBUNMSWebcatCode:EUROPE</subjectCode> </subject> <description typeLabel="pubStatus"> <dc:description>NMSStatusCode:TBC</dc:description> </description> <description typeLink="EBUNMSDescription:dopesheet"> <dc:description> The Berlin Wall (German: Berliner Mauer) was a barrier constructed by the German Democratic Republic (GDR, East Germany) starting August 13, 1961, that completely cut off West Berlin from surrounding East Germany…
Key Length Value
16 bytes16 bytes BER-encoded length
BER-encoded length Length lengthLength length
......
Dump the document as a whole into the KLV value?Not the best approach…
How to put metadata in MXF?
• On top of this raw KLV structure is a conceptual model– Elements can be grouped into
logical metadata sets identified by a single key, i.e., “classes”
– Logical metadata sets can be reconstructed by means of a dictionary
<Identification base="InterchangeObject" detail="Identification set" type="localSet" baseline="yes" key="06 0e 2b 34 02 53 01 01 0d 01 01 01 01 01 30 00"> <ThisGenerationUID use="required" type="UUID" key="3c 09" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 01 00 00 00"/> <CompanyName use="required" type="UTF16String" key="3c 01" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 02 01 00 00"/> <ProductName use="required" type="UTF16String" key="3c 02" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 03 01 00 00"/> <ProductVersion use="optional" type="ProductVersionType" key="3c 03" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 04 00 00 00"/> <VersionString use="required" type="UTF16String" key="3c 04" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 05 01 00 00"/> <ProductUID use="required" type="AUID" key="3c 05" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 07 00 00 00"/> <ModificationDate detail use="required" type="Timestamp" key="3c 06" globalKey="06 0e 2b 34 01 01 01 02 07 02 01 10 02 03 00 00"/> <ToolkitVersion use="optional" type="ProductVersionType" key="3c 07" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 0a 00 00 00"/> <Platform use="optional" type="UTF16String" key="3c 08" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 06 01 00 00"/></Identification>
How to put metadata in MXF?
• On top of this raw KLV structure MXF definesa conceptual model– Metadata sets can reference other sets– Metadata set can be
constructed as trees or graphs
How to put metadata in MXF?
• Don’t just throw the metadata in there– Link the descriptive metadata
to the structural metadata
The DM Framework hooks custom descriptive
metadata to (a part of ) the MXF structural timeline
How to put metadata in MXF?
• KLV encoding + conceptual data model + timeline model
• MXF has a proper way of structuringmetadata natively– Whenever we can, we should use it!
Contents
• Metadata exchanges?• Material Exchange Format• How to put metadata in MXF?• Putting EBUCore into MXF• EBU MXF SDK • EBU MXF SDK – Demo• EBU MXF SDK – Improvements & Community
Putting EBUCore into MXF
• A case study in embedding MXF metadata and with practical merit– Embed an EBUCore XML document in MXF file
using a proper KLV mapping and encoding– Build software that handles
MXF container and its embedded EBUCore metadata as a whole
ITM504483_EBUcore.
XML
EBU MXF SDK
• Initiative by EBU to provide community with reliable and open-source software to manipulate MXF files– Handle metadata– Handle essence and work on
conformity of MXF files– Available as free and open source software• First repository: https://github.com/Limecraft/bmx
EBU MXF SDK
• Based on BBC librarieslibMXF, libMXF++ and bmx– Reference implementations for MXF app. specs.• AMWA AS-02 (Versioning) and • AMWA AS-11 (MXF for Contribution)
– Written in C/C++• Milestone #1– Mux and de-mux EBUCore metadata
in MXF containers
EBU MXF SDK – Demuxing/Reading
CompressedVideo
Frames
CompressedVideo
Frames
CompressedAudio
Samples
CompressedAudio
Samples
FFmpeg(libavcodec)
MXF SDK
ReadPartitions / Structural Metadata
ReadDescriptiveMetadata
ReadIndices
Read Essence
SMPTE 434 XML Representation
Metadata(EBU Core)
MXF AnalysisReport
CompressedAudio
Samples
CompressedVideo
FramesDecodeVideo
DecodeAudio
AudioSamples
VideoFrames
EBU Player/GUI
Playback
Metadata Display/Analysis
(Files)
EBU MXF SDK – Wrapping/EBUCore
CompressedVideo
Frames
CompressedVideo
Frames
CompressedAudio
Samples
CompressedAudio
Samples
MXF SDK
EBUCore Parser
MUX
Essence Processor
Schema-basedEBUCore
object model
Structural Metadata
Essence
EBUCore Document
KLV-basedEBUCore
object model
<<Other metadata parsers>>
OtherKLV-basedmetadata
object models
EBUCoreXML Schema
XSD Code Generator
EBUCoreKLV Dictionary
gen_classesCode Generator
Bi-directional mapping
EBU MXF SDK – Demo
MXF Dump Report
MXF SDK
raw2bmxPCM Audioa0.raw
MPEG-2 LGvideo.raw
ITM504483_EBUcore.
XML
PCM Audioa1.raw
PCM Audioa2.raw
PCM Audioa3.raw
EBU CoreKLV Dictionary
mxflib
mxfdump
Standard MXF KLV Dictionary
playout
vlc
Contents
• Metadata exchanges?• Material Exchange Format• Metadata in MXF• Putting EBUCore into MXF• EBU MXF SDK • EBU MXF SDK – Demo• EBU MXF SDK – Improvements & Community
EBU MXF SDK – Improvements
• Input from the metadata community is welcomed and encouraged
• Experiment, use, and improve the existing code– Instructions on how to use, compile and embed in
the coming weeks
MXF SDK – Metadata Improvements
• Implementation of 377-1 revision changes– Extension of Descriptive Metadata elements with
grouping elements to aid identification of DM elements
• Foreign objects can be identified, but what about their semantics??– Can we store the dictionary in the MXF container?• Solution in proposed
SMPTE 377-2 – KLV Encoded Extension Syntax
Conclusions
• In B2B exchanges of AV material, embedded metadata makes sense
• In MXF files custom descriptive metadata, modeled as a graph of metadata set classes, can be embedded
• As an example, we illustrated how we can map EBUCore to a KLV-encoded representation
• This reference implementation SDK, along with other MXF manipulation tools, is being released as open source software by EBU
