The MXF Mastering FormatThe MXF Mastering Format

A proposal to address existing MXF A proposal to address existing MXF interoperability issues.interoperability issues.

Clyde Smith, Steve Fish, Bruce Devlin, et al.

NBA-TV DomesticNBA-TV DomesticNBA-TV NBA-TV InternationalInternationalNBA-TV HDNBA-TV HD

5 Feeds5 FeedsSuper Station EastSuper Station EastSuper Station West Super Station West WTBS -17WTBS -17WTBS-DTV WTBS-DTV WTBS CanadaWTBS Canada

IndiaIndia

UKUK

11 Feeds11 FeedsUnited StatesUnited StatesBrazilBrazilLA Pan RegionalLA Pan RegionalUKUKFranceFranceGermanyGermany

SpainSpainItalyItalyPan EuropeanPan EuropeanSouth AsiaSouth AsiaPan AsiaPan Asia

13 Feeds13 FeedsTCM Domestic TCM Domestic TCM CanadaTCM CanadaTCM Latin AmericaTCM Latin AmericaLA Pan RegionalLA Pan RegionalFranceFranceSpainSpainUKUKNordicNordic

Pan EuropeanPan EuropeanGermanyGermanySkySkyAfricaAfricaPan AsiaPan Asia

8 Feeds8 FeedsEast CoastEast CoastWest CoastWest CoastTNT in HDTNT in HDBrazilBrazilArgentinaArgentinaMexicoMexicoVenezuelaVenezuelaPan RegionalPan Regional

24 Feeds24 FeedsU.S. East CoastU.S. East CoastU.S. West CoastU.S. West CoastBrazilBrazilArgentinaArgentinaMexicoMexicoLA Pan RegionalLA Pan RegionalU.K.U.K.NordicNordicDenmarkDenmarkFranceFranceGermanyGermany

Pan EuropeanPan EuropeanSpainSpainItalyItalyPolandPolandRomaniaRomaniaHungaryHungaryPan AsiaPan AsiaAustraliaAustraliaTaiwanTaiwanPhilippinesPhilippinesKoreaKorea

IndiaIndiaPakistanPakistan

ItalyItaly

U.S. East U.S. East CoastCoastU.S. West U.S. West CoastCoast

Turner Entertainment NetworksTurner Entertainment Networks

Serving North America from Atlanta, South America from Buenos Serving North America from Atlanta, South America from Buenos Aires,Aires,EMEA from London and Asia/Pacific from Hong KongEMEA from London and Asia/Pacific from Hong Kong

Turner Entertainment NetworksTurner Entertainment Networks

Warner BrothersWarner BrothersTechnical OperationsTechnical Operations

Turner Broadcasting SystemTurner Broadcasting SystemEntertainment OperationsEntertainment Operations

Retail SalesRetail SalesVHSVHSDVDDVD

Licensed MerchandiseLicensed Merchandise

New MediaNew MediaInteractiveInteractive

On LineOn LineBroadbandBroadband

Mobile PhonesMobile PhonesGamesGames

Satellite OperatorsSatellite OperatorsBranded ChannelsBranded Channels

PPVPPV

Cable OperatorsCable OperatorsBranded ChannelsBranded Channels

SVODSVODVODVODPPVPPV

Other Program ProvidersOther Program Providers

Turner Studios and Turner Studios and Original ProductionsOriginal Productions

Creation and Creation and AcquisitionAcquisition

DistributionDistribution

““Playout” OperationsPlayout” OperationsRepurposing Repurposing

Requires Requires ReprocessingReprocessing

Operational StrategyOperational Strategy

Locating key network support functions within their respective Locating key network support functions within their respective markets is a strategymarkets is a strategy that we’ve employed successfully in London, that we’ve employed successfully in London, Hong Kong and Buenos Aires. Hong Kong and Buenos Aires.

Our international operations tend to be Our international operations tend to be leanlean, which makes their , which makes their staffs staffs highly collaborative.highly collaborative.

The marketplace dynamics are changing so we must be innovativeThe marketplace dynamics are changing so we must be innovative

More and more “TV” content moving online means more choice for More and more “TV” content moving online means more choice for consumers and more opportunity for advertisers to reach highly consumers and more opportunity for advertisers to reach highly targeted, highly desirable audiences. GameTap, CNN Pipeline, targeted, highly desirable audiences. GameTap, CNN Pipeline, Dramavision, VeryFunnyAds.com, PGATour.com, ACCSelect.comDramavision, VeryFunnyAds.com, PGATour.com, ACCSelect.com and and Toonami Jetstream offer broadband users a mix of familiar and new. Toonami Jetstream offer broadband users a mix of familiar and new.

Workflow Key EnablersWorkflow Key Enablers

• I.T. based facilities

• Automated Repurposing and Reprocessing

• Standardized protocols and formats• MXF and AAF

• BUT are they ready for implementation in our operations?

We all knowWe all know• Building facilities based on IT workflows saves

money and speeds up time to market of products!

• But…. To succeed you need standards for interoperability

• If vendors are using their own “optimised” yet different wrapper formats:

Program Stream, VOB, Transport Stream, DV,AVI, Quicktime, MP4, TARGA, MXF OP1a,MXF OP1a (Sony), MXF P2, OMF, TIFF, etc.

• And…If many users have their own specific implementation options as well

We all knowWe all know

• Then eventually all users will pay all the vendors to implement all the formats and all the options to achieve interoperability

The problemThe problem – make multiple versions cheaply – make multiple versions cheaply

MXFMXF

• Was created to deliver business value to the industry

• Q: So what’s missing?Why can’t we find a solution off the shelf?

• A: Too many MXF toolsMXF has many options to address to many

problems

Principle Developers and Principle Developers and Supporters of the Supporters of the MXF Mastering FormatMXF Mastering Format

The GoalsThe Goals• Data entry. Data should be entered once at the appropriate point in the asset

life cycle. This data should be available to any system looking at that asset.

• Data updates. There should be the ability to revise or edit data. Any revisions or edits must be updated across the ‘Inventory’

• Files usable before being closed (i.e. partitioned.) This allows preview work to start before the write cycle is completed and the file closed. The time between first byte arriving and first use should be reasonable short.

• The concept of an asset ‘Inventory’. This is where everything related to a single asset is stored. Asset in this context means a single episode of a show.

• The ability to extract a copy of this ‘Inventory’ and be able to send it to an independent site. Once received, it must be importable to the asset management system (which may be different to the source system), based solely on the supplied file. This means it must be self describing with all essence and metadata bound in. One benefit of this is that best of breed local solutions can be employed and reduce reliance on single monolithic systems across organisations.

The GoalsThe Goals• Storage system independent. As assets are expected to be used in many

different places at different times there must be no reliance or limit on particular physical storage devices or systems.

• Flexible restores. A need was identified for ‘Partial Restore’. This is where a selection of essence tracks from the inventory would be restored to form a new version of the asset. This functionality would be further extended to give ‘Partial Partial Restore’ where a time sliced selection of essence tracks would be restored. Finally, a ‘Partial Partial Partial Restore’ requirement was identified where a group of time sliced sections of a selection of essence tracks would be restored, this last requirement was particularly aimed at clip selection for edit purposes.

• Flexible stores. The need to add tracks (essence or metadata) to an ‘Inventory’ without the overhead of having to re-write the entire ‘Inventory’.

• Metadata must reside with, and be updated throughout an assets lifecycle.

• Essence files to be external. As flexibility of adding (and removing!) of essence was required it was realised that for this to be most efficient all essence tracks should be external.

The GoalsThe Goals• Files must represent Multiple versions

> Different numbers of breaks in a program> Different language variants> Different titles & credits> Different output deliverables from one master

• Using MXF must lead to efficiency improvements> Reduction in transfer bandwidths> Reduction in storage requirements> Increased automation> Partial restore direct from tape> Standardised version identification> Greater workflow choices

• Risk reduction for the end user> Choice of multiple vendors working to a common standard

Defining the problemDefining the problem

• Look at the lifecycle of material

•Put all functional blocks on a diagram, label them and link Put all functional blocks on a diagram, label them and link them at a physical / networking level.them at a physical / networking level.

•For each major workflow create a table that shows the For each major workflow create a table that shows the individual actions required in that workflow. Each action individual actions required in that workflow. Each action should be documented to describe what happens at the should be documented to describe what happens at the physical, essence and data levels.physical, essence and data levels.

•Convert each action into a UML activity diagram that shows Convert each action into a UML activity diagram that shows in sufficient detail what is required from the action, and the in sufficient detail what is required from the action, and the messages that are likely to flow. messages that are likely to flow.

Defining the problemDefining the problem

• Look at the lifecycle of material

Defining the problemDefining the problem

• Work through some material lifecycle examples> Identify processes> Identify common states> Identify properties of file which reduce complexity> Identify properties of file which improve operational

flexibility

> Example…

Src Dst aWorkflow Reference Notes src dst stimulus

0 1 a 0.1.a Data populates london databse scarlett Xen external1 18 b 1.18.b Update vision with new program Xen Vision Automatic on programme arriving

9 1 b 9.1.b An order is raised by the ordering department Xen Xenhuman issues an asset order (program planning actor)

1 2 a 1.2.a Order is placed on external supplier Xen ??? 9.1.a

1 7 a 1.7.a Notify DAM of new Series 0.1.a DAMnotification of completion of 1a, triggers human actor "publishes" series title

1 7 b 1.7.b Notify DAM of new episode 0.1.a DAMnotification of completion of 1a, triggers human actor "publishes" episode title

1 7 f 1.7.f Notify DAM of new version variant 0.1.a DAMnotification of completion of 1a, triggers human actor "publishes" version variant

3 1 a 3.1.a The asset arrives and a barcode is added 0.1.a

completion of 1a, human matches title information on the tape with master title on Xen & prints barcode

3 6 a 3.6.a Tape is stored in library 3.1.a Tape arrived

9 1 c 9.1.c Asset is linked to varient in Xen 3.1.aHuman Actor links arrived tape to actual varient (version) and Job can be raised

1 7 c 1.7.c Notify DAM of a new asset 9.1.c25a causes operator links asset to tape on Xen

1 7 d 1.7.d Notification of job in mediaflex7 24 b 7.24.b Operator notified of job alert operator to ingest requirement

6 22 a 6.22.a Tape moved into ingest25a causes operator requests tape to be sent

22 8 a 22.8.a Tape is ingested and previewed 1.7.d24 7 a 24.7.a metadata insertion Actor 24 724 7 b 24.7.b decision on succesful ingest Successful preview7 16 a 7.16.a Notify Automation 24 7 triggered on succesful preview

7 16 b 7.16.b Request Archive via automation 24 7triggered on succesful preview - DAM to automation

16 10 a 16.10.a Request Archive automation to avalon 16 10 automation to archive10 16 a 10.16.a Archive success Avalon to Pro-Bel16 7 b 16.7.b Notify DAM of successful job 16 7 Automation notifies TMD of job finished7 1 e 7.1.e Mediaflex notifies Xen of finished job 7 1 triggered on successful ingest1 18 a 1.18.a Xen notify Vision 1 18 7e22 6 a 22.6.a Return tape to library 22 6 7e10 16 c 10.16.c Detect successful archive 10 167 24 a 7.24.a Notify of success / failure of archive 7 24 ENDPROC

Defining the problemDefining the problem• Identify all the interfaces clearly – an example

> Src ID - 8> Dst ID - 15> Level - a> Worflow ref - 8.15.a> Description - move from Ingest to playout> Type - Data & Essence> Req. mech - automation> Metadata src - file> Metadata dst - file> Schema - n/a> Essence src - ingest station> Essence dst - on-line server> Notes - FTP based move of MXF wrapped data

Defining the problemDefining the problem• Identify all the interfaces clearly

> Src ID - ID of the source of the data> Dst ID - ID of the destination of the data> Level - sub ID to distinguish which workflow is in use> Worflow ref - unique ID of this workflow step> Description - textual description of workflow step> Type - Data / essence / physical / control / SDI> Req. mech - A Trigger e.g. API, hot folder, keyboard, automation> Metadata src - e.g. Business DB, Asset DB, DAM, Automation, file> Metadata dst - e.g. Business DB, Asset DB, DAM, Automation, file> Schema - schema for metadata if available> Essence src - which machine> Essence dst - which machine> Notes - anything else

• Results in a very big table

Parameterizing the specificationParameterizing the specificationProgramme Component filesParameter Value

embed_VBI True – VBI is carried in the MPEG user_data as well as VBI atom

encode_VBI False – no rendering of VBI in active picture within the MXF domain

video_essence <TBD – Add profile labels for the A/V combinations>SMPTE 381M MPEG2 with the following options:50 Mbit I frame only15 Mbit long-GOP, dynamic GOP, nominally 15 frame GOP for 525, 12 frame GOP for 625 systems25 Mbit long-GOP, dynamic GOP, nominally 15 frame GOP for 525, 12 frame GOP for 625 systems

audio_essence SMPTE 382M uncompressed WAV audioPCM audio shall be mono, stereo or surround uncompressed Broadcast Wave audio –described with

a single track and be manipulated without having to demultiplex separate language tracks from an interleaved bundle.

ingest_TC LTC – ingested timecode shall take the LTC from the source material

DM_ASlang xx.xx.xx.xx.xx.xx.xx.xx – the application specific language tag

Simple Programme Version filesParameter Value

min_spv_dur 1 second

Complex Programme Version filesParameter Value

sub_clip_limit 1 second. In a long GOP file, this may start or end within a GOP

complex3b True.

Document results…Document results…

• What do the files look like?> List the different physical requirements of the file> List the metadata requirements of the file> Initially document them as a proprietary format

• Categorise requirements> Those that are generally applicable – RPzzz

recommendation> Those that are business specific – AS-TBS

The SpecificsThe Specifics

• Refer to files by their function not their OP

> Programme Components> Simple Programme Versions> Complex Programme Versions> Programme Inventories

Components,VersionsComponents,Versions and Inventories and Inventories

‘‘Components’ are single essence or data tracks Components’ are single essence or data tracks e.g. a Video track, Audio track or Metadata track.e.g. a Video track, Audio track or Metadata track.

‘‘Versions’ are pointers to Components and Versions’ are pointers to Components and describe how a collection of Components may be describe how a collection of Components may be played. There can be many different Versions played. There can be many different Versions pointing to the same Components. pointing to the same Components.

‘‘Inventories’ are a repository for all the Versions Inventories’ are a repository for all the Versions and Components for a single asset. There should and Components for a single asset. There should be no duplication of Components within the be no duplication of Components within the Inventory. There maybe many similar Components Inventory. There maybe many similar Components of the same type, for instance there could be three of the same type, for instance there could be three different Videos – High Resolution edit master, different Videos – High Resolution edit master, Transmission master and a Browse resolution copy. Transmission master and a Browse resolution copy. At least three versions would exist to allow each to At least three versions would exist to allow each to be played.be played.

Programme Componentsclips

MPFP

MPOP1b

reference external file packages

MPFP

MPFP

example.mxf

media

example_v0.mxf

example_a0.mxf

example_a1.mxf

Video Programme Components

Audio Programme Components

Audio Programme Components

Example of ComponentsExample of Components

PrincipalPrincipal – versions link to essence – versions link to essence

clips

MPFP

MPOP1b

reference external file packages

MPFP

MPFP

example.mxf

media

example_v0.mxf

example_a0.mxf

example_a1.mxf

Simple Programme VersionInventory

File Structure ExamplesFile Structure Examples

clips

MPFP

MP1OP1c

reference external file packages

MP

MP2

FP

MPFP

example.mxf

media

example_v0.mxf

eng

descr.

fre

example_a0.mxf

example_a1.mxf

MPFPexample_a2.mxf

InventoryA Simple Programme Version

within the Inventory

Inventories, Components Inventories, Components and Versionsand Versions

These definitions can be loosely related to These definitions can be loosely related to Operating Patterns in the following way: Operating Patterns in the following way:

Components OP1a.Components OP1a.Versions OP2B or OP3B depending on how Versions OP2B or OP3B depending on how complex the program structure is.complex the program structure is.

Inventories could OP3B or OP3C depending Inventories could OP3B or OP3C depending on how many versions are in the on how many versions are in the Inventory. Inventory.

This shows that talking in ‘OPs’ is variable This shows that talking in ‘OPs’ is variable and also that all Operating Patterns are and also that all Operating Patterns are likely to be met in a complex system.likely to be met in a complex system.

Example – adding a subtitleExample – adding a subtitle

clips

MPFP

MPOP1b

reference external file packages

MPFP

MPFP

example.mxf

media

example_v0.mxf

example_a0.mxf

example_a1.mxf

Simple Programme Version

example_vbi0.mxf (SMPTE436M)

Different title/creditsDifferent title/credits

MP1OP2c

MP2

eng freclips

MPFP

MPFP

MPFP

MPFP

MPFP

MPFP

MPFP

swim.mxf

media

swim_body_v0.mxf

eng

eng

eng

eng

eng

common

fre

fre

fre

fre

fre

swim_title_a0.mxf

swim_title_v0.mxf

swim_title_a1.mxf

swim_title_v1.mxf

swim_credit_v0.mxf

swim_credit_a0.mxf

MPFP

MPFP

swim_credit_v1.mxf

swim_credit_a1.mxf

MPFP

MPFP

swim_eng_a0.mxf

swim_fre_a1.mxf

A Complex Programme Version within the Inventory

Different Different title/creditstitle/credits

MP1OP2c

MP2

eng freclips

MPFP

MPFP

MPFP

MPFP

MPFP

MPFP

MPFP

swim.mxf

media

swim_body_v0.mxf

eng

eng

eng

eng

eng

common

fre

fre

fre

fre

fre

swim_title_a0.mxf

swim_title_v0.mxf

swim_title_a1.mxf

swim_title_v1.mxf

swim_credit_v0.mxf

swim_credit_a0.mxf

MPFP

MPFP

swim_credit_v1.mxf

swim_credit_a1.mxf

MPFP

MPFP

swim_eng_a0.mxf

swim_fre_a1.mxf

Inventoryeng Complex Programme Version within the Inventoryfre Complex Programme

Version within the Inventory

Metadata ExampleMetadata Example

Annotation Language en-USSpoken Language en-GB

Full Spoken language en-GB-x-DolbyE-a-PostWatershedDescription Dolby E surround track with profanity

Audio track

Metadata track

RFC 4646(ISO 639 ++)

Annotation Language en-USSpoken Language gr-Grek

Full Annotation gr-GrekDescription Greek subtitling for use in Greece

Data track

Metadata track

RFC 4646(ISO 639 ++)

Process Benefit exampleProcess Benefit example

Archive CentralStorage

PlayoutServer

BroadcastAdd “fre”

audio track

Copy back entire fileRewrite tapeto archivenew version

Archive CentralStorage

PlayoutServer

BroadcastAdd “fre”

audio track

Copy back just the“fre” audio track

Append “fre”audio to archivenew version

Interleaved files

component files

Format SpecificsFormat SpecificsBasic structure of the MXF Master format includes:

• Single-essence Programme Component files

> OP1a files that enable Read While Write & random access

• Programme Version files that reference Components

> Simple Programme Versions for playout servers

> Complex Programme Versions for editing and version management

• Programme Inventories that collate multiple versions

> An Inventory contains all the versions of an asset

• Controlled Descriptive metadata

> DM Tracks used to identify the language of an audio component

• Controlled way to specify the system

> Paramaterizing the Specification for a local facility is done in a consistent way for vendors – enhancing interoperability

Process AnalysisProcess Analysis

• If the underlying file format isstandardised & pervasive

• Standardised Analysis tools canbe used to…

> model workflows> define interfaces between departments> create service oriented software interfaces> define, track & model interactions between departments> define, track & model active, passive & redundant operations

ManagementSystem

Tape Library MediaGridOperator ArchiveIngestStation

Notify of Job1:

Request MXF Full Wrap7:

Request Archive10:

Request Tape Return13:

Ingest3:

Wrap Done9:

Pull File11:

Request Tape2:

Metadata Entry5:

Ingest Accepted6:

Success12:

Review4:

Pull File8:

Tape Returned14:

MXFMXF

• Needs to be constrained in order to deliver its value.

• Q: How can this value be delivered for you?What needs to be done to MXF in general?

• A: Accurate problem specificationand a Recommended practise to constrain range of options in the MXF toolbox

Formats ComparisonFormats Comparison

• OP1a Interleaved

Lengthy tape rewrites

Bandwidth hogging file transfers

Incompatible proprietary formats

• MXF Master Format

Simple to repurpose files

Built-in Metadata Storage

Highly Interoperable

StandardisationStandardisation

• SMPTE> Many are keen to see this work in committee> RDD option fast but is simply a static “we did this” document> EG option lacks enforcement> RP option feels right> Full standard wanted by some

• We would welcome your

review and comments

As well as your support

MXF API OverviewMXF API Overview

• In order to promote digital interoperability, and the exchange of program material using files Turner is promoting a standardisation of the use of MXF in the broadcast production and playout industry.

• As part of the standardisation is a requirement not only to specify exact MXF operating patterns to be used but also how they will be manipulated.

• Efforts are underway to develop a common MXF Processor API

• This API is intended as a universal language for the manipulation of MXF media objects.

A Standardized API A Standardized API for MXF manipulationsfor MXF manipulations

• With a standard file format• Many operations become standardized

> Restore> Partial restore (of a subclip)> Partial restore (of a subset of tracks within a subclip)

• MXF version files> Can be represented in MXF> Are small in size> Fully describe the source file> Can fully specify a destination file> Are web services friendly

Why?Why?

Video Server

MXF

MXF

MXF

Character Generator

MXF

MXF

MXF

Non Linear Editor

MXF

MXF

MXF

Graphics device

MXF

MXF

MXF

Automation

Proprietary API 1

Proprietary API 2

Proprietary API 3

Proprietary API 4

• Vendors currently develop their own proprietary APIs

• These restrict interoperability and thus automation (or control) systems must implement many different APIs

• Since we’re trying to recommend an MXF standard let’s do the same for the API

BenefitBenefit

• A single open API that automation (or controller) can interface to

• Vendors interface to a common API

• Reduces integration costs and time

• Provides interoperability at a control layer for MXF aware devices

Video Server

MXF

MXF

MXF

Character Generator

MXF

MXF

MXF

Non Linear Editor

MXF

MXF

MXF

Graphics device

MXF

MXF

MXF

Automation

Common MXF API

Processor

More than an APIMore than an API

• An open API that each vendor interfaces to• Each toolkit vendor will support the API and will

provide a subset (or maybe all) of the transformations/functionality

• The calling application (automation) may issue a query to see which of the underlying processors can support a particular function

• This approach will allow multiple vendors to collectively provide the overall system functionality.

More than an APIMore than an API

• Proposed that the API runs under SOAP> In general set up as a Call / Response system with a ticket or

handle to the job in progress > allowable exception to this is when the job terminates and

the processor wants to flag the calling system so it doesn’t have to persist the result set

• Management of name space

• Referential integrity

Basic FunctionalityBasic Functionality

• Create a media object• Append to that object• Move a whole object• Copy a whole object• Delete a component from within that object• Extract one or more components from an object• Delete a whole object• Interrogate an object• Manage the activities

Use casesUse cases• Complex Program Version to Simple Program Version• Simple Program Version to Complex Program Version• Create/remove empty Program Inventory• Add/delete Complex Program Version to Inventory• Add/delete Program Component to/from Program

Inventory• Add/delete Program Component to/from Complex

Program Version • Complex Program Version to Complex Program Version

Referential IntegrityReferential Integrity

• This is a set of rules not a set of functions within the API

• Only packages actually handled by the processor, or appearing in a configured directory will be managed by the referential integrity checking.

• Ensures the header and associated essence files are valid

Name Space?Name Space?

• Should the processor embody & enforce a naming convention for component and version files? If so is this naming convention part of the MXF application specification?

• Should this name space be determined solely by the end user, or should a recommended practice be developed to try to encourage standardisation?

SummarySummary

• MXF Processor API will provide a common set of functions that all MXF compatible devices will support

• Advantage that a given device will require minimal testing/development for integration into environment supporting API

• Allow customers/users to quickly determine if a device should be compatible within their environment

• Reduce overall system integration time (and cost) for new device

• Vendors will have immediate solutions their device can integrate within should they adopt open API

Master Once

Re

pu

rpo

se

M

an

y

We Are Here

NAB 2007NAB 2007• Tuesday 1p-5p MXF Papers in Broadcast Engineering Conference

Reception (Immediately Following - Free Drinks!)

• Demonstrations Area

• files ready now or being developed> Snell & Wilcox> Omneon> Marquis> Softel

• API & metadata interop being defined> Metaglue> TMD> Opencube> Pro-bel

Developers and Developers and Supporters of the Supporters of the MXF Mastering FormatMXF Mastering Format

ReferencesReferences

• The MXF Book • SMPTE 377M et al• RPzzz MXF Master

format• The File Interchange

Handbook

Questions?Questions?