origo mdn 2015

OrigoTormod Værvågen, with major contributions from Robert HP Engels Systems Architect, NRK

Metadata, archive, publishing, sharing, rights, graph-based, unique identifiers, authority registers, ontologies, ccdm, ebucore

Architecture Problems with current workflow Radioarchive Metadatabank Storytelling

OrigoA new story based metadata driven

architecture for broadcast

OrigoA new story based metadata driven

architecture for broadcast

But how did it start?

Simple

Linear

Metadata

Planning Studio Archive

Radio

Planning Production Archive

Radio


Radio


Television


Radio


Television

Expensive


Radio


Television

Planning Production ArchiveWWW

Forking

Planning System

Production System

Playout System

Publishing System

Forking

Planning metadata

Production changes metadata

Playout changes metadata

Publishing changes metadata adaptation

version

Error check

Wrong dataWrong data

PREPRODGranitt

Database-skjema

(Gjenstår)

STAGEGranitt

MAORACLE23

VMAODA-STAGE01 VMAODAPROD01

PREPRODGranitt

Database-skjema

Distribusjons-motor (Prod)

VMAODADIST01

SAN

What’s On/PRF

Abit/Playout

PROGRAMME BANK

Oracle Link (Pi Schema)

Media management Programm

Contracts Planning/Scheduling

VoD

NRK Internal Network NRK DMZ

Indexing

Transcode

Archive/Retrieve

Ingest

ODA (GUI) PROD

3

DigAs

Indexing

Ingest

Trandskodede filer

Akamai

Trandskodede filer

12

TEKO

Push

Mediafile transfer

Metadatatransfer

56

21

2

3

4

7

8

5

Existing system

n New system

n

sFTP

Opptak av sendinger

med Live

innhold

Dag Hallaråker /TPSM Programspiller - Systemgruppen

n

7

Grunnstrukturen Ver. 15.5 (04.01.13)Versjon 1.0

IntegrasjonerOD-Verktøy og distribusjon

2

Opus Interchange

(SOAP)

MgX Web-service(SOAP)

MySQL12

Oppdatering Kontinuerlig

Oppdatering 1 /24t

PRF pusher nytt«Pattern» når en ProgID blir

satt til aktiv eller det blir gjort en endring i på

aktive ProgID

SigmaVMALXSIGMA01

Timesfiler

RabbitMQPull4Tagging og

indeksering

Push

Gluon spin

MgX Web-service(SOAP)

Tekstefilene på KNF-format20

10

8

PiDist

MDW

Oracle Link (Pi Schema)

Push13

Playout

15

25

«24/t XML» Programinfo (Ikke innslag)

23

Pac2KNFKonverter

VMAKNF01

Opus Interchange

(SOAP)

ProgramspillerBACKEND

ProgramspillerFrontendAZURE

9

25

OUTSIDE

API House-keeping

10

PRODGranitt

MAORACLE17

Granitt til Granitt

i DMZ Synkroni-

sering

(Erstatte GG med

Tungsten gjenstår)

Services and adapters developed

spesific for Programspiller

Søke-opti-mali-

sering

13

6

PiData-varehus

Oracle

ENPS

10

9

Push

9

23

Thomas programmet

V.2.12

Indeksering fra ENPS kjøreplan

(Program)

11

Oracle link

«5 min etter XML» Info og indeks for innslag i program

Konvertering «5 min etter XML»

Transformere timesfiler til en fil

pr program

«5 min etter XML» utvidet med filnavn og status Pushes når transkoding ferdig

23

SIFT

Media DBOpus

Interchange(SOAP)

Pi til Granitt synkronisering hver

gang det er en endring i Pi

Avgrenset til alle relevante metadata for Programspiller

TESTGranitt

MAORACLE22

5

PRODGranitt

Database-skjema

Golden Gate

STAGEGranitt

Database-skjema

TESTGranitt

Database-skjema

MySQL Synk

MySQL Synk(Datafly velger synk fra

Prod eller Preprod)

TV.NRK.NORADIO.NRK.NO

TV.PREPROD.NRK.NORADIO.PREPROD.NRK.NO

TV.STAGE.NRK.NORADIO.STAGE.NRK.NO

ODA SIGMA/Gluon

AdapterPROD

ODA Omnibus AdapterPROD

ODA Subtitle AdapterPROD

ODA MGX

AdapterPROD

ODA (GUI)

STAGE

Distribu-sjons-motor

STAGE

Prod

Stage

Tungsten

Servicene sjekker xml-syntax og legger i sin kø på RabbitMQ.

PROD- og STAGE-Adapterne leser fra kø, lagrer i Granitt og MongoDB (etter suksessfull

operasjon mot Granitt)

9

VMAITEMINFO01

MASRV05RADIOMETADATA-

LØYPA 14

ODA SIGMA/Gluon

AdapterStage

ODA Omnibus AdapterStage

ODA Subtitle AdapterStage

ODA MGX

AdapterStage

7

ODA MGX

ServicePROD/Stage

ODA SIGMA/Gluon

ServicePROD/Stage

ODA Omnibus ServicePROD/Stage

ODA SubtitleServicePROD/Stage

«24/t XML» Programinfo (Ikke

innslag)

24

MongoDB(xml)

MySQL Synk(styres av dataflyt)

ODA Omnibus Service spør hvert minutt om endinger siden

sist. Opus Interchange returnerer liste med Carrier ID(external_key/ref). ODA spør om «full clip data» for alle

CarrierID som er aktive og lagrer (overskriver) i Granitt.

1

Back to start

Storytelling, not technology

Describing what we do

in every step

Architecture and standards

• A service-bus is not the solution

• Service-busses are just tools, as message quest and network

• One clear architecture is a solution

• One Metadata standard with a common repository

• Stores every state and any new metadata for each story

• Inform the user about the state

New requirements:

• Web first - News on web, then radio and tv

• Keep all the versions together, story focus

• Web ties production directly to publishing

Goals:

• Keep all adequate metadata from each step

• Planning

• At the time of recording, in-house or outside

• Production

• Archiving - publishing - play-out

Information architecture &

Data model(s)

EBU CoreEBU CCDM

Administrative

Technical

Social/ editorial

Administrative

Technical

Social/ editorial

Planning Buying Contracts

Administrative

Technical

Social/ editorial


Creation Editing Transcoding

Administrative

Technical

Social/ editorial


Creation Editing Transcoding

Contextualising Describing Relating

PLANNING, ACQUIREMENT & UPLOADING

PRODUCTION

Linear Playout

& Streaming

ARCHIVING & AVAILABILITY

Ondemand & SoMe

METADATA

AUTHORITY REGISTERS

Demo

http://localhost:19000/

Current NRK production

md 1 md 4md 3md 2

Season 1

TV - Series

md 5 md 8md 7md 6

Season 2

Original plan

File 1 File 2 File 3 File 4 File 5 File 6 File 7 File 8

md 1 md 4md 3md 2 md 5 md 8md 7md 6

During production



md 1 md 4md 3md 2

Season 1

TV - Series

md 5 md 8md 7md 6

Season 2

Retrieved or recreated post-production


Storytelling: supported by metadata

Script

Storytelling


md 1 md 4md 3md 2

Season 1

TV - Series

md 5 md 8md 7md 6

Season 2

Script


md 1 md 4md 3md 2

Season 1

TV - Series

md 5 md 8md 7md 6

Season 2

Script

Script writerInterview

screen writer Interview

script writer Storytelling


md 1 md 4md 3md 2

Season 1

TV - Series

md 5 md 8md 7md 6

Season 2

Script

Script writerInterview

screen writer Interview

script writer

Theater play

Event

Place

Interview actor

actor

Storytelling

All by ourselves?

Frontend & User interface

Core production- and editing systems

Meta data core



Meta data core

Yes Search Advanced searchMetadata editing

Desktop integration



Meta data core

Buy & adapt


Desktop integration

Ingest MAM Workflow engine Storage Editing QC Transcode Play-out



Meta data core

Buy & adapt


Desktop integration

Ingest MAM Workflow engine Storage Editing QC Transcode Play-out

Yes, based

on open

source

NCS data CDs Planning data Editorial data Social data Rights Archive

Some early ORIGO project results

ARCHITECTURE • Integrates with production

systems, QA & reports • Provides admin GUIs, otherwise

fully integratable with 3rd parties • Uses new IT platforms and systems • ECO system / modular • Based on «principles» of

community building

DATA MODEL • Graph/Object based datamodel • Globally published Authority

registers • Metadata updated by

production and post-production systems

• All metadata available through REST/full and query-based APIs

• Compatible with Semantic Web • Uses Global Ontologies

Tormod VærvågenSystems Architect [email protected] @tormodv

mailto:[email protected]

origo mdn 2015

Software

planning studio archive

oda gui prod digas indexing

motor stage tungsten

nrk metadata

simple linear metadata

t xml programinfo ikke

mysql synk s

etter xml info og indeks