Thanks !

The Stream Oriented Generic Enabler SO GE

Developing rich multimedia applications with FI-WARE.IVAN GRACIA

igracia@kurento.orgOpen APIs for Open MindsHuman communications

With the advent of unlimited channels, unlimited applications, unlimited news, unlimited information and unlimited entertainmentwhy are we feeling more isolated?

Has the technology of connecting human beings and ideas only begun to tap into fulfilling the most fundamental human needs? Even as we are living through a transitional period where we are now able to instantly attach to other people and ideas, why are we longing less for instant messaging and more for meaningful messaging?1Why technology?

2G3G4G5GAudioVideoHD VideoHD 3D VideoMultimedia communication technologiesOur vision: technology for more than transport

From this to this

processingKnowledgeOur ingredients for cooking the Stream Oriented GE

If you're not familiar with WebRTC, it's the technology which will cause the next drive towards mass adoption of voice, video and file collaboration. In essence, it will enable high quality video and audio in your web browser, which is one of the most deployed applications in the world.

It has been drafted by W3C (World Wide Web Consortium) with protocol work done by IETF (Internet Engineering Task Force). WebRTC ultimately simplifies the incorporation of real-time communications into a web browser.

The objective of WebRTC is to trigger a real-time session in your browser, without having to download anything, simply functioning as part of the normal operation of browser. Everything required to deliver a high quality experience at the endpoint is supported natively within a WebRTC-capable web browser.

5WebRTC

WebRTC

Developing the client sideDeveloping the infrastructureside

Before WebRTCAfter WebRTCNext natural stepBeginEndBeginEndBeginEnd

Unified APIsStandardsFOSSMultiplatformUnified APIsStandardsFOSSMultiplatformUnified APIsStandardsFOSSMultiplatformWebRTC defines several APIs together with a collection of communication processes and protocols. From a development perspective, core functions are encapsulated into three main JavaScript APIs: getUserMedia, RTCPeerConnection and RTCDataChannel. These APIs are incorporated into browsers that support WebRTC, hence a web developer who has JavaScript programming experience can bring an interactive video collaboration experience to the web.

WebRTC allows a mesh-based technology to enable users to send and receive streams to and from each other.7WebRTC as a disruptive technology

Number of usersGeneral purposeRTC multimedia ServicesSpecific purpose RTC multimedia services

Commodity, no business opportunities hereWho cares about the immense minority?Many business opportunities hereDisruption directionof WebRTCAlthough WebRTC isn't confined solely to web applications, embedding real-time communications directly into web browsers has been the focus for most of the industry. With WebRTC it becomes possible to embed real-time video into a range of vertical applications, including for business, medical and education purposes.8WebRTC is a WWW technology

WWW TECHNOLOGY & STANDARDSBankingCRMTVCommerceDocumentSkype?Traditional phone service?10

Peer-to-Peer Application (without media infrastructure)WebRTC data streamApplication with media infrastructure

Media infrastructureWebRTC and WebRTC infrastructuresWebRTC media servers

Transcoding media serverVP8H.264

MCU/SFU media server

Recording media server

KurentoWhat common WebRTC Media Servers do:TranscodingMCURecordingWhat future MediaServers will do:Flexible processingAugmented realityBlendingMixingAnalyzingEtc.Media ishereMedia goesthere

TranscodingMCU/SFURecordingCurrent media serversMedia ishereRich Mediagoes there

Future media serversTranscoding, MCU/SFU,Recording,Enrich, Augment, Analyze, Combine, Transform, Adapt, ContextContentCommandsMedia EventsKey concepts: media elements and pipelines13Media ElementProvides a specific media functionalitySend/receive media. These are the EndpointsProcess mediaTransform mediaReady to be usedNew media elements can be addedMedia pipelineChain of media elements implementing the desired media logicThe Media Server provides the capability of creating media pipelines by joining media elements of the toolboxMedia ElementSinkSRC

Media Pipeline

14

14

15

15

Kurento applications as a Lego game

Application 2

Application 3

Application 1

Toolbox of media elements

Creating applications with Kurento

18Campus Party Brazil - 2014

18

19

19

Campus Party Brazil - 2014SinkSRC

Sink

WebRtcEndpointSinkSRC

SinkSRC

MirrorFilterPointerDetectorFilterSinkSRC

ChromaFilterSinkSRC

FaceOverlayFilterMedia Server

20

20

Ready?

What you should learn firstWebRTC basicshttp://www.html5rocks.com/en/tutorials/webrtc/basics/Signaling basics (STUN/TURN)http://www.html5rocks.com/en/tutorials/webrtc/infrastructure/

Where to startKurento official documentationhttp://www.kurento.org/documentationKurento FIWARE documentationCatalogue sitehttp://catalogue.fiware.org/enablers/stream-oriented-kurento Documentationhttp://catalogue.fiware.org/enablers/stream-oriented-kurento/documentation

Installing Kurento Media ServerRequirementsUbuntu 14.04 box (sudo)Internet connectivityInstallsudo add-apt-repository ppa:kurento/kurento sudo apt-get update sudo apt-get install kurento-media-serverLaunchsudo service kurento-media-server start

Kurento Hello World

Kurento Hello WorldTutorialhttp://www.kurento.org/docs/current/tutorials/js/tutorial-1-helloworld.htmlCodehttps://github.com/Kurento/kurento-tutorial-js/tree/release-5.1/kurento-hello-world Understanding this examplePeerConnection APIgetUserMedia APIkurento-utilskurento-client-jsApplication CodeWebSocket APIKurento Media ServerBrowserSignalingMediaUnderstanding this example

You dont catch the point?

Kurento Hello World with Application ServerPeerConnection APIgetUserMedia APIkurento-utilsApplication CodeComm APIKurento Media ServerBrowserSignalingMediaApplicationServerKurento-client-jsApplication CodeSignalingKurento Hello World with application serverJava EE Application Serverhttp://www.kurento.org/docs/current/tutorials/java/tutorial-1-helloworld.html

Node.js Application Serverhttp://www.kurento.org/docs/current/tutorials/node/tutorial-1-helloworld.html

Understanding thisexample

Why an application serverSecurityKMS does not provide AAA in its control interfaceIn WWW applications, DD.BB. access is directly exposed to the browser?Call signalingHow several browsers can communicate?Recording and playingSinkSRC

Sink

SinkSRC

SRC

Recording and playing: source codeGet code herehttps://github.com/Kurento/kurento-tutorial-js/tree/release-5.1/kurento-hello-world-recorder-generator

Understanding this exampleGeneratorsBlack magic for avoiding callback hellProgram asynchronously with synchronous philosophyWarningEnable Experimental JavaScript flagAdvanced topics: Working with calls

P2P WebRTC callGroup calls

Mesh topologyStart topologyWebRTC one-to-many exampleMedia PipelineWebRTCstreamingSinkSRC

WebRTCstreaming

SRCSink

SRCSink

WebRTCstreaming

WebRtcEndpoint 1WebRtcEndpoint 3WebRtcEndpoint 2SRCSink

WebRTCstreamingWebRtcEndpoint 4User 1User 2

User 3

User 4WebRTC one to many example: codeJava EEhttp://www.kurento.org/docs/current/tutorials/java/tutorial-3-one2many.html

Node.jshttp://www.kurento.org/docs/current/tutorials/node/tutorial-3-one2many.html

Understandingthis exampleBack-to-back full-duplex calls

Understanding this exampleEnriching full duplex calls

Back-to-back full-duplex calls: source codeJava EEhttp://www.kurento.org/docs/current/tutorials/java/tutorial-4-one2one.htmlhttps://github.com/Kurento/kurento-tutorial-java/tree/release-5.1/kurento-one2one-call-advanced

Node.jshttp://www.kurento.org/docs/current/tutorials/node/tutorial-4-one2one.html

Advanced topics: working with filters

From this to this

processingKnowledgeOther filters: Augmented realitySinkSRC

SinkSRC

Other filters: Motion detector

Other filters: CrowdDetectorCrowdEventsSRC

SinkSRCCrowdDetector

RtspPlayerSinkSRC

Create your own filter: Kurento modules

Create a custom moduleTwo flavorsOpenCV moduleGStreamer module

Instructionshttp://www.kurento.org/docs/current/mastering/develop_kurento_modules.html

http://fi-ppp.euhttp://fi-ware.euFollow @Fiware on Twitter !

Thanks !5252