what's inside 09 2013: gstreamer: black magic and witchcraft
DESCRIPTION
Introduction into GStreamer, how it is actually nothing like the titleTRANSCRIPT
GStreamerBlack Magic and Witchcraft
--what it is not--
Marc LeemanBarco Networking Division
Content● Goal● Introduction● Common misconceptions● Fun and Games● Development
Goal● The presentation should allow the audience to
leave the room and start playing with it○ No fancy products○ No hocus-pocus with hardware acceleration○ No real-life examples○ Really, nothing useful...
Introduction● What is GStreamer
○ set of libraries with a couple of example programs
○ highly configurable○ streaming framework, modular
■ autoplugging■ dynamic loading■ caps, incremental buildup pipeline,
visualisation of procedure, ...
Common misconceptions● GStreamer is slow
○ Pretty harmless: educate them○ Performance issues are mostly due to bugs (e.g.
alsa, h.264 parsing, …)● GStreamer is not for embedded
○ Stuck in the previous century?○ Axis encoders, cell phones, Settop boxes, ...
Basic Tools● What GStreamer instance do I have?
○ (jhbuild)marc@drd1812:~$ gst-inspect-1.0 |grep barco○ barcortp: rtpsink: RtpSink○ barcortp: rtpsrc: rtpsrc○ barco: edgeblend: EdgeBlend○ barco: vbcpsrc: VBCP client source○ (jhbuild)marc@drd1812:~$ gst-inspect-1.0 rtpsink○ Factory Details:○ Rank: none (0)○ Long-name: RtpSink○ Klass: Generic/Bin/Sink○ Description: Barco Rtp sink○ Author: Thijs Vermeir <[email protected]>, Marc Leeman <[email protected]>, Paul Henrys
<[email protected]>○○ Plugin Details:○ Name: barcortp○ Description: Barco RTP Plugins○ Filename: /home/marc/gstreamer.jhbuild/build/lib/gstreamer-1.0/libgst-plugins-barcortp.so○ Version: 1.0.0.11.1○ License: LGPL○ Source module: gst-plugins-barcortp○ Binary package: Barco○ Origin URL: http://www.barco.com/
….● Experiment with gst-launch
Fun and Games● Get a copy of GStreamer 1.x
○ apt-get install …○ make menuconfig && make (hardcore)
● Here we go…● Capture from the V4L2 interface, and show
○ $ gst-launch-1.0 v4l2src ! videoconvert ! xvimagesink
● Scale the video○ $ gst-launch-1.0 v4l2src ! videoscale ! video/x-raw, width=640, height=360 ! videoconvert !
xvimagesink
● Do both○ gst-launch-1.0 v4l2src ! tee name=t t. ! queue ! videoscale ! video/x-raw, width=640, height=360 !
videoconvert ! xvimagesink t. ! queue ! videoconvert ! videoflip method=2 ! xvimagesink
Fun and Games● What is happening here?
○ gst-launch-1.0 v4l2src ! tee name=t t. ! queue ! videoscale ! video/x-raw, width=640, height=360 ! videoconvert ! xvimagesink t. ! queue ! videoconvert ! videoflip method=2 ! videocrop left=400 right=400 ! xvimagesink
● Capture red colour, live○ gst-launch-1.0 v4l2src ! tee name=t t. ! queue ! videoscale ! video/x-raw, width=640, height=360 !
videoconvert ! xvimagesink t. ! queue ! videoflip method=2 ! videocrop left=400 right=400 ! alpha method=custom target-r=245 target-g=0 ! videoconvert ! xvimagesink
● Use camera and overlay “snow in a box”○ gst-launch-1.0 videomixer name=mixer sink_1::alpha=0.5 sink_1::xpos=50 sink_1::ypos=50 ! video/x-
raw,width=1280,height=720 ! videoconvert ! ximagesink sync=0 videotestsrc pattern=snow timestamp-offset=3000000000 ! "video/x-raw,format=AYUV,width=640,height=480,framerate=(fraction)30/1" ! timeoverlay ! queue2 ! mixer. v4l2src ! timeoverlay ! queue2 ! videoconvert ! mixer.
Fun and Games● What about other formats?
○ gst-launch-1.0 videomixer name=mixer sink_1::alpha=0.5 sink_1::xpos=50 sink_1::ypos=50 ! video/x-raw,width=1280,height=720 ! videoconvert ! fpsdisplaysink sync=0 videotestsrc timestamp-offset=3000000000 horizontal-speed=1 ! "video/x-raw,format=AYUV,width=640,height=480,framerate=(fraction)30/1" ! timeoverlay ! queue2 ! mixer. v4l2src ! timeoverlay ! queue2 ! videoconvert ! mixer.
● Something more nerdy?○ gst-launch-1.0 videomixer name=mixer sink_1::alpha=0.5 sink_1::xpos=50 sink_1::ypos=50 ! video/x-raw,
width=1280,height=720 ! videoconvert ! fpsdisplaysink sync=0 filesrc location=/home/marc/battlestar.galactica.blood.and.chrome.2012.1080p.bluray.x264-geckos.mkv ! decodebin ! videoconvert ! videoscale ! "video/x-raw,format=AYUV,width=640,height=480" ! timeoverlay ! queue2 ! mixer. v4l2src ! timeoverlay ! queue2 ! videoconvert ! mixer.
● More work○ gst-launch-1.0 videomixer name=mixer sink_1::alpha=0.5 sink_1::xpos=50 sink_1::ypos=50 ! video/x-raw,
width=1280,height=720 ! videoconvert ! fpsdisplaysink sync=0 filesrc location=/home/marc/battlestar.galactica.blood.and.chrome.2012.1080p.bluray.x264-geckos.mkv ! decodebin ! videoconvert ! videoscale ! "video/x-raw,format=AYUV,width=640,height=360" ! timeoverlay ! queue2 ! mixer. v4l2src ! timeoverlay ! queue2 ! videoconvert ! mixer.
Development● Develop against git master
○ lots-a-fixes, lots-a-features○ cooperation on IRC○ avoid fixing something that was fixed○ jhbuild (cerbero): standalone development
environment, free from system configuration○ 0.10 is dead, long live 1.0
● Runs on lots-a OSes (Linux, OSX, Windows) on lots-a architectures○ Active development is mostly done in Linux○ There is hardware impact