html5 video filters
TRANSCRIPT
HTML5 VIDEO FILTERSIN SEARCH OF THE DRAGON BALLS
Created by [email protected]
OBJECTIVEShow in a browser a video with the following controls:
GrayscaleBrightnessContrastRGB channels
LOOKING FOR AN IDEA1. The mysterious SVG filters?
✗ potentially the best option, but 2014 is too early because...✗ not so good browsers' support✗ IE9- (XP/Vista) don't support SVG at all
2. The lightweight CSS filters?✗ bad browsers' support✗ no support for feColorMatrix SVG filter
3. The powerful WebGL?✗ bad browsers' support✗ unfamiliar enviroment for most developers✗ not W3C, depends from graphic board's drivers
4. The hungry Canvas?✓ good browsers' support (IE9+)✓ polyfills for IE8 (but don't rely on them)✓ familiar enviroment for most developers
A VIDEO IN A CANVAS var videoElement = document.getElementById('myVideo') ; var canvasElement = document.getElementById('myCanvas') ; var canvasContext = canvasElement.getContext('2d') ; canvasContext.drawImage(videoElement,0,0,width,height) ; // loop
DEMO
VIDEO854 × 480 = 409.920 px× 4 channels = 1.639.680 byte
CANVAS
Red 80%
Green 80%
Blue 80%
Contrast 80%
Brightness 80%
Black & white
13.9FPS
PROGRAMMING TIPSconsider not using frameworks or polyfillsaccess DOM as little as possiblereduce the number of dotscache everything (i said EVERITHING!)don't write 2 times the same thinguse while(i--) insead of for(i++)multiply instead of dividerequestAnimationFrame()['a','b'].join('') instead of 'a'+'b'bitwise operations when convenient (i.e. floor if n>=0)O'Reilly - High Performance JavaScriptjsPerf, StackOverflow, Google
TYPED ARRAYSDirect access to RAM (IE10+)
// Floating point arrays.var f64 = new Float64Array(8);var f32 = new Float32Array(16);
// Signed integer arrays.var i32 = new Int32Array(16);var i16 = new Int16Array(32);var i8 = new Int8Array(64);
// Unsigned integer arrays.var u32 = new Uint32Array(16);var u16 = new Uint16Array(32);var u8 = new Uint8Array(64);var pixels = new Uint8ClampedArray(64);
Examplevar a = new Uint8Array(64); // length = 64 // 0 < value < 2^8 ;a[0] = 300 ;console.log (a[0]) // 44 = 300 - 256
BUFFERS AND VIEWSvar buffer = new ArrayBuffer(16); // 16 byte = 128 bit of RAM// to access it, you need to create a viewvar int32View = new Uint32Array(buffer); // length = 4 (128/32) // each element = 0 < value < 2^32// you can create multiple views of the same buffervar int8View = new Uint8Array(buffer); // length = 16 (128/8) // each element = 0 < value < 2^8// what does it mean?int32View[0]=300;
console.log(int32View[0]); // 300console.log(int32View[1]); // 0
console.log(int8View[0]); // 44console.log(int8View[1]); // 1
// it means that the Endianess is "Little-endian" (first byte = less significant)
YOU ARE ALREADY USING THEMCanvas 2D
alert(canvasElement.getContext('2d').getImageData(0,0,w,h).data);// in most recent browsers, "data" is an [object Uint8ClampedArray]// in older browsers, "data" is an [object CanvasPixelArray]// the cause is an HTML5 spec change
var a = new Uint8ClampedArray(2); // length = 2 < value < 2^8 ;a[0] = 300 ;console.log (a[0]) // 255console.log (a[1]) // 0
XMLHttpRequest2
File APIs
Media Source API
WebGL
Transferable objects
Binary WebSockets
WEB WORKERS (IE10+)Execute JavaScript in background (multithreading!!!)
Honestly, i had no time to try them :P
demo by Conor Buckley using getUserMedia()http://bit.ly/10YhHoK
PIXI.JS2D WebGL framework with canvas fallback O.O
pixijs.com
(video with audio will autoplay on next slide)
WHAT IS THE (COLOR)MATRIX?var brightness_colorMatrix = [// r g b a /*R*/ 1, 0, 0, x,/*G*/ 0, 1, 0, x,/*B*/ 0, 0, 1, x,/*A*/ 0, 0, 0, 1];
// R = r*1 + g*0 + b*0 + a*x ;// G = r*0 + g*1 + b*0 + a*x ;// B = r*0 + g*0 + b*1 + a*x ;// A = r*1 + g*0 + b*0 + a*1 ;