understanding hardware acceleration on mobile browsers

Understanding Hardware Acceleration on Mobile Browsers Ariya Hidayat

whoami

Challenges

Game vs Web

Game

Animation

Physics

Textured objects

Text

Transformation

Large Area, but Still Bounded

Web Page

Images

Text

From Wave to Pixels

Layout engine

Network stack

GraphicsJavaScript engine

User interface

WebKit Components

WebKit Library

JavaScriptCore

WebCore

HTMLrendering

SVG

DOM CSS

Platform Abstraction

Network Unicode Clipboard

Graphics Theme Events

Thread Geolocation Timer

Graphics Abstraction

Mac Chromium Qt

CoreGraphicsSkia

QPainter

graphics stack

GraphicsContext

Mobile Device

Touch interface

GPU

CPU

Radio

Power

Graphics Processor

Divide and conquer

Very specific purpose

Relationship

CPU GPU

SoC = System-on-a-Chip

Fundamental Physical Limitations

1.Available memory2.Bus bandwith3.Speed difference

Optimized for Games

Transformation

Textured triangles

Fixed geometry

Parallelism

Challenges

Predictable contents (assets) ✔Mostly text ✔Mostly images ✔

immediateInitial response

http://www.trollquotes.org/619-super-spider-bat-troll-quote/

Primitive Drawing

Path is Everything

Path

Triangle Rectangle

PolygonEllipse

Stroke = Outline

Solid Dashed Dotted Textured

Brush = Fill

SolidNone Gradient Textured

Smooth via Antialiasing

Multiple levels of transparency

Perfect for parallelism

Path Approximation

Curves

Polygon

Gradient: Expensive

CPU: sequential, tedious

GPU: parallel, still a lot of work

Blur Shadow: Really Posh

Involved pixel “blending”

GPU: parallel, still tedious

Transformation

Scaling

RotationPerspective

Text Rasterization

Font Atlas

Bye

Buffer

Simple to Complex

Hello!

Simple shapeSolid color

CurvesGradient brushTextured stroke

Blur shadowSerif textRotated

Make it as an image

Backing Store

Maps

Tile

Pinch to Zoom

when you pinch...

Responsive Interface

Processor

Rendering

User interaction

decoupled

Rendering vs Interaction

Web Page

Backing StoreScreen

Rendering

User interaction

Checkerboard Pattern

Progressive Rendering

Crisp but slow

Fast but blurry

Tiling System

CPU GPU

........

Texture upload

Tile Transformation

Panning = Translation Zooming = Scaling

Backing Store Support

Texture-based

Vector-based

Honeycomb and later

Y U NOposition:fixed

Layer & Compositing

Mechanics of Animation

Initialization

Animation step

“Hey, this is good stuff. Keep it around as texture #42.”

“Apply [operation] to texture #42.”

Elements = Layer

Typical Animation

opacity, movement, scaling, rotation, ...

Immediate Mode

every animation tick

setInterval(function() {    box.draw(x, y);    x += 10;}, 20);

Scene Graph

box.drawInto(layer);setInterval(function() {    layer.translate(10, 0);}, 20);

Change transformation matrix

Logical 3-D

Keep the Textures

Compositing Support

Honeycomb and later

Well-known Trick

-webkit-transform: translateZ(0)

forcing 3-D transform

Not needed for CSS animation!

Magical Advice

Use translate3d for hardware acceleration

Misleading (at best)

Debugging in Safari

defaults write com.apple.Safari IncludeInternalDebugMenu 1

Compositing Indicators

Composited layer

Container layer

No texture

Overflow

Texture (number = upload)

Avoid Texture Reupload

No reupload Upload

OpacityPosition

SizeAnimation

Everything else!

“Hardware accelerated CSS”

Examples

Avoid Overcapacity

GPU = Limited silicon spaceLarge layer ➔ broken into “tiles”

Prepare & Reuse

Viewport

Hide the layer offscreen

Alpha Blending

Color change

Alpha Blending: The Trick

Blended with

Wrap-up

Hardware Acceleration

Drawing primitives

Backing stores

Layer & compositing

What Can We Learn?

Traditional graphics programming has been always full of tricks.

It will always be.

There is No Silver Bullet

Thank You

ariya.hidayat@gmail.com

@AriyaHidayat

ariya.ofilabs.com