considering tiling for a better user interface

58
Considering Tiling for a better User Experience Kenneth Rohde Christiansen WebKit Code Camp, Wiesbaden, December 2009

Upload: kenneth-rohde-christiansen

Post on 19-May-2015

5.520 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Considering Tiling for a better User Interface

Considering Tiling for abetter User Experience

Kenneth Rohde ChristiansenWebKit Code Camp, Wiesbaden, December 2009

Page 2: Considering Tiling for a better User Interface

SOMETHING THAT I HAVE BEEN WORKING ON

Page 3: Considering Tiling for a better User Interface

SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I?

Page 4: Considering Tiling for a better User Interface

SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I? KENNETH R. CHRISTIANSEN

Page 5: Considering Tiling for a better User Interface

SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER

Page 6: Considering Tiling for a better User Interface

SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER1+ YEAR WORKING WITH WEBKIT

Page 7: Considering Tiling for a better User Interface

SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER1+ YEAR WORKING WITH WEBKITAPP & FRAMEWORK DEVELOPMENT AT

3+ YEARS WORKING WITH

Page 8: Considering Tiling for a better User Interface

SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER1+ YEAR WORKING WITH WEBKITAPP & FRAMEWORK DEVELOPMENT AT

3+ YEARS WORKING WITHNOKIA TECHNOLOGY

INSTITUTE, INdT, RECIFE, BRAZIL

Page 9: Considering Tiling for a better User Interface

A TILEDBACKING STORE

Page 10: Considering Tiling for a better User Interface

What are the problems we are facing today?The background for considering tiling

Page 11: Considering Tiling for a better User Interface

WE PAINT THINGS WE’VE ALREADY PAINTED WHEN

SCROLLING BACK

What are the problems we are facing today?The background for considering tiling

Page 12: Considering Tiling for a better User Interface

WE CALL INTO WEBCORE FOR EACH

PAINT...WE PAINT THINGS WE’VE ALREADY PAINTED WHEN

SCROLLING BACK

What are the problems we are facing today?The background for considering tiling

Page 13: Considering Tiling for a better User Interface

...CALLING INTO WEBCORE

HAS SOME OVERHEAD

WE CALL INTO WEBCORE FOR EACH

PAINT...WE PAINT THINGS WE’VE ALREADY PAINTED WHEN

SCROLLING BACK

What are the problems we are facing today?The background for considering tiling

Page 14: Considering Tiling for a better User Interface

FOR INSTANCE, CONSTRUCTING GRAPHICS

CONTEXT IS QUITE EXPENSIVE

...CALLING INTO WEBCORE

HAS SOME OVERHEAD

WE CALL INTO WEBCORE FOR EACH

PAINT...WE PAINT THINGS WE’VE ALREADY PAINTED WHEN

SCROLLING BACK

What are the problems we are facing today?The background for considering tiling

Page 15: Considering Tiling for a better User Interface

FOR INSTANCE, CONSTRUCTING GRAPHICS

CONTEXT IS QUITE EXPENSIVE

...CALLING INTO WEBCORE

HAS SOME OVERHEAD

WE CALL INTO WEBCORE FOR EACH

PAINT...WE PAINT THINGS WE’VE ALREADY PAINTED WHEN

SCROLLING BACK

CLEVER TILING CANSOLVE THESE ISSUES

What are the problems we are facing today?The background for considering tiling

Page 16: Considering Tiling for a better User Interface

Cache and join paint events

How to accomplish thisWhat is tiling anyway?

Page 17: Considering Tiling for a better User Interface

Cache and join paint events

How to accomplish thisWhat is tiling anyway?

Cache what you paint in image tiles

Page 18: Considering Tiling for a better User Interface

Cache and join paint events

How to accomplish thisWhat is tiling anyway?

Cache what you paint in image tiles

Blit the existing tiles on scroll

Page 19: Considering Tiling for a better User Interface

Cache and join paint events

How to accomplish thisWhat is tiling anyway?

Cache what you paint in image tiles

Blit the existing tiles on scroll

Don’t paint non-visible dirty areas immediately

Page 20: Considering Tiling for a better User Interface

Cache and join paint events

How to accomplish thisWhat is tiling anyway?

Cache what you paint in image tiles

Blit the existing tiles on scroll

Don’t paint non-visible dirty areas immediately

Avoid too many small tiles, due to the cost of constructing GraphicsContexts

Page 21: Considering Tiling for a better User Interface

Cache and join paint events

How to accomplish thisWhat is tiling anyway?

Cache what you paint in image tiles

Blit the existing tiles on scroll

Don’t paint non-visible dirty areas immediately

Avoid too many small tiles, due to the cost of constructing GraphicsContexts

This can be hardware accelerated!

Page 22: Considering Tiling for a better User Interface

My experimentMy basic algorithm

As an experiment, implement it on the WebKit side

Page 23: Considering Tiling for a better User Interface

My experimentMy basic algorithm

As an experiment, implement it on the WebKit side

Implemented only for QGraphicsWebView

Page 24: Considering Tiling for a better User Interface

My experimentMy basic algorithm

As an experiment, implement it on the WebKit side

Implemented only for QGraphicsWebView

Some changes needed elsewhere:

1) Render methods in abs. coordinates, without clipping

Page 25: Considering Tiling for a better User Interface

My experimentMy basic algorithm

As an experiment, implement it on the WebKit side

Implemented only for QGraphicsWebView

Some changes needed elsewhere:

1) Render methods in abs. coordinates, without clipping

2) Make ScrollView / FrameView send update events outside of the viewport

Page 26: Considering Tiling for a better User Interface

My experimentMy basic algorithm

As an experiment, implement it on the WebKit side

Implemented only for QGraphicsWebView

Some changes needed elsewhere:

1) Render methods in abs. coordinates, without clipping

2) Make ScrollView / FrameView send update events outside of the viewport

Why not just make viewport == contents size?

Page 27: Considering Tiling for a better User Interface

My experimentMy basic algorithm

As an experiment, implement it on the WebKit side

Implemented only for QGraphicsWebView

Some changes needed elsewhere:

1) Render methods in abs. coordinates, without clipping

2) Make ScrollView / FrameView send update events outside of the viewport

Why not just make viewport == contents size?

Because we use WebCore for drawing our scrollbarsand that makes a whole lot of sense ... theming ... etc

Page 28: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Basic algorithm

Page 29: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Basic algorithm

If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction)

Page 30: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Basic algorithm

If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction)

Put in cache, blit to screen

Page 31: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Basic algorithm

If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction)

Each tile stores it’s covered area as well as a dirty area.

Put in cache, blit to screen

Page 32: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Basic algorithm

If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction)

Each tile stores it’s covered area as well as a dirty area.

Put in cache, blit to screen

On update, we update the dirty area of the intersected tiles. If it has such an area already, the bounding rectis uses as the new area. Remember, we try to avoidcalling into WebCore unnecessarily

Page 33: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Basic algorithm

If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction)

Each tile stores it’s covered area as well as a dirty area.

Put in cache, blit to screen

On update, we update the dirty area of the intersected tiles. If it has such an area already, the bounding rectis uses as the new area. Remember, we try to avoidcalling into WebCore unnecessarily

Furthermore, if the dirty area == covered area, remove it from cache

Page 34: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Basic algorithm

Page 35: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Basic algorithm

On update and scroll:

Update all tiles, blit what is in the cache, create tiles for what is not

Page 36: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Basic algorithm

On update and scroll:

Update all tiles, blit what is in the cache, create tiles for what is not

That is more of less the basic algorithm, but there are some problems.

Page 37: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Problems

Page 38: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Problems

Scrollbars

Paint the scrollbars separately, make sure updates to them do not invalidate any tiles. These are not tiled in my implementation!

Page 39: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Problems

Scrollbars

Paint the scrollbars separately, make sure updates to them do not invalidate any tiles. These are not tiled in my implementation!

When enlarging we much make sure that we don’t cover an area already in the cache, so I need to knowwhat area is cached.

Enlarging tiles

Page 40: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Problems

Page 41: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Problems

Cache growth

Storing the whole page in the cache would be expensive, memory-wise, so the cache has a max size.

Page 42: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Problems

Cache growth

Storing the whole page in the cache would be expensive, memory-wise, so the cache has a max size.

The solution is to give each tile an age and increase the age when not used, and reset it when being blit.

Page 43: Considering Tiling for a better User Interface

My experimentMy basic algorithm

Problems

Cache growth

Storing the whole page in the cache would be expensive, memory-wise, so the cache has a max size.

The solution is to give each tile an age and increase the age when not used, and reset it when being blit.

Before adding a new tile, we reserve the needed space for it, removing the oldest tiles.

Page 44: Considering Tiling for a better User Interface

So what is the preliminary results?Did it really pay off?

Benjamin wrote a simple scrolling test app, and the results are quite promising.

Page 45: Considering Tiling for a better User Interface

So what is the preliminary results?Did it really pay off?

Benjamin wrote a simple scrolling test app, and the results are quite promising.

QWEBVIEW PERFORMS RESONABLE, DUE TO THE

XCOPYREGION

Page 46: Considering Tiling for a better User Interface

So what is the preliminary results?Did it really pay off?

Benjamin wrote a simple scrolling test app, and the results are quite promising.

QWEBVIEW PERFORMS RESONABLE, DUE TO THE

XCOPYREGION

THE TILING IS FASTER!

Page 47: Considering Tiling for a better User Interface

So what is the preliminary results?Did it really pay off?

Benjamin wrote a simple scrolling test app, and the results are quite promising.

QWEBVIEW PERFORMS RESONABLE, DUE TO THE

XCOPYREGION

But we had some surprises as well.

NON-TILED QGRAPHICSWEBVIEW IS EXTREMELY SLOW

THE TILING IS FASTER!

Page 48: Considering Tiling for a better User Interface

Where are se’ numbers?

Simple painting

Back up your claims dude!

QWebView

QGraphicsWebView

Tiled QGraphicsWebView

0 2,5 5,0 7,5 10,0

Milliseconds

Page 49: Considering Tiling for a better User Interface

Where are se’ numbers?

Slashdot page

Back up your claims dude!

QWebView

QGraphicsWebView

Tiled QGraphicsWebView

0 25 50 75 100

Milliseconds

Page 50: Considering Tiling for a better User Interface

Where are se’ numbers?

Amazon book page

Back up your claims dude!

QWebView

QGraphicsWebView

Tiled QGraphicsWebView

0 22,5 45,0 67,5 90,0

Milliseconds

Page 51: Considering Tiling for a better User Interface

Where are se’ numbers?

Wikipedia Qt page

Back up your claims dude!

QWebView

QGraphicsWebView

Tiled QGraphicsWebView

0 37,5 75,0 112,5 150,0

Milliseconds

Page 52: Considering Tiling for a better User Interface

Can’t you do better than that?Ideas for improvement

WE CAN!

Page 53: Considering Tiling for a better User Interface

Can’t you do better than that?

More realistic test suite!

Ideas for improvement

WE CAN!

Page 54: Considering Tiling for a better User Interface

Can’t you do better than that?

More realistic test suite!

Ideas for improvement

WE CAN!

My data structures are not that good, nor profiled

Page 55: Considering Tiling for a better User Interface

Can’t you do better than that?

More realistic test suite!

Ideas for improvement

WE CAN!

My data structures are not that good, nor profiled

Should be lower in the stack using WebCore constructs

Page 56: Considering Tiling for a better User Interface

Can’t you do better than that?

More realistic test suite!

Ideas for improvement

WE CAN!

My data structures are not that good, nor profiled

Should be lower in the stack using WebCore constructs

Paint in another thread, not block WebCore

Page 57: Considering Tiling for a better User Interface

Can’t you do better than that?

That is why we are here ;-) Now let’s get on to the work!

More realistic test suite!

Ideas for improvement

WE CAN!

My data structures are not that good, nor profiled

Should be lower in the stack using WebCore constructs

Paint in another thread, not block WebCore

Page 58: Considering Tiling for a better User Interface

Thanks for listening

KENNETH ROHDE CHRISTIANSENext-kenneth.christiansen@[email protected]