mm-4099, adapting game content to the viewing environment, by noman hashim
DESCRIPTION
Presentation MM-4099, Adapting game content to the viewing environment, by Noman Hashim at the AMD Developer Summit (APU13) November 11-13, 2013.TRANSCRIPT
ADAPTING GAME CONTENT TO THE VIEWING ENVIRONMENT
WWW.APICAL.CO.UK
COPYRIGHT APICAL 2012
1
2 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
YOU CAN'T WATCH A MOVIE BY THE POOL…CAN YOU?
2
Amazon Kindle “Poolside” commercial Fall 2010
3 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
THE CHALLENGE OF THE MOBILE VIEWING ENVIRONMENT
Difficult for emissive display to compete with sun’s brightness! O[en need to reduce display brightness to conserve power
3
4 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
PROBLEM STATEMENT
4
Full brightness Dark room
Reduced brightness Dark room
Reduced brightness Bright room
Display contrast ra]o is affected independently from the bright and dark ends of the range Display contrast ra]o must be adapted independently
5 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
CONTRAST RATIO CHANGES FROM 1000:1 TO 50:1 WITH AMBIENT LIGHT
5
Source: hap://www.displaymate.com/Tablet_ShootOut_3.htm
6 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
SOME CONTENT IS HDR WHILE DISPLAY IS VERY LDR
6
7 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
THIS ALSO MEANS THAT DISPLAYS NEED TO BE BRIGHT AND BURN POWER
7
0
20
40
60
80
100
120
140
160
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Display brightne
ss (n
its)
Ambient light (lux)
Windows 8 recommended
Excessive brightness: wastes power
Not bright enough to see images/ movies/ games
From embedded Ambient Light sensor (ALS)
8 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
HOW ABOUT ADJUSTING GAMMA?
8
9 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
YOU CAN’T CHANGE GAMMA WITHOUT CHANGING COLORS
9
Increasing gamma makes image less colorful and reduces contrast –”wash out”
10 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL 10
Asser]ve Display Core
AD_CALC
VIDEO PIPELINE iridix_display gamma LUT
perceptual dither
ALS
RGB RGB
ASSERTIVE DISPLAY
11 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
ASSERTIVE DISPLAY
11
Display at max brightness: contrast ra]o too low for mul]media
Content adapted to display contrast ra]o Same power consump]on
Sample condi]ons: Ambient light 1400 lux –indoors in daylight
12 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
PLAY AN IMMERSIVE GAME IN ANY CONDITIONS
12
Sample condi]ons: Ambient light 1400 lux –indoors in daylight
Display at max brightness: contrast ra]o too low for mul]media
Asser]ve Display: Content adapted to display contrast ra]o
Same power consump]on
13 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
THE 4 SCREENS
13
Ambient light 200 lux. Smartphone: full brightness. TV: Standard Mode
Asser]ve Display ON (No specific calibra]on)
Asser]ve Display OFF
14 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
WHY IS ASSERTIVE DISPLAY EFFECTIVE?
It enables independent adjustment of bright and dark regions of each frame
So it decouples brightness from contrast raDo
Brightness can be set according to preference and power saving Contrast raDo is always correct for op]mal mul]media viewability
The key to Asser]ve Display is:
15 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
ASSERTIVE DISPLAY: XBOX CALL OF DUTY
15
16 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
ASSERTIVE DISPLAY: NOKIA LUMIA 1520
16
17 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
IMPLEMENTATION OF ASSERTIVE DISPLAY ON A HETEROGENOUS PROCESSOR
Headline results on Exynos 5410:
17
Frame size Maximum frame rate (single core)
CPU (Neon) only OpenGL ES2.0 OpenCL
1080P (2MP) 4fps 28fps 45fps
18 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
VIDEO PROCESSING ENGINE: PIXEL-‐BY-‐PIXEL TONE MAPPING
18
19 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
VIDEO PROCESSING ENGINE: PIXEL-‐BY-‐PIXEL TONE MAPPING
Contrast in highlights enhanced
Contrast in midtones preserved
Contrast in shadows enhanced
iridix calculates a unique tone curve for each pixel of each frame
19
20 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
ASSERTIVE DISPLAY INVOLVES A GPU-‐FRIENDLY IMAGE TRANSFORM
y Use of GPU effec]ve since many pixel transforma]ons can be done in parallel
y However, transform is calculated per-‐pixel adap]vely on-‐the-‐fly, so beyond simple texture mapping
⇒ Requires mul]ple CPU and GPU passes with intermediate data store per frame
⇒ Main reason for going from GL to CL
20
21 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
CPU memory
LIMITATIONS OF OPENGL: INTERMEDIATE MEMORY ACCESSES
21
GPU memory
Image frame
Resize
Resized frame Resized frame
Transforma]on LUT
Transforma]on LUT
Upscaled Transforma]on LUT
Transformed frame
Post-‐processed transforma]on LUT
DISPLAY
22 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
BENEFITS OF OPENCL
1. Intermediate memory accesses
With Open-‐CL compliant hardware, CPU and GPU memory can be shared, removing mul]ple memory transfers required by the algorithms
2. Intermediate data precision
Even if input data is 8-‐bit, intermediate calcula]ons require at least 12 bit precision. OpenGL is inefficient for such opera]ons when mul]ple processing stages are needed
3. Transforma]on of different color planes
We apply a different transform to Y,U and V planes. In OpenCL these can be done by a single kernel, whereas in OpenGL separate kernels are required
22
23 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
PERFORMANCE COMPARISONS
y GPU-‐based processing is much beaer than CPU, but dedicated hardware is s]ll best ‒ Partly because HW core does not require system memory access
23
AD v3 ImplementaDon Maximum pixel rate Power consumpDon
at 1080P30
Full HW IP core (28nm TSMC LP) 4K2K at 60fps 3mW
CPU only (Exynos 5410) 1080P at 4fps N/A
OpenGL ES2.0 (Exynos 5410) 1080P at 28fps 112mW
OpenCL (Exynos 5410) 1080P at 45fps 78mW
24 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
CONCLUSIONS
y An immersive experience can be achieved on normal displays in all condi]ons by means of advanced image processing
y Latest processors with CPU/GPUs, including those in PS4 and Xbox One, can run these kind of algorithms in real-‐]me
y Extensions of this approach to deliver cinema-‐like experiences for both video and gaming are in development
25 | PRESENTATION TITLE | NOVEMBER 14, 2013 | CONFIDENTIAL
DISCLAIMER & ATTRIBUTION
The informa]on presented in this document is for informa]onal purposes only and may contain technical inaccuracies, omissions and typographical errors.
The informa]on contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, so[ware changes, BIOS flashes, firmware upgrades, or the like. AMD assumes no obliga]on to update or otherwise correct or revise this informa]on. However, AMD reserves the right to revise this informa]on and to make changes from ]me to ]me to the content hereof without obliga]on of AMD to no]fy any person of such revisions or changes.
AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION.
AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
ATTRIBUTION
© 2013 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo and combina]ons thereof are trademarks of Advanced Micro Devices, Inc. in the United States and/or other jurisdic]ons. SPEC is a registered trademark of the Standard Performance Evalua]on Corpora]on (SPEC). Other names are for informa]onal purposes only and may be trademarks of their respec]ve owners.