mm-4099, adapting game content to the viewing environment, by noman hashim

25
ADAPTING GAME CONTENT TO THE VIEWING ENVIRONMENT WWW.APICAL.CO.UK 1

Upload: amd-developer-central

Post on 15-Jan-2015

1.372 views

Category:

Technology


2 download

DESCRIPTION

Presentation MM-4099, Adapting game content to the viewing environment, by Noman Hashim at the AMD Developer Summit (APU13) November 11-13, 2013.

TRANSCRIPT

Page 1: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

ADAPTING  GAME  CONTENT  TO  THE  VIEWING  ENVIRONMENT    

   

WWW.APICAL.CO.UK          

COPYRIGHT  APICAL  2012  

1  

Page 2: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

2   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

YOU  CAN'T  WATCH  A  MOVIE  BY  THE  POOL…CAN  YOU?  

2  

Amazon Kindle “Poolside” commercial Fall 2010

Page 3: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

3   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

 THE  CHALLENGE  OF  THE  MOBILE  VIEWING  ENVIRONMENT  

Difficult  for  emissive  display  to  compete  with  sun’s  brightness!    O[en  need  to  reduce  display  brightness  to  conserve  power  

3  

Page 4: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

4   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

PROBLEM  STATEMENT  

4  

Full    brightness  Dark  room  

Reduced  brightness  Dark  room  

Reduced    brightness  Bright  room  

Display  contrast  ra]o  is  affected  independently  from  the  bright  and  dark  ends  of  the  range    Display  contrast  ra]o  must  be  adapted  independently  

Page 5: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

5   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

CONTRAST  RATIO  CHANGES  FROM  1000:1  TO  50:1  WITH  AMBIENT  LIGHT  

5  

Source:  hap://www.displaymate.com/Tablet_ShootOut_3.htm  

Page 6: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

6   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

SOME  CONTENT  IS  HDR  WHILE  DISPLAY  IS  VERY  LDR  

6  

Page 7: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

7   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

THIS  ALSO  MEANS  THAT  DISPLAYS  NEED  TO  BE  BRIGHT  AND  BURN  POWER  

7  

0  

20  

40  

60  

80  

100  

120  

140  

160  

0   200   400   600   800   1000   1200   1400   1600   1800   2000  

Display  brightne

ss  (n

its)  

Ambient  light  (lux)  

Windows  8  recommended  

Excessive  brightness:    wastes  power  

Not  bright  enough  to  see  images/  movies/  games  

From  embedded  Ambient  Light  sensor  (ALS)  

Page 8: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

8   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

HOW  ABOUT  ADJUSTING  GAMMA?  

8  

Page 9: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

9   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

YOU  CAN’T  CHANGE  GAMMA  WITHOUT  CHANGING  COLORS  

9  

Increasing  gamma  makes  image  less  colorful  and  reduces  contrast  –”wash  out”  

Page 10: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

10   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL   10  

Asser]ve  Display  Core  

AD_CALC  

VIDEO  PIPELINE  iridix_display  gamma  LUT  

perceptual  dither    

ALS    

RGB   RGB  

ASSERTIVE  DISPLAY  

Page 11: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

11   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

ASSERTIVE  DISPLAY  

11  

Display  at  max  brightness:  contrast  ra]o  too  low  for  mul]media  

Content  adapted  to  display  contrast  ra]o  Same  power  consump]on  

Sample  condi]ons:  Ambient  light  1400  lux  –indoors  in  daylight  

Page 12: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

12   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

PLAY  AN  IMMERSIVE  GAME  IN  ANY  CONDITIONS  

12  

Sample  condi]ons:  Ambient  light  1400  lux  –indoors  in  daylight  

Display  at  max  brightness:  contrast  ra]o  too  low  for  mul]media  

Asser]ve  Display:  Content  adapted  to  display  contrast  ra]o  

Same  power  consump]on  

Page 13: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

13   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

THE  4  SCREENS  

13  

Ambient  light  200  lux.  Smartphone:  full  brightness.  TV:  Standard  Mode  

Asser]ve  Display  ON  (No  specific  calibra]on)  

Asser]ve  Display  OFF    

Page 14: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

14   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

WHY  IS  ASSERTIVE  DISPLAY  EFFECTIVE?  

 

 

 

It  enables  independent  adjustment  of  bright  and  dark  regions  of  each  frame  

 

So  it  decouples  brightness  from  contrast  raDo  

 

 

 

Brightness  can  be  set  according  to  preference  and  power  saving        Contrast  raDo  is  always  correct  for  op]mal  mul]media  viewability        

 

The  key  to  Asser]ve  Display  is:  

Page 15: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

15   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

ASSERTIVE  DISPLAY:  XBOX  CALL  OF  DUTY  

15  

Page 16: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

16   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

ASSERTIVE  DISPLAY:  NOKIA  LUMIA  1520  

16  

Page 17: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

17   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

IMPLEMENTATION  OF  ASSERTIVE  DISPLAY  ON  A  HETEROGENOUS  PROCESSOR  

Headline  results  on  Exynos  5410:  

 

 

 

17  

Frame  size   Maximum  frame  rate  (single  core)  

CPU  (Neon)  only   OpenGL  ES2.0   OpenCL  

1080P  (2MP)   4fps   28fps   45fps  

Page 18: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

18   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

VIDEO  PROCESSING  ENGINE:  PIXEL-­‐BY-­‐PIXEL  TONE  MAPPING  

18  

Page 19: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

19   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

VIDEO  PROCESSING  ENGINE:  PIXEL-­‐BY-­‐PIXEL  TONE  MAPPING  

Contrast  in  highlights  enhanced  

Contrast  in  midtones  preserved  

Contrast  in  shadows  enhanced  

iridix  calculates  a  unique  tone  curve  for  each  pixel  of  each  frame  

19  

Page 20: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

20   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

ASSERTIVE  DISPLAY  INVOLVES  A  GPU-­‐FRIENDLY  IMAGE  TRANSFORM  

y  Use  of  GPU  effec]ve  since  many  pixel  transforma]ons  can  be  done  in  parallel  

y  However,  transform  is  calculated  per-­‐pixel  adap]vely  on-­‐the-­‐fly,  so  beyond  simple  texture  mapping  

⇒  Requires  mul]ple  CPU  and  GPU  passes  with  intermediate  data  store  per  frame  

⇒  Main  reason  for  going  from    GL  to  CL  

20  

Page 21: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

21   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

CPU  memory  

LIMITATIONS  OF  OPENGL:  INTERMEDIATE  MEMORY  ACCESSES  

21  

GPU  memory  

Image  frame  

Resize  

Resized  frame   Resized  frame  

Transforma]on  LUT  

Transforma]on  LUT  

Upscaled  Transforma]on  LUT  

Transformed  frame  

Post-­‐processed  transforma]on  LUT  

DISPLAY  

Page 22: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

22   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

BENEFITS  OF  OPENCL  

1.  Intermediate  memory  accesses  

With  Open-­‐CL  compliant  hardware,  CPU  and  GPU  memory  can  be  shared,  removing  mul]ple  memory  transfers  required  by  the  algorithms  

 

2.  Intermediate  data  precision  

Even  if  input  data  is  8-­‐bit,  intermediate  calcula]ons  require  at  least  12  bit  precision.  OpenGL  is  inefficient  for  such  opera]ons  when  mul]ple  processing  stages  are  needed  

 

3.  Transforma]on  of  different  color  planes  

We  apply  a  different  transform  to  Y,U  and  V  planes.  In  OpenCL  these  can  be  done  by  a  single  kernel,  whereas  in  OpenGL  separate  kernels  are  required    

22  

Page 23: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

23   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

PERFORMANCE  COMPARISONS  

y  GPU-­‐based  processing  is  much  beaer  than  CPU,  but  dedicated  hardware  is  s]ll  best  ‒ Partly  because  HW  core  does  not  require  system  memory  access  

23  

AD  v3  ImplementaDon   Maximum  pixel  rate   Power  consumpDon    

at  1080P30  

Full  HW  IP  core  (28nm  TSMC  LP)   4K2K  at  60fps   3mW  

CPU  only  (Exynos  5410)   1080P  at  4fps   N/A  

OpenGL  ES2.0  (Exynos  5410)   1080P  at  28fps   112mW  

OpenCL  (Exynos  5410)   1080P  at  45fps   78mW  

Page 24: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

24   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

CONCLUSIONS  

y  An  immersive  experience  can  be  achieved  on  normal  displays  in  all  condi]ons  by  means  of  advanced  image  processing  

y  Latest  processors  with  CPU/GPUs,  including  those  in  PS4  and  Xbox  One,  can  run  these  kind  of  algorithms  in  real-­‐]me    

y  Extensions  of  this  approach  to  deliver  cinema-­‐like  experiences  for  both  video  and  gaming  are  in  development  

Page 25: MM-4099, Adapting game content to the viewing environment, by Noman Hashim

25   |      PRESENTATION  TITLE      |      NOVEMBER  14,  2013      |      CONFIDENTIAL  

DISCLAIMER  &  ATTRIBUTION  

The  informa]on  presented  in  this  document  is  for  informa]onal  purposes  only  and  may  contain  technical  inaccuracies,  omissions  and  typographical  errors.    

The  informa]on  contained  herein  is  subject  to  change  and  may  be  rendered  inaccurate  for  many  reasons,  including  but  not  limited  to  product  and  roadmap  changes,  component  and  motherboard  version  changes,  new  model  and/or  product  releases,  product  differences  between  differing  manufacturers,  so[ware  changes,  BIOS  flashes,  firmware  upgrades,  or  the  like.  AMD  assumes  no  obliga]on  to  update  or  otherwise  correct  or  revise  this  informa]on.  However,  AMD  reserves  the  right  to  revise  this  informa]on  and  to  make  changes  from  ]me  to  ]me  to  the  content  hereof  without  obliga]on  of  AMD  to  no]fy  any  person  of  such  revisions  or  changes.    

AMD  MAKES  NO  REPRESENTATIONS  OR  WARRANTIES  WITH  RESPECT  TO  THE  CONTENTS  HEREOF  AND  ASSUMES  NO  RESPONSIBILITY  FOR  ANY  INACCURACIES,  ERRORS  OR  OMISSIONS  THAT  MAY  APPEAR  IN  THIS  INFORMATION.    

AMD  SPECIFICALLY  DISCLAIMS  ANY  IMPLIED  WARRANTIES  OF  MERCHANTABILITY  OR  FITNESS  FOR  ANY  PARTICULAR  PURPOSE.  IN  NO  EVENT  WILL  AMD  BE  LIABLE  TO  ANY  PERSON  FOR  ANY  DIRECT,  INDIRECT,  SPECIAL  OR  OTHER  CONSEQUENTIAL  DAMAGES  ARISING  FROM  THE  USE  OF  ANY  INFORMATION  CONTAINED  HEREIN,  EVEN  IF  AMD  IS  EXPRESSLY  ADVISED  OF  THE  POSSIBILITY  OF  SUCH  DAMAGES.  

 

ATTRIBUTION  

©  2013  Advanced  Micro  Devices,  Inc.  All  rights  reserved.  AMD,  the  AMD  Arrow  logo  and  combina]ons  thereof  are  trademarks  of  Advanced  Micro  Devices,  Inc.  in  the  United  States  and/or  other  jurisdic]ons.    SPEC    is  a  registered  trademark  of  the  Standard  Performance  Evalua]on  Corpora]on  (SPEC).  Other  names  are  for  informa]onal  purposes  only  and  may  be  trademarks  of  their  respec]ve  owners.