evolution of multimedia & display mazen salloum 26 feb 2015
TRANSCRIPT
EVOLUTION OFMULTIMEDIA & DISPLAY
MAZEN SALLOUM26 FEB 2015
2| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
AGENDA
4K Video‒ Emergence of 4K Devices‒ 4K Formats‒ 4K Video Compression & Decode ‒ 4K Graphics‒ 4K & Display Interfaces
Adaptive-Sync‒ Video Playback‒ Graphics Rendering & Current Sync Issues‒ Power Saving
3| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
OVERVIEW
Multimedia and display components of embedded systems are constantly evolving‒ New compression and display standards‒ New types of content
Focus on following specific topics‒ 4K video content‒ Adaptive-Sync
4K Video
5| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
EMERGENCE OF 4K OR ULTRA-HIGH DEFINITION (UHD)
Migration towards higher resolutions than now common High Definition (HD) 4K is four times number of pixels compared to HD’s 1080p format 4K presents much better image quality and finer detail
‒ Perception of detail depends on distance from display
4K panels are higher density (or PPI), as compared to HD, which offers smoother looking image, especially at closer viewing distances
6| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
4K is used across different types of devices‒ High-end monitors and large-size digital signage displays‒ All-in-one PCs‒ Tablets‒ Consumer TVs‒ Notebooks‒ Digital cinema projectors‒ Capture devices: Professional and prosumer camcorders and cameras
Content distribution websites streaming at 4K formats
4K DEVICES AND USAGE
7| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
4K & UHD FORMATS
UHD is a TV standard, defined by ITU, and it covers two resolutions1
‒ UHD-1 is 3840x2160 (8.29 megapixels)‒ 4x number of pixels of High Definition’s 1080p‒ Aspect ratio of 16:9
‒ UHD-2 is 7680x4320 (33.18 megapixels)‒ 16x number of pixels compared to High Definition’s 1080p‒ Sometimes referred to as 8K
4K is mainly tied to film projectors and Digital Cinema Initiative (DCI), with following formats2
‒ DCI 4K (native resolution) is 4096×2160 (8.84 megapixels)‒ Wider aspect ratio compared to UHD-1
‒ DCI 4K (CinemaScope cropped) is 4096×1716 (7.02 megapixels)‒ DCI 4K (flat cropped) is 3996×2160 (8.63 megapixels)
1) "Ultra High Definition Television: Threshold of a new age". ITU press release. May 2012
2) “Digital Cinema Initiatives”. Wikipedia
8| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
4K VIDEO COMPRESSION
HD content is most commonly compressed using H.264 Given size of 4K video content, it requires higher video compression ratios than H.264 Much 4K video will be encoded using High Efficiency Video Coding (HEVC)
‒ Standard co-developed by ITU-T and ISO/IEC‒ HEVC, also referred to as H.265‒ Offers up to double compression rates compared to H.264
Another video compression standard commonly used is VP9‒ VP9 is mostly leveraged today for streaming applications, and is widely supported by mainstream Internet browsers
9| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
4K VIDEO DECODE
Increased compression rates require much higher processing power (compared to H.264) Decoding 4K content, compressed using HEVC or VP9, would consume the capabilities of a typical CPU
‒ Bitrate and other factors also affect playback performance of systems
Desirable that systems possess a dedicated engine that is designed to decode 4K content Larger system memory and bigger memory bandwidth
15Mbps 30FPS (Main Profile) 15Mbps 30FPS (High Profile) 30Mbps 30FPS (Main Profile) 30Mbps 30FPS (High Profile) 50Mbps 30FPS (Main Profile) 50Mbps 30FPS (High Profile)0%
10%20%30%40%50%60%70%80%90%
100%
Average CPU Utilization During Playback (UHD Content)AMD A10 APU & FIREPRO W7100 GPU
AMD A10 APUFIREPRO W7100 GPU
CPU
Util
izati
on (%
)
Dedicated Hardware Decode
CPU Decode
System Specs AMD A10-7800 Radeon R7, 12CU, 3.5GHz 16GB DDR3 2133MHz AMD Catalyst™ Omega Driver-14.12-with-dotnet45-win8.1-64bit
10| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
Decode Pipeline
VIDEO PLAYBACK PIPELINE
Internet
Syst
em
Mem
ory
Network Adapter
USB / SATA Controller
CPU
Fixed-FunctionDecoder
Post Processing( CSC, Scaling, other Filters )
CPU
GPU Compute
Units
Display Output
GPU Display
Controller
Faster Network Bandwidth
Higher Storage Capacity
Faster Interface
Higher Memory Bandwidth & Capacity
Faster CPUs
New Silicon
Faster CPUs New Display Interface
Faster GPUs
11| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
Entire graphics pipeline is affected‒ Geometry shaders‒ Vertex shaders‒ Frame buffers
All required to increase in size to handle 4K formats Larger Graphics memory and bigger memory bandwidth
4K GRAPHICS
Applications Vertex Shader
Geometry Shader
Fragment Shader
Buffer Processing
Graphics Pipeline
12| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
HDMI and DisplayPort (DP) are popular display interfaces Initial versions of HDMI and DP support HD resolutions at 60Hz and 4K formats
only at lower refresh rate of 30Hz‒ Initial versions’ maximum bandwidth are less than what is needed to output 4K at 60Hz
Both interface standards have been updated recently, with higher bandwidth Key versions that can display 4K content
‒ HDMI 1.4 supports up to 4K at 30Hz‒ HDMI 2.0 supports up to 4K at 60Hz‒ DP1.1 supports up to 4K at 30Hz‒ DP1.2 supports up to 4K at 60Hz‒ DP1.3 supports up to 3840x2160 (UHD-1) at 120Hz, or 7680x4320 (UHD-2) at 60Hz
4K AND DISPLAY INTERFACES
13| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
DISPLAY DEVICES RESOLUTION AND INTERFACES
Resolutions and display interfaces vary by type of device 4K TVs support up to 3840x2160 (UHD) resolutions
‒ HDMI 1.4 or 2.0 interfaces as inputs
PC monitors support up to 3840x2160 resolutions‒ Both DP and HDMI as inputs
4K projectors are mostly at 4096×2160 resolution‒ Inputs are HDMI, SDI or DVI interfaces
Adaptive-Sync
15| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
ADAPTIVE-SYNC
Before Adaptive-Sync:‒ Displays refresh at constant rate of 60Hz
‒ Regardless of content being displayed‒ Irrespective of capabilities of graphics source it’s interfacing with
‒ Causes issues such as jittery video playback and jerky graphics
With Adaptive-Sync:‒ New standard Video Electronics Standards Association (VESA®)1
‒ Adaptive-Sync was introduced as part of DisplayPort 1.2a in mid-2014‒ Also part of DP 1.3, as an optional feature
‒ With Adaptive-Sync, monitor no longer controls its refresh rate‒ Instead GPU refreshes monitor based on frequency of generated content, or its graphics rendering rate‒ Solves video playback and graphics issues above, plus offers power saving
1) “VESA DisplayPort Adaptive-Sync White Paper”, Syed Athar Hussain and Shane Parfitt. VESA, May 2013
16| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
ADAPTIVE-SYNC & VIDEO PLAYBACK
Before Adaptive-Sync:‒ Video is typically played back at 24 or 25 Hz
‒ Or multiples of 24/25 Hz: 48/50 and 72/75‒ Other rates are also used, e.g. 23.98, 29.97 or 30Hz
‒ Judder effect occurs when 24Hz video — or multiples of it — is displayed at 60Hz‒ Particularly noticeable in panning scenes or when objects are moving against a static background
‒ Judder reduction techniques are frequently performed in video post processing stages to resolve this
With Adaptive-Sync:‒ These techniques are no longer required with Adaptive-Sync; monitor would update based on natural flow of video
content, e.g. it would refresh at 48Hz ‒ Lowers processing system requirements, since video post processing portion is eliminated‒ Reducing monitor’s refresh rate from 60 to 48Hz (or 50Hz), cuts power consumption
17| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
Many GPUs process graphics workloads variably, depending on how performance intensive they are‒ GPUs don’t render frames at regular, fixed cadence of 60Hz
Mismatch between GPU’s rendering rate and monitor’s set refresh rate, causes visual quality issues When rendering rate is lower than 60Hz, stuttering effect is introduced:
‒ GPU hasn’t finalized rendering of next frame (II), but is forced to output current frame again (I)‒ When monitor displays same frame twice, it creates visually stuttering effect
GRAPHICS RENDERING & STUTTERING
Stuttering effect
Display I
Render I Render III Render IV
Display I Display II Display III Display IV
TIME
Render II
18| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
Stuttering issue can be removed by forcing GPU to output as soon as it finishes rendering of next frame (II)‒ Often occurs midway through monitor’s refresh cycle‒ Eliminates stuttering, but introduces tearing effect
‒ Portion of each of the 2 frames is being displayed simultaneously on screen
Tearing effect
Render I Render II Render III Render IV
Display I Display II Display III Display IV
TIME
GRAPHICS RENDERING & TEARING
Display I Display II
19| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
GRAPHICS RENDERING & ADAPTIVE-SYNC
Adaptive-Sync resolves both stuttering and tearing issues When GPU is taking longer to render frame than monitor’s fixed refresh rate, monitor would hold off from
refreshing until it receives new frame (II)‒ Monitor is fully synchronized with GPU, presenting frames as soon as they available or rendered by GPU
Stutter & Lag
Display I
Render I Render III Render IV
Display I Display II Display III Display IV
TIME
Render II
Render I Render II Render III Render IV
Display I Display II Display III Display IV
TIME
Without Adaptive-Sync:Stutter and Lag
With Adaptive-Sync:No Stutter and Lag
20| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
ADAPTIVE-SYNC & POWER SAVING
Before Adaptive-Sync‒ Applicable when content being displayed is static
‒ Today, panels would still refresh at 60Hz, as there’s no detection of content type‒ Each refresh consumes power
With Adaptive-Sync‒ Panels refresh at their lowest supported rate‒ Power saving is particularly important for battery-operated devices
21| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
SUMMARY
4K and UHD formats are becoming more prevalent, across multitude of devices 4K video, with its higher compression schemes, puts strain on entire video pipeline Video decoding engine desired to support new compression schemes Creates new demands on graphics processors, system memory, and display interfaces Adaptive-Sync provides dynamic synchronization between GPUs and monitors New standard enables smoother visual experiences
‒ Eliminates judder during video playback‒ Resolves stuttering and tearing issues in case of graphics‒ Offers power savings in case of static content
22| EVOLUTION OF MULTIMEDIA & DISPLAY| FEBRUARY 26, 2015 |
DISCLAIMER & ATTRIBUTION
The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors.
The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes.
AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION.
AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
ATTRIBUTION
© 2015 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo, Catalyst and combinations thereof are trademarks of Advanced Micro Devices, Inc. in the United States and/or other jurisdictions. Other names are for informational purposes only and may be trademarks of their respective owners.
THANK YOU