NEXT GENERATION IP VIDEO ARCHITECTURE
TRENDS, CHALLENGES, & SOLUTIONS
Weidong MaoSenior Fellow, Comcast
June 20, 2019
2
THE RISE OF OTT IP VIDEO
SUBSCRIPTION VOD SERVICE• Deep content library• Navigation, discovery, recommendation
VIRTUAL PAY TV SERVICE• Live linear TV programming • Video On Demand and Cloud DVR
ONLINE & SOCIAL MEDIA VIDEO• Short-form and long-tail content
3
TRADITIONAL VIDEO PROVIDERS GO ALL IP
LIVE LINEAR CHANNELS• Broadcast networks, cable networks, PPV, PEG
VIDEO ON DEMAND • Subscription VOD, Transactional VOD, and EST
CLOUD DVR
• Cloud recording/playback and download to go IN-HOME AND OUT-OF-HOME STREAMING
DIGITAL AD INSERTION (DAI)
EMERGENCY ALERT SYSTEM, CLOSED CAPTIONING, BLACKOUT
4
CLOUD DVR EXPANDS IN SCALE
CLOUD DVR (DIGITAL VIDEO RECORDER)• Example deployment with 500GB per subscriber
• ~200 hour of HD content at 5.5Mbps AVC ABR• Up to 6 simultaneous recordings per subscriber• In home streaming to multiple devices • Download to go
• Trending to TB scale of cloud DVR storage per customer• Unique copy requirement drives storage I/O and capacity
5
ENABLING TECHNOLOGIES FOR IP VIDEO
ENCODING (AVC, HEVC, AV1, VVC)
STREAMING (HLS, DASH)
STORAGE (ON PREMISE, CLOUD)
TRANSPORT (HTTP, RELIABLE UDP)
CONTENT DELIVERY NETWORK (HTTP PROXIES)
DIGITAL RIGHTS MANAGEMENT (MULTI-DRM / MPEG CENC)
DYNAMIC AD INSERTION (DAI) (SERVER & CLIENT SIDE)
7
IP VIDEO – SMALL SCREEN
IP VIDEO REACHES SECONDARY SCREEN OF ALL TYPES• U.S. adults spend more time on their mobile devices than in front of TV• OTT and managed video consumption increase significantly on mobile devices• Most devices support high end video format (e.g. 1080p60, 4K, HDR)
8
IP VIDEO – LARGE SCREEN
YET PRIMARY SCREEN IS DOMINATING AND IMPROVING • Higher resolution: from HD 1920x1080 to 4K 3840x2160p and 8K 7680x4320• Higher frame rate: from HD i60/p24/p30 to HD p60 to UltraHD p60/p120• High Dynamic Range (HDR): more bits for brightness/contrast nonlinearly• Wide Color Gamut: more colors for richer video viewing experience
9
US TV HOUSEHOLD PROJECTION
0
20
40
60
80
100
120
140
2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027
Total US TVHH Total HD TVHH Total 4K TVHHs Total 4K HDR TVHHs Total Cable, DBS, Telco HD Subscribers
4K and 4K HDR estimated projections (M)
Source: SNL Kagan, S&P Global Market Intelligence “US TV set and Blu-ray player projections through 2027”
87% of 2017 model year 4k TV’s have HDR support. 2018 model year and beyond 4k TVs all have HDR support
1 1
Codec MPEG-2 MPEG-4 AVC/H.264 HEVC/H.265
Partition size 16x16 Macroblock 16x16 Macroblock Coding Unit (64x64 to 8x8)
Prediction unit Inter 16x16, 16x8, 8x16Intra 8x8
Sub-block down to 4x4 Prediction Unit Quad Tree down to 4x4
Intra prediction DC predictor Up to 9 predictors Up to 35 predictors
Inter prediction Motion vector from one reference picture
Weighted prediction from multiple reference frames
Advanced Motion VectorPrediction (spatial + temporal)
Motion precision 1/2 pixel 1/2, and 1/4 pixel 1/4 and 1/8 pixel
Transform Floating point DCT Integer DCT (8x8, 4x4) Integer DCT (32x32 to 4x4)
Quantization Fixed weighted matrix Extended and finer control Extended and finer control
Entropy coding Variable Length Coding (VLC)
Context Adaptive Binary Arithmetic Coding (CABAC)
Context Adaptive Binary Arithmetic Coding (CABAC)
In-loop Filters None Deblocking Deblocking and SAO
MORE & BETTER TOOLS FOR VIDEO CODEC
1 2
CONTENT-ADAPTIVE ENCODINGCONSTANT QUALITY, VARIABLE BITRATE• IP video delivery enables Variable Bit Rate (VBR) or Capped Variable Bit Rate (cVBR)• Aggressive VBR helps balance the need for quality, reach, and infrastructure capacity• Per-shot content-adaptive encoding shown to perform better than fixed bitrate ladders
• More sophisticated player-side rate adaptation logic is needed
Bitrate
Quality Video Segment #1talking head
Equal Bitrate Allocation
ConsistentQuality Video Segment #2
action
1 3
VIDEO QUALITY METRICSPERCEPTUAL QUALITY METRICS ESSENTIAL FOR LARGE SCALE VIDEO SERVICES • Non-reference metrics as “gating criteria” for sources• Full-reference metrics to monitor and tune encoder performance• Full-reference metric as “target quality” for constant-quality encoding and streaming
1 5
• Choosing the bitrate/resolution pairs to make up/downshifts least visible• Picking the best segment durations• Modeling streaming dynamics for different genres
Content Preparation
• Providing network conditions to streaming clients• Achieving controlled fairness in the network• New transport options such as HTTP/2
Distribution and Delivery
• Modeling the impact of faster zapping and trick modes on the QoE • Understanding the impact of QoE on viewer engagement• Optimization across clients based on the stream utility
QoE Modeling and Client Design
• Understanding the interaction of adaptive streaming with caching in CDNs• Extracting actions based on real-time analytics• Fixing issues faster and remotely
Analytics, Fault Isolation and Diagnostics
ABR TREND – FOCUS AREAS
1 6
NETWORK IMPACTS OF IP VIDEO
CONCURRENCY• Number of simultaneous viewers for popular live/on-demand content
• High concurrency increases access network bandwidth consumption • High concurrency increases CDN caching efficiency, but also no. of connections
RELIABILITY • Packet loss in various network links (backbone, regional, access, in-home Wifi)
LATENCY • Live point latency, start up latency
1 7
VIDEO TRANSPORT OPTIONSMPEG-2 TS OVER UDP• Unreliable (UDP w/o FEC)• Extremely low latency• Multicast and unicast
ABR OVER HTTP • Reliable (TCP)• High latency• Unicast only
CMAF ABR OVER CHUNKED HTTP
• Reliable (TCP)
• Low latency
• Unicast
NEXT GENERATION TRANSPORT?
Bandwidth efficiency
Latency
Reliability
1 8
EVOLUTION OF VIDEO TRANSPORT
ReliableHigh latencyUnicast only
Next Generation Video TransportMPEG-2 TS over UDP
ABR over HTTP (DASH, CMAF)
1 9
CHOOSE THE RIGHT STORAGE SOLUTIONS
HIGH THROUGHPUT LOW LATENCY
HIGH COSTEncoding / Packaging
CDN CacheCloud DVR (hot)
MEDIUM PERFORMANCE HIGH VOLUME
LOW COST
VOD LibraryCloud DVR (warm)
HIGH AVAILABILITY & ELASTICITY
HIGH LATENCY LOWEST COST
VOD libraryCloud DVR (cold)
Metadata
Cloud Storage
On-prem HDDOn-prem SSD Cloud + SLA
~5GB/sec transfer speed~550K IOPs (r/w mix)~20 μs latency
~250 MB/sec transfer speed~80 IOPs~10 ms latency (incl. seek)
0.5-200 MB/sec transfer speedSLA pay-per-performance
2 0
IP VIDEO STANDARDS
HTTP 1.1/2/3, TCP, UDP, NTP, RTP, IPV4, IPV6MPEG AVC/H.264, MPEG HEVC/H.265, VVC, MPEG-H,MPEG DASH, SAND, MPEG CENC (Cbcs, Ctr), DASH-IF IOP, CPIX, DASH Events, ATSC 3.0ISO-BMFF, CMAF, MPEG-TS, ST 2022, ST 2110,SCTE 128-1/2, 215-1/2/1-1, 214-1/2/3/4, 223SCTE 35, 224, 250, 130ATSC A300, A341, A342, A343, A334, A330,ETSI 101 154, DVB-DASH, IPTVBT.2100-2, ST 2086, PQ, HLG, IMF, IMSC, SMPTE 2094-10, 2094-40, SL-HDR1, ST 2113, ISO/IEC 23091CSTF, DPCTF, HATF, Streaming Media QoE, 864.G, HDMI. QUIC, XML, HTML 5, H.770, H.730, H.740, H.750, H.760, H.7770, H.780, TR 26.969, TR 26.939, TR 26.907, ….
EncodingCreation
Indexing
Delivery
Playout
2 1
ML/AI APPLICATIONS TO IP VIDEO
VIDEO COMPRESSION• Scene detection and classification• Content adaptive encoding
CONTENT DELIVERY NETWORK• Intelligent caching algorithm
ABR STREAMING• Improve ABR quality of experience
CONTENT DISCOVERY• Voice remote based discovery• Smart video recognition and search
BIG DATA ANALYTICS
2 2
DEPLOYING LARGE SCALE IP VIDEO
Scalability Reliability Simplicity
Security Velocity Cost Effectiveness
Quality Operability Ubiquity
Multi-tenancy
Multi-tenancyM
ulti-
tena
ncy M
ulti-tenancy
2 3
SUMMARY
IP VIDEO IS BEING WIDELY DEPLOYED
SCALABILITY, RESILIENCY, AND QUALITY ARE STILL A CHALLENGE
ENABLING TECHNOLOGIES CONTINUE TO EVOLVE• Compression algorithms• Storage technologies• Network infrastructure• Transport protocols• Applications of ML/AI
STANDARDS, OPEN SOURCE, INDUSTRY/ACADEMIA COLLABORATION