http live streaming
DESCRIPTION
Overview of Apple\'s HTTP Live Streaming (HLS)TRANSCRIPT
(Apple) HTTP Live Streaming
(HLS) Dynamically adapting video
streaming to network conditions
Auro Tripathy
Outline
• The basics of adaptive streaming
• A Wireshark view
• New tags defined by the HLS protocol
• A comparison with other adaptive
streaming protocols
November 8, 2011 Auro 2
HLS Streaming Architecture
November 8, 2011 Auro 3
Content Preparation
Media Encoder
Media Segmenter
HTTP Content Server
Index File
Segment
HTTP Cache
The Basics of HLS
• Protocol for transmitting alternate versions (i.e., bitrates) of the stream
• Media can be served soon after its created – Not actually “live”, but near real-time
• Specify a presentation with the URI of a playlist file
• A playlist file is an ordered list of media URI and info tags
• Each media URI refers to a media file which is a segment of a single contiguous file
4 Auro November 8, 2011
How does the client play the
stream?
• Client first obtains the playlist file
– Initial playlist file may contain alternates (see
next slide)
• Client decides which alternate playlist file
to play
• Gets and plays each media file in the
playlist.
5 Auro November 8, 2011
Server
Server Provides Alternate
Playlists to Clients
November 8, 2011 Auro 6
Alternate 1 Index File TS Segment
Alternate 2 Index File TS Segment
Alternate 3 Index File TS Segment
Index File
w/alternates
M3U format – the History
• M3U stands for MP3 URL.
• Files that contain the .M3U file-extension are MP3 playlist files – commonly referred to as Winamp playlist files
• Important to remember, this file does not contain media, only references to media
• Files whose name end with .m3u are encoded with US-ASCII.
• The unicode version of "m3u" is "m3u8", which uses UTF-8 unicode characters
7 Auro November 8, 2011
Extended M3U
• Defines two tags: EXTM3U & EXTINF
• First line must be #EXTM3U
– This distinguishes it from a M3U file
• EXTINF is a record marker that describes the media file identified by the URI that follows it.
• Example #EXTM3U
#EXTINF:<duration>, <title>
http://media.example.com/entire.ts
8 Auro November 8, 2011
Server Tasks to Achieve HLS
• Encode video in H.264 format and Single Program Transport Stream “container” – Encode in different bitrates
– taken from live feed or from a file
• Use a stream segmenter to generate short “chunks” of content – typically 10 seconds each - and generate a playlist file (m3u or m3u8) indicating from where to download the chunks
• Serve via HTTP server, and provide appropriate caching for performance
9 Auro November 8, 2011
Who supports HLS?
• Apple, of course
– Authored the protocol
http://tools.ietf.org/html/draft-pantos-http-live-
streaming-07
• Support available in Android 3.0
– http://developer.android.com/sdk/android-3.0-
highlights.html
10 Auro November 8, 2011
Deconstructing HLS with
Wireshark
Server Offering Multiple Playlist Files Different encodings of the same presentation
12 Auro November 8, 2011
Client request
Server offers four
alternative playlists at
200 Kbps, 311.1 Kbps,
484.4 Kbps & 737.8 Kbps
Client fetches the first Playlist File
13 Auro November 8, 2011
Client request 200 Kbps
playlist
Duration of segment in
seconds; appears once
Segment URLs
Note - Playlist file ends with a #EXT-X-ENDLIST
14 Auro November 8, 2011
Indicates no more media files will be added to the Playlist file
Client fetches the second Playlist File
15 Auro November 8, 2011
Client fetches the 3rd Playlist File
16 Auro November 8, 2011
Client fetches the 4th Playlist File
17 Auro November 8, 2011
Client requests a Media File to Play
18 Auro November 8, 2011
Summarizing the Sequence
November 8, 2011 Auro 19
Server Client
Respond with four alternate playlists
Location of all segments in Playlist
Location of all segments in Playlist
Respond with A/V data
VLC Plays HLS Test File http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8
20 Auro November 8, 2011
New Tags defined in HLS
HLS defines the following new tags Each with it own attribute list
• EXT-X-TARGETDURATION
• EXT-X-MEDIA-SEQUENCE
• EXT-X-KEY
• EXT-X-PROGRAM-DATE-TIME
• EXT-X-ALLOW-CACHE
• EXT-X-PLAYLIST-TYPE
• EXT-X-STREAM-INF
• EXT-X-ENDLIST
• EXT-X-DISCONTINUITY
• EXT-X-VERSION
22 Auro November 8, 2011
Tag – EXT-X-TARGETDURATION
• EXT-X-TARGETDURATION:<s>
– Specifies the maximum media file duration
– Appears only once in the playlist file
– <s> is an integer indicating the duration in
seconds
23 Auro November 8, 2011
Tag – EXT-X-MEDIA-SEQUENCE
• Each media file URI in a playlist must have
a unique integer number
• Sequence # := Sequence # of URI that
preceded it + 1
• Tag indicates the sequence number of the
first URI that appears in the playlist file
• Format
EXT-X-MEDIA-SEQUENCE:<number>
24 Auro November 8, 2011
Tag – EXT-X-KEY
• Contains information to decrypt media files that follow it
• Format #EXT-X-KEY:<attribute-list>
• METHOD attribute specifies encryption method (AES-128 or NONE)
• URI attribute identifies the Key file – Key file contains the cipher key to decrypt subsequent
media files in the Playlist
– Quoted string with URI
• IV attribute specifies the initialization vector to be used with the key
25 Auro November 8, 2011
Tag – EXT-X-PROGRAM-DATE-
TIME
• Associated with the next media file
• Should indicate a time zone
• Format EXT-X-PROGRAM-DATE-TIME:<YYYY-MM-DDThh:mm:ssZone>
26 Auro November 8, 2011
Tag – EXT-X-ALLOW-CACHE
• Indicates whether a client MAY or MUST-
NOT cache downloaded media files for
later playback.
• Tag applies to all segments in the playlist
• Format
#EXT-X-ALLOW-CACHE:<YES|NO>
27 Auro November 8, 2011
Tag - EXT-X-PLAYLIST-TYPE
• Optional tag provides mutability
information about the playlist file
• Format:
#EXT-X-PLAYLIST-FILE:<Event|VoD>
28 Auro November 8, 2011
Tag – EXT-X-ENDLIST
• Indicates no more media files will be
added to the Playlist file
• Occurs only once
• Format:
#EXT-X-ENDLIST
29 Auro November 8, 2011
Tag – EXT-X-STREAM-INF
• Tag indicates that the next URI in the playlist file indentifies another playlist file
• Format/example #EXT-X-STREAM-INF:<attribute-list>
<URI of playlist file>
• Defined attributes – BANDWIDTH
– PROGRAM-ID
– CODECS
– RESOLUTION
30 Auro November 8, 2011
Tag – EXT-X-DISCONTINUITY
• Tag indicates an encoding discontinuity between the media file that follows it and the one that preceded it
• Reason – File format change
– Number and type of tracks
– Encoding parameters
– Encoding sequence
– Timestamp sequence
• Format – #EXT-X-DISCONTINUITY
31 Auro November 8, 2011
Tag – EXT-X-VERSION
• Tag indicates the compatibility version of
the Playlist file
• One occurrence
• Format
#EXT-X-VERSION:<n>
32 Auro November 8, 2011
Media Files
• Media files are formatted as MPEG-2
transport streams
• It’s a singe program transport stream
• Should have a PAT/PMT at the start of
each files
• Client should be able to handle any
encoding parameter change such as a
resolution change
33 Auro November 8, 2011
Key Files
• The EXT-X-KEY tag has the URI and the
IV attributes to identify the key file and the
Initialization Vector
• If the IV is not present, client must use the
sequence number of the media file as the
IV
• Note, varying the IV increases the strength
of the cipher
34 Auro November 8, 2011
Conclusion – Lots of upside with
the protocol
• Uses the HTTP protocol
– Firewall-friendly
– HTTP server are ubiquitous thus reducing operational complexity
– Unifies the delivery for three screens (TV/Mobile/Tablet)
– Scales, no session state on the HTTP server
• Chunked transfers allow the client to compensate/optimize in near real-time for varying network conditions
November 8, 2011 Auro 35
Conclusion – Potential
Downsides
• Choice of MPEG-2 single program
transport streams
– 188 byte packet structure with 4-byte header
creates non-negligible overhead
• VCR-like trick-modes (seek to a time
forward or backward) can cause
noticeable delays if the segment size is
large
November 8, 2011 Auro 36