media workloads on aws
TRANSCRIPT
Digital Media Workflows on AWS
Usman Shakeel
Principal Solutions Architect
Amazon Web Services
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Mobile ServicesDatabase
Storage & Content DistributionCompute & Networking
Application Services
Deployment & Management
AWS Global Infrastructure
Analytics
Applications
Partner
Solutions
Autoscaling S3 CloudFront Storage
GatewayAmazon
Glacier
Elastic Load
BalancingRoute 53
Regions
Edge
Locations
Elastic
Beanstalk
CloudWatch CloudFormationIdentity &
Access Management
OpsWorks
Simple Queue
Service
Simple Workflow
Service
Elastic
TranscoderCloudSearch Simple Email
Service
Elastic Cloud
ComputeElastic Block
Store
Virtual Private
Cloud
Direct
Connect
Virtual Private
Network
RDS DynamoDB Amazon
Redshift
ElastiCache
Libraries & SDKs
Management
Console
CloudTrail
AppStream
Elastic Map
Reduce
Command Line
Interface
Cli
Cognito Mobile
AnalyticsSimple Notification
Service
Amazon Kinesis AWS Data
Pipeline
Workspaces Zocalo
Import/
Export
Availability
Zone A
Availability
Zone B
Availability
Zone C
Amazon Web Services Platform
AWS Services Stack in a Media Workflow
AWS Direct
Connect
Elastic Load
Balancing
AWS Import/
Export
Amazon
S3AWS Storage
Gateway
Amazon
EBS
CloudFront
Amazon
CloudSearch Amazon
SQS
Amazon
Elastic
Transcoder
Amazon
EC2Amazon
EMRAmazon
VPC
Ingest/Create Store
Amazon
RDSElasti
CacheRoute
53
DeliverProcess
Amazon
EC2
Partner Solutions
Media Production & Distribution Value Chain
Shared IT Services
NetworkInfrastructure OperationsSecurity
Creative
Applications
Content
Distribution
Content
Processing
Content
Management
Modelling
Video editing
Post production
Rendering
visualization (i.e.
Maya, 3dsmax,
nuke etc.)
Color correction,
others
B2C
OTT
Broadcast hybrid
live-linear
VOD, PVR,
Catchup TV
Internet publishing
(i.e. B2C OVP)
B2B distribution
High speed ingest
Content/asset
management
Video and images
HSM
Collaboration and
shared storage
Archive
Workflow
Rendering
En/Transcode
Video QC
Packaging
Encryption,
watermarking
Digital Rights
Management
Customer
Engagement
Analytics
Content discovery
Recommendation/
Personalization
Targeting
Customer
experience
E-commerce /
Subscription
Media Ingest Solutions
AWS Import/
Export
Amazon S3
(multipart Upload)
AWS Storage
Gateway Service
AWS Direct
Connect
Partner SolutionsAWS Solutions
…
Media Storage Services
Amazon EBS
EBS
Block storage for use
with Amazon EC2
Amazon S3
Images
Videos
Files
Binaries
Snapshots
Internet scale
storage via API
AWS Storage Gateway
S3,
Amazon
Glacier
Integrates on-premises
IT and AWS storage
Amazon Glacier
Images
Videos
Files
Binaries
Snapshots
Storage for archiving
and backup
EC2
EBS
Media Processing Solutions
Amazon S3
Instance
Store
EC2 Instance AMI
EBS
Amazon Elastic
Transcoder
4K – H.264 outputs for the price of HD
PlayReady packaging for HLS and Smooth
HLS Content Protection
Encryption (integrated with KMS)
CloudTrail Integration
Media Operations – Other considerations
Amazon SNS(Simple Notification
Service)
Amazon SQS(Simple Queuing Service)
Amazon SWF(Simple Workflow)
Push notifications to:
Apple, Google, FireOS,
Windows & Android
with Baidu Cloud Push
Internet Connected
Devices & Distributed
Services
Fast, reliable, durable
and fully managed
Queue
Integrate with your apps
through an API
endpoint
Fully managed state
tracker and task
coordinator
Simple API calls that
can be called from your
code in any
programming language
and initiate tasks in or
outside the cloud
Fully managed
function service
Run your code based
on a notification
Pay for only running a
code snippet without
any server or
infrastructure builds
AWS Lambda(Cloud Functions)
Key Management Service
Fully managed service
to create and control
keys
Integration with EBS,
S3, ETS, Redshift etc.
Low cost, automated
way to manage and
integrate encryption
keys within applications
Reference Architecture – Digital Asset Management on AWS
AWS Import/ Export
S3 multi-part API
For Source, Renditions, Metadata Sidecar Files
Ingest
AWS Elastic
Beanstalk
DAM
Web Service
CloudFront
Distribution
Amazon
DynamoDB
Amazon
CloudSearch
DAM Catalog
Rendition Workers
EC2 ASG
Amazon SNS Topic
Amazon SQS Queue
Rendition Jobs
Amazon SQS Queue
Metadata Processing Jobs
S3 Notification
Elastic Transcoder
Proxy /
Thumbnail/Rendition
s
Generation
Processing
Metadata Workers
EC2 ASG
Reference Architecture – Digital Asset Management (Using
Lambda)
AWS Import/ Export
S3 multi-part API
For Source, Renditions, Metadata Sidecar Files
Ingest
AWS Elastic
Beanstalk
DAM
Web Service
CloudFront
Distribution
S3 Notification
Processing
AWS Lambda
Metadata
Extract & Persist
Amazon
DynamoDB
Amazon
CloudSearch
DAM Catalog
Elastic Transcoder
Proxy /
Thumbnail/Rendition
s
Generation
Reference Automated
Media Workflow
Input files
in S3
DRM Wrapper
DRM Wrapper (Spot)
SNS
AWS Import/ Export
S3 multi-part API
Streaming
Server
Amazon Elastic
Transcoder
Amazon SWF
S3 as backend storage for Content Files accessible
to other processing tasks
Reference Architecture – Content Processing Pipeline
(Using Lambda)
AWS Import/ Export
S3 multi-part API
S3 as backend storage for Content Files acesable to
other processing tasks
Amazon Elastic
Transcoder
S3 Notification
Trigger an AWS
Lambda Function to
Start a transcoding job
Ingest
S3 Notification
AWS Lambda
function to
generate a signed
URL to share the
file
Update CMS or
Metadata
Reference Architecture – Key Management, Content Encryption & Security
Key Management Service
Provide CPK for S3
encryption at rest
EC2, Elastic Transcoder can
request the data-key on behalf of
customer Store and deliver object
specific keys in Dynamo
S3 Ingest For Source, Renditions, Metadata Sidecar Files
Ingest
AWS Elastic
Beanstalk
Content
Consumption
CloudFront
Distribution
Amazon
DynamoDB
Individual Key Storage
Other Media
processing on EC2
Elastic
Transcoder PlayReady Packaging
for HLS and Smooth
Processing
Authentication/
Authorization
Content owner provides
the master key
Live
Amazon EC2
VOD
Amazon S3
OR
Custom Origin
Amazon CloudFront
Amazon EC2
OR
Elastic
Transcoder
Reference Architecture – Content Streaming
Signed URLs
Signed Cookies
Expiration Policies on URLs
Key Management Service
Reference Architecture – An OTT Workflow on AWS
EC2, ETS can request the data-key on behalf of customer
Provide CPK for S3 encryption at rest
AWS Import/ Export
S3 multi-part APIAmazon S3 For Source, Renditions, Metadata, Sidecar Files
Ingest
AWS Elastic
BeanstalkCloudFront
Distribution
DynamoDB CloudSearch
S3 Notification
Content Processing
Amazon SWF, SQS, SNS and
AWS Lambda can be used to
orchestrate the processing
workflow tasks
CloudFront
Distribution
User Logs
Amazon DynamoDB
Amazon Redshift
Users Connected Devices
Analytics and Recommendations Engine
Amazon EMR
Amazon
DataPipeline
Elastic Transcoder
EC2 ASGRendition Workers
EC2 ASGMetadata Workers
Metadata Proc queueRenditions Queue
Amazon Elastic
Transcoder
Rights Management
EC2 ASGELB
DRM Wrap
Encrypt
Package
Watermark
QCEC2 ASG
Amazon SNS Topic
DAM Web Service
Personalization Recommendations
EC2 ASGRDS
Front End Web Application
DAM Catalog
Content
Archive
Amazon
Glacier
Key Management ServiceContent owner provides
the master key
BI & Reporting
Users Web Browser
Content Owner