amazon ecs

Download Amazon ECS

Post on 18-Jul-2015

1.315 views

Category:

Technology

3 download

Embed Size (px)

TRANSCRIPT

  • 2015, Amazon Web Services, Inc. or its aliates. All rights reserved

    Amazon EC2 Container Service: Manage Docker-Enabled Apps in EC2

    Ian Massingham AWS Technical Evangelist

    @IanMmmm

  • Agenda

    Containers

    EC2 Container Service

    Common patterns

    Demo

    Q&A

  • Containers

  • What are containers?

    OS virtualization

    Process isolation

    Automation

    Images Server

    Guest OS

    Bins/Libs Bins/Libs

    App2 App1

  • Container advantages

    Portable

    Server

    Guest OS

    Bins/Libs Bins/Libs

    App2 App1

  • Container advantages

    Flexible

    Server

    Guest OS

    Bins/Libs Bins/Libs

    App2 App1

  • Container advantages

    Fast

    Server

    Guest OS

    Bins/Libs Bins/Libs

    App2 App1

  • Container advantages

    Efficient

    Server

    Guest OS

    Bins/Libs Bins/Libs

    App2 App1

  • A container pipeline IT Operations

    Base Image

    Patches Utilities

  • A container pipeline IT Operations

    Base Image

    Patches Utilities

    Ruby

    Redis

    Logger

  • A container pipeline IT Operations Developer

    Base Image

    Patches Utilities

    Ruby

    Redis

    Logger

    App

  • A container pipeline

    Base Image

    Patches Utilities

    Ruby

    Redis

    Logger

    App

    IT Operations Developer

  • Server

    Guest OS

    Bins/Libs Bins/Libs

    App2 App1

  • EC2 Container Service Benefits

  • Easily manage clusters for any scale

    Nothing to run

    Complete state

    Control and monitoring

    Scale

  • Flexible container placement

    Applications

    Batch jobs

    Multiple schedulers

  • Designed for use with other AWS services

    Elastic Load Balancing Amazon Elastic Block Store

    Amazon Virtual Private Cloud AWS Identity and Access Management

    AWS CloudTrail

  • Extensible

    Comprehensive APIs

    Open source agent

    Custom schedulers

  • Common Patterns

  • Pattern 1: services and applications

    Simple to model

    Decompose to smaller (micro) services

    Blue / green deployments

  • Pattern 2: batch jobs

    Share pools of resources

    APIs provide cluster state

    Auto Scaling, Spot, Reserved Instances

  • EC2 Container Service terminology

  • Amazon EC2 instances

    Docker daemon

    Amazon ECS agent

    Key components: container instances

    https://github.com/aws/amazon-ecs-agent

  • Regional

    Resource pool

    Grouping of container instances

    Start empty, dynamically scalable

    Key components: clusters

  • Key components: task definitions

    Volume Definitions

    Container Definitions

  • Key components: task definitions

    Shared Data Volume

    PHP App Time of day App

  • Key components: task definitions

    Container Instance

    Schedule

    Shared Data Volume

    PHP App Time of day App

    Shared Data Volume

    PHP App

    Time of day App

  • Key components: task definitions { "environment": [], "name": "simple-demo", "image": "my-demo", "cpu": 10, "memory": 500, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ], "mountPoints": [ { "sourceVolume": "my-vol", "containerPath": "/var/www/my-vol" } ], "entryPoint": [ "/usr/sbin/apache2", "-D", "FOREGROUND" ], "essential": true },

    { "name": "busybox", "image": "busybox", "cpu": 10, "memory": 500, "volumesFrom": [ { "sourceContainer": "simple-demo" } ], "entryPoint": [ "sh", "-c" ], "command": [ "/bin/sh -c \"while true; do /bin/date > /var/www/my-vol/date; sleep 1; done\"" ], "essential": false }

  • { "environment": [], "name": "simple-demo", "image": "amazon/amazon-ecs-sample", "cpu": 10, "memory": 500, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ], "mountPoints": [ { "sourceVolume": "my-vol", "containerPath": "/var/www/my-vol" } ], "entryPoint": [ "/usr/sbin/apache2", "-D", "FOREGROUND" ], "essential": true },

    Key components: task definitions [ { "image": "mysql", "name": "db", "cpu": 10, "memory": 500, "essential": true, "entryPoint": [ "/entrypoint.sh" ], "environment": [ { "name": "MYSQL_ROOT_PASSWORD", "value": "pass" } ], "portMappings": [] }]

    Essential to our task

    Create and mount volumes

    Expose port 80 in container to port 80 on host

    10 CPU units (1024 is full CPU), 500 megabytes of memory

  • { "name": "busybox", "image": "busybox", "cpu": 10, "memory": 500, "volumesFrom": [ { "sourceContainer": "simple-demo" } ], "entryPoint": [ "sh", "-c" ], "command": [ "/bin/sh -c \"while true; do /bin/date > /var/www/my-vol/date; sleep 1; done\"" ], "essential": false }

    Key components: task definitions [ { "image": "tutum/wordpress-stackable", "name": "wordpress", "cpu": 10, "memory": 500, "essential": true, "links": [ "db" ], "entryPoint": [ "/bin/sh", "-c" ], "environment": [ ], "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] }, ]

    From Docker Hub

    Mount volume from other container

    Command to exec

  • Unit of work

    Grouping of related containers

    Run on container instances

    Key components: tasks

  • Key components: run a task

    Good for short-lived containers, for example batch jobs

  • Key components: create a service

    Good for long-running applications and services

  • Microservices and elastic resource pools with ECS

    Boyan Dimitrov, Platform Automation Lead @ Hailo @nathariel

  • Microservices intro

    Discovery Configuration A/B testing capabilities Monitoring & Instrumentation and much more

    Each service (at Hailo) gets for free:

    Service B

    Service A

    Service C

    Service E

    Service D

    Small, self-contained units of execution with well defined API Built around business capabilities or domain objects Responsible for one thing and one thing only Fully automated lifecycle

    AWS Summits 2015

    Monolith App

  • What do we have

    AWS Summits 2015

    Microservices ecosystem based on Go Designed specifically for the cloud dierent building blocks and components

    will constantly be in flux, broken or unavailable

    1000+ AWS instances spanning multiple regions 200+ services in production

  • Service interactions not as scary as it looks!

    AWS Summits 2015

  • Service deployment at present

    Main goals: Reliability, Ease of Use, Resource Eciency

    AWS Summits 2015

    Each service is decoupled from the rest and deployed individually We run multiple services on the same instance We rely on auto scaling groups for organizing and scaling our workload We use static partitioning to match a service to an auto scaling group An automated deployment system takes care of all service lifecycle details

  • Provisioning Service

    Provisioning Service

    CI Pipeline

    Amazon S3

    Provisioning Manager

    Provisioning Service

    Docker Registry

    Deployment overview and our journey towards containers

    Instance Instance

    Process Container

    Auto Scaling Group Auto Scaling Group

  • How hard is to deploy a service?

    service name version

    auto scaling group

    AWS Summits 2015

  • Is this good enough?

    Main goals: Reliability, Ease of Use, Resource Eciency

    service name version

    auto scaling group How do I figure this one out? Would my service live there forever? What if my team owns 20+ services ?

    As a developer:

    AWS Summits 2015

  • Provisioning Service

    What about resource eciency?

    35% Utilization

    85% Utilization

    Auto Scaling Group A

    Auto Scaling Group B

    AZ eu-west-1a AZ eu-west-1c AZ eu-west-1b

    instance instance instance instance instance instance

    instance instance instance

    Main goals: Reliability, Ease of Use, Resource Eciency

    AWS Summits 2015

  • Challenges

    AWS Summits 2015

    Our overall utilization across the services auto scaling groups is between 25% and 50%

    Performance of individual services is way more complex than simple CPU and memory cal