docker deployment options - bangalore container conference
TRANSCRIPT
Compare Docker Deployment Options in Public Cloud
Presenter Name: Sreenivas Makam
Presented at: Container conference, Bangalore
Presentation Date: April 7, 2017
About me
• Senior Engineering Manager at Cisco Systems Data Center group
• Author of “Mastering CoreOS” https://www.packtpub.com/networking-and-servers/mastering-coreos/ )
• Docker Captain(https://www.docker.com/community/docker-captains )
• Blog: https://sreeninet.wordpress.com/
• Code hacks: https://github.com/smakam
• LinkedIn: https://in.linkedin.com/in/sreenivasmakam
• Twitter: @srmakam
Agenda
• Deployment options – Overview
• Methodology used to compare
• Comparison of Deployment options
• Pick the right option for your use case
Considerations for Container solution
Which Orchestration solution to choose?
Should it be deployed in VM or bare-metal?
How to run stateful applications?How to achieve Service discovery and DNS?
Should it be deployed on premise or public cloud?
Is central logging required?
How to monitor and troubleshoot container platform?
Is the solution secure?
How to inject secrets and dynamic configuration?
Docker deployment in public cloud
Docker Machine(for AWS, Azure, GCE)
Cloud provider’s CaaS(AWS ECS, Google GKE,
Microsoft ACS)
Docker’s custom cloud(Docker for AWS, Azure,
GCE)
Docker’s CaaS (Docker cloud, Docker datacenter)
Docker deployment
Docker Machine
• Tool from Docker to create and manage Docker nodes.
• Supported for majority of cloud providers including AWS, Azure and Google cloud.
• Creates Docker node on the cloud with 1 command.
• Latest Docker version is pre-installed in the node.
• Both old and new Swarm modes can be used to create clusters.
• Provides automatic secure access to Docker node.
• Minimal integration to native cloud provider features.
• Targeted as a developer tool.
Docker custom cloud
• This is Docker’s solution to get Containers integrate well with the cloud infrastructure.
• Docker for AWS and Azure is available for general use. Docker for GCE is available as beta currently.
• Swarm mode cluster gets automatically created using a cloud provider template and with few simple inputs from the user.
• Solution integrated with Cloud networking, storage, logging, security group, load balancer.
• Targeted for hybrid cloud or multi-cloud deployments.
Docker CaaS• Container service offered by Docker.
• Docker Datacenter(DDC) - Docker’s enterprise grade container platform
• Docker cloud - Hosted service from Docker to manage Containers
• DDC can be run on-premise or in any of the major public cloud. Official support is present now for AWS and Azure. GCE would be added soon.
• UCP and DTR are main components of DDC and they can be deployed in a highly available manner.
• DDC provides enterprise grade features like high availability, RBAC and LDAP integration.
• DDC and Docker cloud provides nice user interface for management and is also compatible with regular Docker API.
• Docker cloud is targeted as a simple hosted solution for relatively small deployments.
• DDC is targeted as a complete Container platform since it includes all associated services like service discovery, logging, networking, storage with a decent cloud integration.
Docker datacenter - Components
https://www.docker.com/enterprise-edition
Docker datacenter for AWS - Architecture
https://docs.docker.com/datacenter/install/aws/
Docker datacenter for Azure - Architecture
https://blog.docker.com/2016/06/docker-datacenter-aws-azure-cloud/
Cloud provider CaaS
• Container service offered by Cloud providers.• ECS – From Amazon(Docker with Amazon’s proprietary orchestrator)• GKE – From Google(Docker with Kubernetes orchestrator)• ACS – From Microsoft(Docker with either Swarm, Kubernetes, Mesos)
• Provides very tight cloud integration.
• In some cases, there seems to be a lag with latest Docker version and features available with this option.
• Targeted for folks who want to manage VMs and Containers together. This solution suits companies which have a big cloud presence and already using public cloud.
Amazon ECS - Architecture
Uses proprietary orchestrator currently. There is a plan to make orchestrator as pluggable module in the future.
http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html
Azure Container service- Architecture
• Supports major orchestrators like Docker Swarm, Kubernetes, Mesos.• ACS only takes care of installing and deploying the cluster. Management of
containers and services is done by orchestrator itself.
https://docs.microsoft.com/en-in/azure/container-service/container-service-intro
GKE Architecture
Kubernetes Architecture GKE Architecture – Built on Docker and Kubernetes
• GKE makes Kubernetes easier to manage and integrates well with Google compute’s other cloud services.
https://www.cloudbees.com/blog/demand-jenkins-slaves-kubernetes-and-google-container-enginehttps://www.slideshare.net/wattsteve/kubernetes-48013640
Methodology used to compare
• Built multi-node cluster in all cases. Tried to use Swarm mode in most of the cases where it is supported. In some cases, used custom orchestrator, old Swarm mode or Kubernetes where Swarm mode is not supported.
• Deployed multi-container voting application using Docker containers in the multi-node cluster.
• Accessed the externally exposed services using load balancer where it is available.
• Tried scaling up/down services as well as scaling up/down the cluster.
• Checked features like logging capability, service discovery, high availability, networking and storage.
• Evaluated ease of management, upgradability and the cloud integration.
• Details are captured here(https://github.com/smakam/dockerdeploy)
Multi-Container Voting app
https://github.com/docker/example-voting-app
Comparison parameter - Orchestration
• Both old and new Swarm modes can be used.Docker
Machine
• Swarm modeDocker’s
custom cloud• ECS – Amazon’s proprietary scheduler• GKE – Kubernetes• ACS – Supports Docker Swarm, Kubernetes,
Mesos
Cloud provider’s CaaS
• DDC – Swarm mode• Docker cloud – proprietary orchestration and
Swarm mode, Swarm mode available as beta
Docker’s CaaS
Comparison parameter - Management
• Docker-machine interface for managing cluster. Container management using Docker tools. Docker Machine
• Cluster creation using cloud provider template, Cluster management using Cloud provider tools, Container management using Docker tools.
Docker’s custom cloud
• ECS – ECS GUI and CLI• GKE – Cluster management using Google cloud,
Kubernetes dashboard to manage Containers• ACS – Cluster management using Azure cloud,
Container management using Docker tools.
Cloud provider’s CaaS
• DDC – UCP and DTR has GUI to manage, Supports Docker API.
• Docker cloud – GUI to manage. Supports Docker API.Docker’s CaaS
Comparison parameter – Networking & Storage
• Libnetwork with overlay for networking. Docker volume plugin supported by cloud provider can be used.
Docker Machine
• Libnetwork with overlay for networking. Cloudstor volume plugin is supported currently.
Docker’s custom cloud
• ECS – Overlay network not supported. Docker volume is supported with limited drivers
• GKE – Kubernetes based networking. Storage through Kubernetes persistent disk and google cloud storage
• ACS – Libnetwork with overlay for networking. Docker volume driver using Azure file storage is supported.
Cloud provider’s CaaS
• Libnetwork with overlay for networking. Docker volume plugin supported by cloud provider can be usedDocker’s CaaS
Comparison parameter – Registry
•Docker hub is default. Can be used with any Docker registry.
Docker Machine
•Can be used with any Docker registry.
Docker’s custom cloud
•ECS –Docker registry and Amazon’s ECR•GKE – Docker registry and Google’s container registry•ACS – Docker registry and Microsoft’s ACR
Cloud provider’s CaaS
•DDC – DTR•Docker cloud – Docker cloud registry and any other Docker registry
Docker’s CaaS
Comparison parameter – Cloud integration
•Minimal cloud integration.Docker Machine
•Integrated with cloud networking, firewall, load balancer, logging. This can be improved further.
Docker’s custom cloud
•Provides native integration of Containers to other cloud offerings from provider.
Cloud provider’s CaaS
•DDC – Good integration with cloud services.•Docker cloud – Swarm mode has minimal integration now. Non-swarm mode has better integration.
Docker’s CaaS
Comparison parameter – Application file format
• Compose formatDocker
Machine
• Compose formatDocker’s
custom cloud
•ECS – Proprietary task definition format, limited compose support•GKE – Kubernetes task definition format•ACS – Compose can be used with Docker Swarm
Cloud provider’s CaaS
•DDC – Compose format•Docker cloud – Compose format, supported stackfile format earlier which is similar to compose
Docker’s CaaS
Comparison parameter – Docker version & upgrade
•Uses CE latest version 17.03. No easy way to use older versions. •Docker upgrade has to be done manually.
Docker Machine
•Uses CE latest version 17.03. No easy way to use older versions. •Rolling upgrade is supported.
Docker’s custom cloud
• ECS – Docker version 1.12.6. No flexibility with Docker version. Container agent upgrade will also upgrade Docker version. Upgrade is manual.
• GKE – Docker 1.11.2, Kubernetes 1.5.4. Rolling upgrade is supported.
• ACS – CE version 17.03. No option to upgrade the cluster, suggestion is to create new cluster.
Cloud provider’s CaaS
• DDC – Uses EE 17.03 version. Need to upgrade UCP, DTR and Docker engine separately. No rolling upgrade yet.
• Docker cloud – Uses CE latest version 17.03. Provides integrated upgrade option.
Docker’s CaaS
Comparison parameter – Enterprise features
•Enterprise features has to be manually setup.Docker Machine
•Enterprise features has to be manually setup
Docker’s custom cloud
•Integrated with cloud provider’s enterprise features like user management, high availability.
Cloud provider’s CaaS
•DDC – Highly available. Provides RBAC and LDAP integration.•Docker cloud – Supports organizations and teams. Docker’s CaaS
Comparison parameter - Cost
•Cost is for cloud resource usage, no separate cost for Containers
Docker Machine
•Cost is for cloud resource usage, no separate cost for Containers
Docker’s custom cloud
•AWS, ACS - Cost is for cloud resource usage, no separate cost for Containers•GKE – Chargeable after 5 Container nodes.
Cloud provider’s CaaS
•DDC – Separate cost for Container nodes.•Docker cloud – Separate charge for private repositories
Docker’s CaaS
How do I pick the right option for me?Production use case
VM and Container
usedDocker CaaS
Enterprise grade DDC
Docker CloudHybrid/Multi cloud
Cloud provider CaaS
Kubernetes GKEAzure
ECSAzure
Docker customcloud
Docker Machine
yes
yes
yesyes
yesno
no
no
no
no
Caution:This flowchart oversimplifies the problem. There are more considerations than this.
References• https://github.com/smakam/dockerdeploy
• https://sreeninet.wordpress.com/2016/09/03/docker-for-aws-deployment-options/
• https://docs.docker.com/machine/drivers/aws/
• https://docs.docker.com/machine/drivers/azure/
• https://docs.docker.com/machine/drivers/gce/
• http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html
• https://docs.microsoft.com/en-in/azure/container-service/container-service-intro
• https://cloud.google.com/container-engine/
• https://docs.docker.com/docker-for-aws/
• https://docs.docker.com/docker-for-azure/
• https://docs.docker.com/docker-for-aws/
• https://beta.docker.com/docs/gcp/
• https://docs.docker.com/docker-cloud/
• https://www.docker.com/enterprise-edition
Thank you!
Any questions?