there's more to docker than the container · there's more to docker than the container...

There's More to Docker than the Container The Docker Platform

Fabio Chiodini

dotnext Team – Dell EMC @FabioChiodini

github.com/kacole2 github.com/FabioChiodini

Kendrick Coleman

@KendrickColeman {code} by Dell EMC

© Copyright 2017 Dell Inc. 2

Open source at Dell EMC

– Contribute to meaningful OSS projects – Create new thought leading OSS applications – Drive awareness of OSS opportunities with Dell EMC

product teams – Participate in relevant community engagement projects – Act in the interest of building a community

{code} by Dell EMC is a group of passionate open source engineers and advocates working to build a community around software-based infrastructure.

Platinum Sponsor

© Copyright 2017 Dell Inc. 3

© Copyright 2017 Dell Inc. 4

© Copyright 2017 Dell Inc. 5

Monolithic vs. Microservices

© Copyright 2017 Dell Inc. 6

Applications Are Changing

Loosely Coupled Services

Many Small Servers

~2000 Today

Monolithic

Big Servers

Slow changing Rapidly

updated

© Copyright 2017 Dell Inc. 7

Challenge: The Matrix From Hell

Virtual machines

Server Public Cloud

Disaster Recovery

Developer Laptop

Server Cluster

Data Center

Static Website

Web Front End Background Workers

User DB Analytics DB

Queue API Endpoint

Development Test & QA Production Scale Out

© Copyright 2017 Dell Inc. 8

2013: Enter The Docker Container

• Packages up software binaries and dependencies

• Isolates software from each other

• Container is a standard format

• Easily portable across environment

• Allows ecosystem to develop around its standard

© Copyright 2017 Dell Inc. 9

Eliminating The Matrix

Static Website

Web Front End

Background Workers

User DB Analytics DB

Queue API Endpoint

Any App Anywhere

Composable Dynamic Portable

© Copyright 2017 Dell Inc. 10

Docker Image The basis of a Docker container

Docker Container The standard unit in which the application service resides

Docker Engine Creates, ships and runs Docker containers deployable on physical or virtual host locally, in a datacenter or cloud service provider

Docker Registry/Hub On-premises registry or Docker Hub for image storing and collaboration

Docker Basics

DEMO Docker basics

© Copyright 2017 Dell Inc. 12

What Just Happened? docker run –d –-name webserver –p 5000:80 nginx

Docker Hub

Dockerfile

############################################################ FROM debian:jessie MAINTAINER NGINX Docker Maintainers "docker-maint@nginx.com" ENV NGINX_VERSION 1.9.15-1~jessie RUN apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 \ && echo "deb http://nginx.org/packages/mainline/debian/ jessie nginx" >> /etc/apt/sources.list \ && apt-get update \ && apt-get install --no-install-recommends --no-install-suggests -y \ ca-certificates \ nginx=${NGINX_VERSION} \ nginx-module-xslt \ nginx-module-geoip \ nginx-module-image-filter \ nginx-module-perl \ nginx-module-njs \ gettext-base \ && rm -rf /var/lib/apt/lists/* # forward request and error logs to docker log collector RUN ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log EXPOSE 80 443 CMD ["nginx", "-g", "daemon off;"]

Github

© Copyright 2017 Dell Inc. 13

Cui Prodest?

• Clean and portable runtime environment • No missing dependencies during

deployments • Run each app in its own isolated container • Improves speed and reliability of CI/CD

systems • Eliminates overhead issues normally

associated with VMs

DEV - Build once, run anywhere

• Universal Packaging • Built-in orchestration • Secure by default • App centric networking and service

discovery • Extensible architecture to work with 3rd

party integrations like storage and networking with no code changes

OPS - Configure once, run anything

© Copyright 2017 Dell Inc. 14

Containers “vs” VMs App

A App

A App A’

Guest OS

Guest OS

Guest OS

Bins/Libs

Bins/Libs

App

A

Bins/Libs

App A

Original app

Copy of app

App A’

V M s

C O N T A I N E R S

Modified app

Hypervisor

Infrastructure

Operating System

Infrastructure

Container Runtime

© Copyright 2017 Dell Inc. 15

DOCKER != CONTAINERS

© Copyright 2017 Dell Inc. 16

Docker =! Containers

Docker Engine

Linux Container

Registry

docker build ……. docker push ……. docker pull ……… docker run ……...

Docker (Engine) provides

application life cycle capabilities

Containers provide a mechanism to instantiate the code (shipped

as a docker image)

*Borrowed from@mreferre

© Copyright 2017 Dell Inc. 17

DOCKER != DOCKER

© Copyright 2017 Dell Inc. 18

3/2/17 https://news.ycombinator.com/item?id=13774295

© Copyright 2017 Dell Inc. 19

CONTAINERD != DOCKER

© Copyright 2017 Dell Inc. 20

CONTAINERD SUPPORTS THE DOCKER PLATOFRM

© Copyright 2017 Dell Inc. 21

DOCKER != CLOUD NATIVE

© Copyright 2017 Dell Inc. 22

THE CLOUD NATIVE LANDSCAPE

*From Cloud Native Computing Foundation (CNCF)

© Copyright 2017 Dell Inc. 23

Docker in 2017

Open Source Project* • 3300 contributors • 43K+ GitHub stars • 900K+ Dockerized apps • 240 Meetups in 70 countries • 95K Meetup members • 12B+ Docker Image Downloads • 14M Docker hosts

Containers as a Service provider • Integrated platform for dev and IT • Commercial technical support Docker project sponsor • Primary sponsor of Docker project • Supports project maintainers

Dockercon 2017 • Closed with 5500 attendees (4k in 2016)

The docker Project Docker Inc

*The original docker project is now under the name of Moby: https://github.com/moby/moby

© Copyright 2017 Dell Inc. 24

CE vs EE DOCKER COMMUNITY EDITION FEATURES

• The latest Docker version with integrated tooling to build, test and run container apps

• Available for free with software maintenance for the latest shipping version

• Integrated and optimized for developer desktops, Linux servers and clouds

• Monthly Edge and quarterly Stable release channels available

• Native desktop or cloud provider experience for easy onboarding

• Unlimited public and one free private repo storage as a service *

• Automated builds as a service *

• Image scanning and continuous vulnerability monitoring as a service *

© Copyright 2017 Dell Inc. 25

CE vs EE

DOCKER ENTERPRISE EDITION FEATURES

• Certified Infrastructure provides an integrated environment for enterprise Linux (CentOS, Oracle Linux, RHEL, SLES, Ubuntu) Windows Server 2016 and Cloud providers like AWS and Azure

• Certified Containers provide trusted ISV products packaged and distributed as Docker containers - built with secure best practices cooperative support

• Certified Plugins provide networking and volume plugins and easy to download and install containers to the Docker EE environment.

• REX-Ray in the Docker Store!

© Copyright 2017 Dell Inc. 26

Why Does Dell EMC Care?

• Databases – Postgres, MongoDB, MySQL, MariaDB, Redis, Cassandra

• Search, Analytics, Messaging – ElasticSearch, LogStash, Kafka, RabbitMQ

• Content Management – Wordpress, Joomla, Drupal, SugarCRM

• Service Discovery – Consul, Zookeeper, etcd

• Continuous Integration and Delivery – Jenkins, GitLab, SonarQube, Selenium, Nexus

• Custom Applications – That Java app your company built

Stateful and persistent applications

© Copyright 2017 Dell Inc. 27

What's the problem?

• When I run a persistent application in a container, where does my data get stored?

– The container holds the data directory and structure of the entire application

– Optionally use local volumes

• Stateless applications work well – nginx, httpd, kibana, haproxy,

memcached, solr, celery

$ docker r un - v r edi sDat a: / dat a r edi s

redisData

/etc /var /bin /opt /data

© Copyright 2017 Dell Inc. 28

What's the problem?

• Lose a container – Lose the data

• Lose a server – Lose the data

• Local data storage – Failed hard drives or failed RAID – Can not scale beyond the physical

limit of the server

/etc /var /bin /opt /data

© Copyright 2017 Dell Inc. 29

Introducing REX-Ray

REX-Ray

The leading container storage orchestration engine enabling persistence for cloud native workloads

rexray.codedellemc.com

• Cloud Native Interoperability

• Open Source

• Enterprise Ready – High Availability – CLI Intuitiveness – Effortless Deployment – Architectural Choices

• Multi-Platform Storage Management – Storage agnostic (block/file/object)

© Copyright 2017 Dell Inc. 30

Solving the problem

• REX-Ray is installed and configured on all hosts in the cluster as a stateless service

• REX-Ray acts as storage plug-in and container engines re-direct storage operations to REX-Ray

– Create/Mount/Unmount/Delete/Snapshot

$ docker r un - - vol ume- dr i ver =r exr ay - v r edi sDat a: / dat a r edi s

/redisData

/etc /var /bin /opt /data

© Copyright 2017 Dell Inc. 31

Solving the problem

• Lose the container or lose the server

– Data persists and remains intact on the remote storage platform

/etc /var /bin /opt

/redisData

© Copyright 2017 Dell Inc. 32

Solving the problem

• Attach the volume to a new container on a different host

– Equivalent of a hard reset. Application starts and resumes from the last write to disk

– Container schedulers can perform automated failover

• Scalability – Application data can scale to the

maximum supported by the storage platform

/etc /var /bin /opt /data /redisData

DEMO REX-Ray = Container Persistence

© Copyright 2017 Dell Inc. 34

Docker For…

© Copyright 2017 Dell Inc. 35

Introducing Docker Infrakit

Declarative, Self-Healing Configuration

Common Infrastructure Experience

Declarative & Self-Healing Infrastructure

Groups

Cluster Configuration Comprised of Instances & Flavor Combination

Instances

Logical Cluster Instance. i.e. EC2 Image, Vagrant VM, Physical Host

Flavors

Role & Healthcheck. i.e. Zookeeper, Swarm Node, etc.

Consistent User Experience for Docker Everywhere

RackHD

What Makes RackHD Different?

DISCOVERY & CATALOGING

TELEMETRY & GENEALOGY

DEVICE MANAGEMENT

CONFIGURATION PROVISIONING

FIRMWARE MANAGEMENT

LOGGING ENVIRON-MENTALS

FAULT DETECTION

ANALYTICS DATA

RackHD Capabilities

© Copyright 2017 Dell Inc. 40

Docker Infrakit + RackHD Process Flow

WORKFLOW API

MONGODB

RAB

BIT

MQ

DH

CP

PXE

/ TFT

P

SYSL

OG

IPM

I / O

BM

RACKHD

INFRAKIT

GR

OU

P

DOCKER INFRAKIT

INST

ANC

E

FLAV

OR

https://github.com/codedellemc/infrakit.rackhd

© Copyright 2017 Dell Inc. 41

How to get Started

1. PXE Boot Bare Metal for Passive Discovery

2. Define a server SKU

3. Create a custom workflow, if needed

Discover Hardware

1. Configure an Infrakit Cluster Configuration

2. Run Infrakit Provisioning

Provision Clusters

https://github.com/codedellemc/infrakit.rackhd

© Copyright 2017 Dell Inc. 42

DOCKER SWARM (swarmkit)

• Provides native clustering capabilities to turn a group of Docker engines into a single, virtual Docker Engine.

• Scale out your application as if it were running on a single, huge computer

DEMO Docker Swarm

© Copyright 2017 Dell Inc. 44

Learn More

• Cloud Native Training – https://github.com/dotnext/training

• {code} Labs – https://github.com/codedellemc/labs

• Join {code} Community – community.codedellemc.com/

© Copyright 2017 Dell Inc. 45

codedellemc.com

community.codedellemc.com

@codeDellEMC

blog.codedellemc.com

{code} by Dell EMC is a group of passionate open source engineers and

advocates working to build a community around software-based infrastructure.

rexray.codedellemc.com

github.com/codedellemc/labs

HOL01 Use REX-Ray & ScaleIO w/ Docker, Mesos and Kubernetes

Kendrick Coleman

@KendrickColeman github.com/kacole2

Fabio Chiodini

@FabioChiodini github.com/FabioChiodini