2016 Newt Global |www.NewtGlobal.com | Confidential

Newt Global Services and Offerings

Docker 101 Tri Series WebinarBy:Venkatnadhan ThirunalaiJai Karthik

Founded in 2004 ,HQ at Dallas TX, present in multiple locations in USA and India

Leader in DevOps Transformation, Cloud Enablement and Test Automation

One of top 100 fastest growing companies of Dallas twice in a row

Clientele includes Fortune 50 companies

About Newt Global

Speakers

05/01/2023 Copyright 4

• Venkat is DevOps Practice Leader, His area of expertise includes DevOps Practice, Consult Fortune 100 customers on DevOps IT Strategy. Responsible for building the global pre-sales, consulting and delivery team for NewtGlobal

• He has 16+ years of IT industry experience and delivered multiple enterprise scale projects for Fortune 500 customer base

Venkatnadhan ThirunalaiDevOps, Practice Leader

NewtGlobal

• AWS solution specialist, DevOps strategist. Area of expertise includes AWS infrastructure management and architectural design, Docker container management solution, DevOps strategy for automation, Ansible scripter for automation, Jenkins work practice for design architecture. Responsible for AWS management, Docker management and DevOps automation works with jenkins and ansible

• Industry experience of 6+ years in IT and worked on 24 projects with smooth deliverables for International/Natinonal enterprise clients

Jayakarthi DhanabalanAWS Solution Specialist

Newt Global

Housekeeping Instructions

• All phones are set to mute. If you have any questions, please type them in the Chat window located beside the

presentation panel

• We have already received several questions from the registrants, which will be answered by the speakers during

the Q & A session

• We will continue to collect more questions during the session as we receive and will try to answer them during

today’s session

• In case if you do not receive answers to your question today, you will certainly receive answers via email shortly

• Thanks for your participation and enjoy the session!

05/01/2023 Copyright 5

Contents

Introduction to Docker, Containers, and the Matrix from Hell

Why people care: Separation of Concerns

Technical Discussion

Ecosystem

Docker Basics

Dockerfile

Docker Compose

Docker Swarm

Static website

Web frontend

User DBQueue Analytics DB

Background workers

API endpoint

nginx 1.5 + modsecurity + openssl + bootstrap 2

postgresql + pgv8 + v8

hadoop + hive + thrift + OpenJDK

Ruby + Rails + sass + Unicorn

Redis + redis-sentinel

Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs

Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client

Development VM

QA server

Public Cloud

Disaster recoveryContributor’s laptop

Production Servers

The ChallengeM

ultip

licity

of S

tack

sM

ultip

licity

of

hard

war

e en

viro

nmen

ts

Production Cluster

Customer Data Center

Do services and apps interact

appropriately?

Can I migrate

smoothly and quickly?

The Matrix From Hell

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development VM QA Server Single Prod

ServerOnsite Cluster

Public Cloud

Contributor’s laptop

Customer Servers

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

Mul

tiplic

ity o

f Goo

dsM

ultip

ilici

ty o

f m

etho

ds fo

r tr

ansp

ortin

g/st

orin

g

Do I worry about

how goods interact

(e.g. coffee beans next to spices)

Can I transport quickly and sm

oothly(e.g. from

boat to train to truck)

Cargo Transport Pre-1960

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

Also a matrix from hell

Mul

tiplic

ity o

f Goo

dsM

ultip

licity

of

met

hods

for

tran

spor

ting/

stor

ing

Do I worry about

how goods interact

(e.g. coffee beans next to spices)

Can I transport quickly and sm

oothly(e.g. from

boat to train to truck)

Solution: Intermodal Shipping Container

…in between, can be loaded and unloaded, stacked, transported efficiently over long distances, and transferred from one mode of transport to another

A standard container that is loaded with virtually any goods, and stays sealed until it reaches final delivery.

Static website Web frontend User DB Queue Analytics DB

Development VM

QA server Public Cloud Contributor’s laptop

Docker is a shipping container system for code M

ultip

licity

of S

tack

sM

ultip

licity

of

hard

war

e en

viro

nmen

ts

Production Cluster

Customer Data Center

Do services and apps interact

appropriately?

Can I migrate

smoothly and quickly

…that can be manipulated using standard operations and run consistently on virtually any hardware platform

An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development VM QA Server Single Prod

ServerOnsite Cluster

Public Cloud

Contributor’s laptop

Customer Servers

Docker eliminates the matrix from Hell

Why Developers Care

Build once…(finally) run anywhere

A clean, safe, hygienic and portable runtime environment for your app.

No worries about missing dependencies, packages and other pain points during subsequent deployments.

Run each app in its own isolated container, so you can run various versions of libraries and other dependencies for each app without worrying

Automate testing, integration, packaging…anything you can script

Reduce/eliminate concerns about compatibility on different platforms, either your own or your customers.

Cheap, zero-penalty containers to deploy services? A VM without the overhead of a VM? Instant replay and reset of image snapshots? That’s the power of Docker

Why Devops Cares?

Configure once…run anything

Make the entire lifecycle more efficient, consistent, and repeatable

Increase the quality of code produced by developers.

Eliminate inconsistencies between development, test, production, and customer environments

Support segregation of duties

Significantly improves the speed and reliability of continuous deployment and continuous integration systems

Because the containers are so lightweight, address significant performance, costs, deployment, and portability issues normally associated with VMs

Why it works—separation of concerns

• Dan the Developer• Worries about what’s “inside” the container

• His code• His Libraries• His Package Manager• His Apps• His Data

• All Linux servers look the same

• Oscar the Ops Guy• Worries about what’s “outside”

the container• Logging• Remote access• Monitoring• Network config

• All containers start, stop, copy, attach, migrate, etc. the same way

More technical explanation

High Level—It’s a lightweight VM

Ow n pr ocess space

Ow n netw o rk int erface

Can r un s tu ff as r oo t

Can h ave i ts ow n /sb in /init (d ifferen t f ro m ho st)

<<mach ine co n tainer >>

Low Level—It’s chroot on steroidsCan also not have its own /sbin/init

Container=isolated processes

Share kernel with host

No device emulation (neither HVM nor PV) from host)

<<application container>>

Run everywhere Regardless of kernel version

Regardless of host distro

Physical or virtual, cloud or not

Container and host architecture must match*

Run anything If it can run on the host, it can run in the container

i.e. if it can run on a Linux kernel, it can run

WHY WHAT

AppA

Containers vs. VMs

Hypervisor (Type 2)

Host OS

Server

GuestOS

Bins/Libs

AppA’

GuestOS

Bins/Libs

AppB

GuestOS

Bins/Libs

App A’

Docker

Host OS

Server

Bins/Libs

App ABins/Libs

App B

App B’

App B’

App B’VM

Container

Containers are isolated,but share OS and, whereappropriate, bins/libraries

GuestOS

GuestOS

…result is significantly faster deployment, much less overhead, easier migration, faster restart

Why are Docker containers lightweight?

Bins/Libs

AppA

Original App(No OS to takeup space, resources,or require restart)

App Δ

Bins/

AppA

Bins/Libs

AppA’

GuestOS

Bins/Libs

Modified App

Copy on write capabilities allowus to only save the diffsBetween container A and containerA’

VMsEvery app, every copy of anapp, and every slight modificationof the app requires a new virtual server

AppA

GuestOS

Bins/Libs

Copy ofApp

No OS. CanShare bins/libs

AppA

GuestOS

GuestOS

VMs Containers

What are the basics of the Docker system?

Source Code

Repository

DockerfileFor

A

Docker Engine

DockerContainer

Image Registry

Build

Docker

Host 2 OS (Linux)

Container A

Container B

Container C

Container A

Push

Search Pull

Run

Host 1 OS (Linux)

Changes and Updates

Docker Engine

DockerContainer

Image Registry

Docker Engine

Push

Update

Bins/Libs

AppA

App Δ

Bins/

Base Container

Image

Host is now running A’’

Container Mod A’’

App Δ

Bins/

Bins/Libs

AppA

Bins/

Bins/Libs

AppA’’

Host running A wants to upgrade to A’’. Requests update. Gets only diffs

Container Mod A’

Some Docker vocabulary

Docker ImageThe basis of a Docker container. Represents a full application

Docker ContainerThe standard unit in which the application service resides and executes

Docker Engine Creates, ships and runs Docker containers deployable on a physical or virtual, host locally, in a datacenter or cloud service provider

Registry Service (Docker Hub or Docker Trusted Registry)Cloud or server based storage and distribution service for your images

22

Image Layers

23

Kernel

Alpine Linux

Install Python and Pip

Upgrade Pip

Copy Requirements

Install Requirements

…

24

Docker File System

• Logical file system by grouping different file system primitives into branches (directories, file systems, subvolumes, snapshots)

• Each branch represents a layer in a Docker image

• Allows images to be constructed / deconstructed as needed vs. a huge monolithic image (ala traditional virtual machines)

• When a container is started a writeable layer is added to the “top” of the file system

25

Copy on Write

• Super efficient:• Sub second instantiation times for containers• New container can take <1 Mb of space

• Containers appears to be a copy of the original image• But, it is really just a link to the original shared image

• If someone writes a change to the file system, a copy of the affected file/directory is “copied up”

26

What about data persistence?

• Volumes allow you to specify a directory in the container that exists outside of the docker file system structure

• Can be used to share (and persist) data between containers

• Directory persists after the container is deleted • Unless you explicitly delete it

• Can be created in a Dockerfile or via CLI

Basic Docker Commands

$ docker pull mikegcoleman/catweb:1.0

$ docker images

$ docker run –d –p 5000:5000 –-name catweb mikegcoleman/catweb:latest

$ docker ps

$ docker stop catweb (or <container id>)

$ docker rm catweb (or <container id>)

$ docker rmi mikegcoleman/catweb:latest (or <image id>)

$ docker build –t mikegcoleman/catweb:2.0 .

$ docker push mikegcoleman/catweb:2.0

Dockerfile – Linux Example

28

• Instructions on how to build a Docker image

• Looks very similar to “native” commands

• Important to optimize your Dockerfile

Dockerfiles

• Dockerfiles = image representations• Simple syntax for building images• Automate and script the images creation

FROM

• Sets the base image for subsequent instructions• Usage: FROM <image>• Example: FROM ubuntu• Needs to be the first instruction of every Dockerfile

• TIP: find images with the command: docker search

RUN

• Executes any commands on the current image and commit the results• Usage: RUN <command>• Example: RUN apt-get install –y memcached

FROM ubuntuRUN apt-get install -y memcached• Is equivalent to:docker run ubuntu apt-get install -y memcacheddocker commit XXX

docker build

• Creates an image from a Dockerfile• From the current directory = docker build• From stdin = docker build - < Dockerfile• From GitHub = docker build github.com/creack/docker-firefox

• TIP: Use –t to tag your image

Compose

• One binary to start/manage multiple containers and volumes on a single Docker host

• Originated from Fig

• Move your lengthy docker run commands to a YAML file

Installation

• Via pip• Or just get the binary

$ sudo curl -L https://github.com/docker/compose/releases/download/1.1.0/ docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

Use

$ docker-compose up -dCreating vagrant_mysql_1...Creating vagrant_wordpress_1...$ docker-compose psName Command State Ports ---------------------------------------------------------------------------------vagrant_mysql_1 /entrypoint.sh mysqld Up 3306/tcp vagrant_wordpress_1 /entrypoint.sh apache2-for ... Up 0.0.0.0:80->80/tcp

YAML description

wordpress: image: wordpress links: - mysql ports: - "80:80" environment: - WORDPRESS_DB_NAME=wordpress - WORDPRESS_DB_USER=wordpress - WORDPRESS_DB_PASSWORD=wordpresspwd mysql: image: mysql volumes: - /home/docker/mysql:/var/lib/mysql environment: - MYSQL_ROOT_PASSWORD=wordpressdocker - MYSQL_DATABASE=wordpress - MYSQL_USER=wordpress - MYSQL_PASSWORD=wordpresspwd

Machine

• One binary to create a remote Docker host and setup the TLS communication with your local docker client.

• Automates the TLS setup and the configuration of the local environment

• Can manage multiple machines in different clouds at the same time

Installation (e.g OSX)

$ wget https://github.com/docker/machine/releases/download/v0.1.0/docker-machine_darwin-amd64

$ mv docker-machine_darwin-amd64 docker-machine

$ chmod +x docker-machine

$ ./docker-machine --versiondocker-machine version 0.1.0

Local Use

$ docker-machine lsNAME ACTIVE DRIVER STATE URL SWARMdev * virtualbox Running tcp://192.168.99.100:2376 $ docker-machine env devexport DOCKER_TLS_VERIFY=1export DOCKER_CERT_PATH=/Users/sebastiengoasguen/.docker/machine/machines/devexport DOCKER_HOST=tcp://192.168.99.100:2376

$ docker imagesREPOSITORY TAG … CREATED VIRTUAL SIZEwordpress latest … 2 weeks ago 451.4 MBmysql latest … 2 weeks ago 282.8 MBmysql 5.5 … 2 weeks ago 214.5 MB

Cloud Use

• Many drivers• Many more waiting for merge (i.e cloudstack )

$ ./docker-machine create -d digitalocean foobarINFO[0000] Creating SSH key... INFO[0001] Creating Digital Ocean droplet... INFO[0005] Waiting for SSH... INFO[0072] Configuring Machine...

Swarm

• Docker client endpoint that proxies requests to docker daemons running in a cluster.

• Cluster manager that keeps state of the cluster nodes

• Easily run as a container itself

• Multiple service discoveries for cluster nodes (docker hosted, etcd, consul, zookeeper, file based)

Swarm

Use

• No install, run the container• docker pull swarm

$ docker run -v /vagrant:/tmp/vagrant -p 1234:1234 -d swarm manage file://tmp/vagrant/swarm-cluster.cfg -H=0.0.0.0:123472acd5bc00de0b411f025ef6f297353a1869a3cc8c36d687e1f28a2d8f422a06

$ docker -H 0.0.0.0:1234 infoContainers: 0Nodes: 3 swarm-2: 192.168.33.12:2375 └ Containers: 0 └ Reserved CPUs: 0 / 1 └ Reserved Memory: 0 B / 490 MiB…$ docker -H 0.0.0.0:1234 run -d -p 80:80 nginx

Use Machine to create Swarm

• Get a token for discovery• Start nodes with machine using the --swarm option

$ docker run swarm create31e61710169a7d3568502b0e9fb09d66$ docker-machine create -d virtualbox --swarm --swarm-master --swarm-discovery token://31e61710169a7d3568502b0e9fb09d66 head...$ docker-machine create -d digitalocean --swarm --swarm-discovery token://31e61710169a7d3568502b0e9fb09d66 worker-00...$ docker-machine create -d azure --swarm --swarm-discovery token://31e61710169a7d3568502b0e9fb09d66 swarm-worker-01

Put it all together: Build, Ship, Run Workflow

Developers IT Operations

BUILDDevelopment Environments

SHIPCreate & Store Images

RUNDeploy, Manage, Scale

Contact Us

For any questions/clarifications please contact:

Satheesh Reddy, Sales ManagerNewt Global Consulting LLC.

satheeshr@newtglobalcorp.comhttp://newtglobal.com/

05/01/2023 Copyright 46

05/01/2023 Copyright 47