linux container technology inside docker with rhel7

Linux Container Technologyinside Docker with RHEL7

Etsuji NakaiSenior Solution Architect

and Cloud EvangelistRed Hat K.K

v1.0 2015/06/22

2

Linux Container Technology inside Docker with RHEL7

Contents

What is Docker? Container Technology inside Docker Architecture of Kubernetes References

What is Docker?

4


Quick Demo!

5


Dockerfile

① Auto-build Docker images

OS Image

Application Library / Framework

Application Binary

Describe steps to build an image

Dockerimage

Everything you need to run applicationis included in the image

② Upload and publish images

③ Download and run

What you can do with Docker

Container Technologyinside Docker

7


"Linux Container" is a Linux kernel feature to contain a group of processes in an independent execution environment.

Linux kernel provides an independent application execution environment for each container including:– Independent filesystem.– Independent network interface and IP address.–Usage limit for memory and CPU time.

Linux Kernel

Use

r Pr

oces

s

・・・

Physical Host / VMPhysical Host / VM

OS

ContainerNo Container

Use

r Pr

oces

s

Use

r Pr

oces

s

User Space

Linux Kernel

Use

r Pr

oces

s

Use

r Pr

oces

s

User Space

Use

r Pr

oces

s

Use

r Pr

oces

s

User Space

・・・

What is container technology?

Container

8


Container supports separation of various resources. They are internally realized with different technologies called "namespace."– Filesystem separation　 → Mount namespace (kernel 2.4.19) – Hostname separation → UTS namespace (kernel 2.6.19)– IPC separation → IPC namespace (kernel 2.6.19)– User (UID/GID) separation → User namespace (kernel 2.6.23〜kernel 3.8)– Processtable separation　 → PID namespace (kernel 2.6.24)　– Network separation　　 → Network Namespace (kernel 2.6.24)– Usage limit of CPU/Memory → Control groups

Linux container is realized with integrating these namespace features. There are multiple container management tools such as lxctools, libvirt and docker. They may use different parts of these features.

Under the hood

9


Processes in all containers are executed on the same Linux kernel. But, inside a container, you can see processes only in the container.– This is because each container has its own process table. On host linux, which is outside

containers, you can see all processes including ones in containers.

Process table

# ps -efUID PID PPID C STIME TTY TIME CMDroot 1 0 0 09:49 ? 00:00:00 /bin/sh /usr/local/bin/init.shroot 47 1 0 09:49 ? 00:00:00 /usr/sbin/httpdapache 49 47 0 09:49 ? 00:00:00 /usr/sbin/httpdapache 50 47 0 09:49 ? 00:00:00 /usr/sbin/httpd...apache 56 47 0 09:49 ? 00:00:00 /usr/sbin/httpdroot 57 1 0 09:49 ? 00:00:00 /bin/bash

# ps -efUID PID PPID C STIME TTY TIME CMD...root 802 1 0 18:10 ? 00:01:20 /usr/bin/docker -d --selinux-enabled -H fd://...root 3687 802 0 18:49 pts/2 00:00:00 /bin/sh /usr/local/bin/init.shroot 3748 3687 0 18:49 ? 00:00:00 /usr/sbin/httpd48 3750 3748 0 18:49 ? 00:00:00 /usr/sbin/httpd...48 3757 3748 0 18:49 ? 00:00:00 /usr/sbin/httpdroot 3758 3687 0 18:49 pts/2 00:00:00 /bin/bash

Processes seen inside container

Processes seen outside container

10


Process table

fork/exec

PID namespace

In the example of previous page, docker daemon fork/exec-ed the initial process "init.sh" and put it in a new "PID namespace." After that, all processes fork/exec-ed from init.sh are put in the same namespace.

Inside container, the initial process has PID=1 independently from the host. Likewise, child processes of it have independent PID's.

PID=1

bash

/bin/sh /usr/local/bin/init.sh

httpd

httpd

・・・

#!/bin/sh

service httpd startwhile [[ true ]]; do /bin/bashdone

init.sh

docker daemon

11


Filesystem

A specific directory on the host is bind mounted as a root directory of the container. Inside container, that directory is seen as a root directory, very similar mechanism to the "chroot jail."

When using traditional container management tools such as lxctools or libvirt, you need to prepare the directory contents by hand.– You can put minimum contents for a specific application such as application binaries

and shared libraries in the directory.– It's also possible to copy a whole root filesystem of a specific linux distribution to

the directory.– If necessary, special filesystems such as /dev, /proc and /sys are mounted in the

container by the management tool.

Mount namespace

/ |--etc |--bin |--sbin...

/export/container01/rootfs/ |--etc |--bin |--sbin ...

bind mount

12


Filesystem

Container

App

licat

ion

Directory Tree

Mounted on the host

Assign as / filesystem

With Docker, you don't need to prepare the directory tree by hand.

Docker image is mounted on the host and used as root filesystem of the container.

Docker Image

13


Network namespace

Network

Container uses Linux's "veth" device for network communication.– veth is a pair of logical NIC devices connected through a (virtual) crossover cable.

One side of the veth pair is placed in a container's network namespace so that it can be seen only inside the container. The other side is connected to a Linux bridge on the host.– A device name in the container is renamed such as "eth0." By means of the namespace, network

settings such as IP address, routing table and iptables are independently configured in the container。

– The connection between the bridge and a physical network is up to the host configuration.

Host LinuxvethXX

eth0

docker0

eth0

IP masquerade

Physical network

Docker creates a bridge "docker0" and packets from containers are forwarded with IP masquerade.– Packets from the physical network targeted to specified

ports are forwarded to the container using the port forwarding feature of iptables.

172.17.42.1

14


Network

Example container network for 3-tier application running on the same host.

Accessing to the external IP of the host

Container：Web Server

REST_PORT_5555_TCP_ADDR

eth0

DB_PORT_3306_TCP_ADDR

Container：App Server

eth0

Container：Database

eth0

Linux bridge（docker0）

External IP

Port 80 Port 5555 Port 3306

Port 80

15


Network

Example container network for 3-tier application running on different hosts.


eth0

External IP


eth0

External IP

eth0

External IP

Container：Web Server Container：App Server Container：Database

Port 80 Port 5555 Port 3306

Architecture of Kubernetes

17


Server configuration

etcd

・・・

Backend Database（KVS）

Kubernetes MasterKubernetes Node (Minion)

・・・

Scale-out cluster

Docker Docker Docker

Add more minionsif necessary.

Docker Registry

Kubernetes manages multiple nodes (minions) from a single master.– Clustering of multiple masters is not available now. You may use active-standby

configuration with standard HA tools for high availability.– etcd (KVS) is used as a backend database. It can be configured as a scale-out cluster.

18


Network configuration

etcd KubernetesMaster

DockerRegistry

Configured asan overlay network.

・・・

Physical network is simple. Kubernetes works just by connecting all servers to a single service network.

However, you need to create an internal network for container communication using an overlay network.– You may use Flannel, Open vSwitch, etc. as an overlay technology.

Service network192.168.122.0/24

Minion

docker0

Minion

docker0

Internal network10.1.0.0/16

19


Internal network details

The internal network needs to be prepared independently from Kubernetes.– Flannel is the most convenient tool for this purpose.

Flannel configures an internal network as follows:– Assign non-overlapping subnets to the Linux bridge (docker0) of each minion. (eg.

10.1.x.0/24 with x=1,2,3,...)– Create a virtual interface "flannel.1" which works as a gateway to other minions.– Linux kernel on each minion transfers packets from/to flannel.1 using the VXLAN

encapsulation. (Flannel daemon "flanneld" provides necessary information for VXLAN processing to the kernel.)

flannel.1

docker0

10.1.1.0/24

10.1.1.0

etn0

10.1.1.1

Gateway to10.1.0.0/16

Encapsulation flannel.1

docker0

10.1.2.0/24

10.1.2.0

etn0

10.1.2.1

Gateway to10.1.0.0/16

minion01 minion02

10.1.0.0/16

flanneld flanneld

20


External access

etcd KubernetesMaster Minion Docker

RegistryMinion

API requests Image upload

・・・

Service access

There are following cases for the external access.– API requests are sent to the master.– Services running on containers are accessed from minions' external IPs via proxy

mechanism.– Docker registry is an independent component from Kubernetes. You may use a

registry server running on a container.

Service network

Internal network

21


Baremetal / VM ・・・

Docker

Baremetal / VM

Docker

Kubernetes

Platform as a Service

・・・

Execution Resource

ContainerManagement

Container Orchestration

UI, Monitoring,Image build workflow,

etc.

RHEL Atomic Host

OpenShift 3.0

Beyond Kubernetes: OpenShift v3

Container

Container

Container

Container

・・・・・・

References

23


References

Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1– http://www.slideshare.net/enakai/architecture-overview-rubbernecks-with-red-

hat-enterprise-linux-71

Inside Docker for Fedora20/RHEL7– http://www.slideshare.net/enakai/docker-technology-v18e

OpenShift 3 Technical Architecture– https://docs.google.com/presentation/d/1Isp5UeQZTo3gh6e59FMYmMs_V9QIQeBel

mbyHIJ1H_g/pub

OpenShift v3 Internal networking details– http://www.slideshare.net/enakai/openshift-45465283

EMPOWER PEOPLE,

EMPOWER ENTERPRISE,

OPEN INNOVATION.

linux container technology inside docker with rhel7

Technology

linux container technology

rhel7 linux container

rhel7 container

linux kernel feature

host linux

image docker image

usrsbinhttpd root

rhel7 processes