docker orchestration

1

2015. 5. 28

㈜ 오픈소스컨설팅

김호진 , 박현익

Using Docker and How to orchestrate it.

2

ContentsContents1.1. Container image 관리

1.11.1 Docker registry

2.2. Docker Compose

2.12.1Compose install & define the project

2.22.2PHP wordpress programming

3.3. Docker Swarm

2.32.3 Build by docker file

2.42.4Docker registry & start by docker-compose

4.4. Docker Kubernetes

3

yum install docker-registry

Create private Docker registries

Docker local registry 만들기

[root@docker187 ~]# systemctl status docker.service[root@docker187 ~]# systemctl -l status docker-registry

[root@docker187 ~]# docker loginUsername (oscinfra): WARNING: login credentials saved in /root/.dockercfg.Login Succeeded

https://registry.hub.docker.com/repos/ [root@docker187 ~]# netstat -ntpl | grep 5000tcp 0 0 0.0.0.0:5000 0.0.0.0:* LISTEN 729/python

firewall-cmd --zone=public --list-servicesfirewall-cmd --zone=public --list-portsfirewall-cmd --permanent --zone=public --add-service=httpsfirewall-cmd --permanent --zone=public --add-port=5000/tcpfirewall-cmd --zone=public --add-port=5000/tcpfirewall-cmd –reload

[root@docker187 ~]# nmap localhost

PORT STATE SERVICE5000/tcp open upnp

4



Root image 가지고 오기

[root@docker188 ~]# docker pull mysql:latest

latest: Pulling from docker.io/mysql

[root@docker188 ~]# docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=qwerqwer -d

mysql

d10d4821e9b96f7f1f5065351f83e014f2799bf8b4febc9da6d3573d12f8aa4e

[root@docker188 ~]# docker run --name wordpress-instance --link some-mysql:mysql -p 8000:80 -d

wordpress

edb109fc4ba635f77f9e506b786b3b5529f8d6b7fd6a12ab4adb53116907d02e

[root@docker188 ~]# nmap localhost

5000/tcp open upnp

8080/tcp open http-proxy

출처 : http://www.infosiftr.com/2014/11/13/docker-tutorial-wordpress/

5



Install 후 commit 작업 실시


6




[root@docker188 ~]# docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

edb109fc4ba6 wordpress:latest "/entrypoint.sh apac 11 minutes ago Up 11 minutes 0.0.0.0:8080->80/tcp wordpress-instance

d10d4821e9b9 mysql:latest "/entrypoint.sh mysq 13 minutes ago Up 13 minutes 3306/tcp some-mysql

[root@docker188 ~]# docker commit edb109fc4ba6 osc_wordpress

9655c5be2aa9013d72dc4908d38149e9938b03f32039f1bb89fe226ee9478a11

[root@docker188 ~]# docker commit d10d4821e9b9 osc_mysql

d1341c6a3b5860cbb55d08e96e239224786e4c3767294e2d13705609a183f353

[root@docker188 ~]# docker images

REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE

osc_mysql latest d1341c6a3b58 About a minute ago 282.9 MB

osc_wordpress latest 9655c5be2aa9 About a minute ago 511.5 MB

docker.io/wordpress latest 2bed879f48b4 44 hours ago 511.5 MB

docker.io/mysql latest ff78d9bb5f46 3 days ago 282.9 MB

[root@docker188 ~]# docker ps -a


edb109fc4ba6 wordpress:latest "/entrypoint.sh apac 14 minutes ago Exited (0) 36 seconds ago wordpress-instance

d10d4821e9b9 mysql:latest "/entrypoint.sh mysq 16 minutes ago Exited (0) 34 seconds ago some-mysql

7




[root@docker188 ~]# docker stop `docker ps -qa`

[root@docker188 ~]# docker run --name osc_mysql -e MYSQL_ROOT_PASSWORD='qwerqwer' -d osc_mysql

d15491be7bc85bf764f1a2bb2c2b9d7c00e76e1eae476ee572ff52bc609daa68

[root@docker188 ~]# docker run --name osc_wordpress2 --link osc_mysql:mysql -p 8090:80 -d osc_wordpress

12dceb757464a858326658a8c8d26b45d5efa7631d47c7b33ac0bcc14d8bced6

[root@docker188 ~]# docker ps -a


12dceb757464 osc_wordpress:latest "/entrypoint.sh apac 11 seconds ago Up 10 seconds 0.0.0.0:8080->80/tcp

osc_wordpress2

d15491be7bc8 osc_mysql:latest "/entrypoint.sh mysq About a minute ago Up About a minute 3306/tcp osc_mysql

edb109fc4ba6 wordpress:latest "/entrypoint.sh apac 17 minutes ago Exited (0) 3 minutes ago wordpress-instance

d10d4821e9b9 mysql:latest "/entrypoint.sh mysq 19 minutes ago Exited (0) 3 minutes ago some-mysql

8



Commit 후 private Docker registry 에 등록 후 파일 전송



osc_mysql latest 967e48b02b99 2 hours ago 282.9 MB

osc_wordpress latest d590073094fd 2 hours ago 511.5 MB

docker.io/wordpress latest 2bed879f48b4 2 days ago 511.5 MB


[root@docker188 ~]# docker tag osc_mysql:latest localhost:5000/osc/osc_mysql




localhost:5000/osc/osc_mysql latest 967e48b02b99 2 hours ago 282.9 MB




[root@docker188 ~]# docker push localhost:5000/osc/osc_mysql

The push refers to a repository [localhost:5000/osc/osc_mysql] (len: 1)

Sending image list

967e48b02b99: Image successfully pushed

Pushing tag for rev [967e48b02b99] on {http://localhost:5000/v1/repositories/osc/osc_mysql/tags/latest}

9



Commit 후 private Docker registry 에 등록 후 파일 전송

[root@docker188 ~]# docker tag osc_wordpress localhost:5000/osc/osc_wordpress



localhost:5000/osc/osc_mysql latest 967e48b02b99 3 hours ago 282.9 MB


localhost:5000/osc/osc_wordpress latest d590073094fd 3 hours ago 511.5 MB




[root@docker188 ~]# docker push localhost:5000/osc/osc_wordpress

The push refers to a repository [localhost:5000/osc/osc_wordpress] (len: 1)

Sending image list

Pushing repository localhost:5000/osc/osc_wordpress (1 tags)

d590073094fd: Image successfully pushed

Pushing tag for rev [d590073094fd] on {http://localhost:5000/v1/repositories/osc/osc_wordpress/tags/latest}

10

Docker Compose

소개

도커 복잡한 응용 프로그램을 정의하고 실행하기위한 도구입니다멀티 컨테이너를 하나의 파일로 운용하나의 command 로 모든 doker file 들 실행

11

Docker compose

Compose install & define the project


[root@docker188 ~]# curl -L https://github.com/docker/compose/releases/download/1.2.0/docker-compose-`uname -s`-`uname -m` >

/usr/local/bin/docker-compose

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

100 403 0 403 0 0 343 0 --:--:-- 0:00:01 --:--:-- 343

100 5140k 100 5140k 0 0 1031k 0 0:00:04 0:00:04 --:--:-- 1871k

[root@docker188 ~]# chmod +x /usr/local/bin/docker-compose

[root@docker188 ~]# curl https://wordpress.org/latest.tar.gz | tar -xvzf -

[root@docker188 wordpress]# cat Dockerfile

FROM orchardup/php5

ADD . /code

[root@docker188 wordpress]# cat docker-compose.yml

web:

build: .

command: php -S 0.0.0.0:8000 -t /code

ports:

- "8000:8000"

links:

- db

volumes:

- .:/code

db:

image: orchardup/mysql

environment:

MYSQL_DATABASE: wordpress

12

Docker compose

Php wordpress programming

간단 program

[root@docker01 wordpress]# cat wp-config.php<?phpdefine('DB_NAME', 'wordpress');define('DB_USER', 'root');define('DB_PASSWORD', '');define('DB_HOST', "db:3306");define('DB_CHARSET', 'utf8');define('DB_COLLATE', '');

define('AUTH_KEY', 'test');define('SECURE_AUTH_KEY', 'test');define('LOGGED_IN_KEY', 'test');define('NONCE_KEY', 'test');define('AUTH_SALT', 'test');define('SECURE_AUTH_SALT', 'test');define('LOGGED_IN_SALT', 'test');define('NONCE_SALT', 'test');

$table_prefix = 'wp_';define('WPLANG', '');define('WP_DEBUG', false);

if ( !defined('ABSPATH') ) define('ABSPATH', dirname(__FILE__) . '/');

require_once(ABSPATH . 'wp-settings.php');

[root@docker01 wordpress]# cat router.php<?php$root = $_SERVER['DOCUMENT_ROOT'];chdir($root);$path = '/'.ltrim(parse_url($_SERVER['REQUEST_URI'])['path'],'/');set_include_path(get_include_path().':'.__DIR__);if(file_exists($root.$path)){ if(is_dir($root.$path) && substr($path,strlen($path) - 1, 1) !== '/') $path = rtrim($path,'/').'/index.php'; if(strpos($path,'.php') === false) return false; else { chdir(dirname($root.$path)); require_once $root.$path; }}else include_once 'index.php';

13

Docker compose

Build by docker file

실행

[root@docker188 wordpress]# docker build -t oscinfra_web .Sending build context to Docker daemon 19.79 MBSending build context to Docker daemon Step 0 : FROM orchardup/php5Trying to pull repository docker.io/orchardup/php5 ...7113324d9d9e: Download complete 511136ea3c5a: Download complete e2aa6665d371: Download complete f0ee64c4df74: Download complete 2209cbf9dcd3: Download complete 5e019ab7bf6d: Download complete Status: Downloaded newer image for docker.io/orchardup/php5:latest ---> 7113324d9d9eStep 1 : ADD . /code ---> 4ea4c8b5cb7fRemoving intermediate container 5ab560447a8bSuccessfully built 4ea4c8b5cb7f[root@docker188 wordpress]# docker imagesREPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZEoscinfra_web latest 4ea4c8b5cb7f 10 seconds ago 348.9 MB

[root@docker01 wordpress]# docker-compose upCreating wordpress_db_1...Pulling image orchardup/mysql:latest...Pulling repository orchardup/mysql061b756f7e0d: Download completeStatus: Downloaded newer image for orchardup/mysql:latestCreating wordpress_web_1...Building web...Step 0 : FROM orchardup/php5---> 7113324d9d9e..db_1 | 150301 14:54:42 InnoDB: highest supported file format is Barracuda.db_1 | 150301 14:54:42 InnoDB: Waiting for the background threads to startdb_1 | 150301 14:54:43 InnoDB: 5.5.38 started; log sequence number 1595675db_1 | 150301 14:54:43 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306db_1 | 150301 14:54:43 [Note] - '0.0.0.0' resolves to '0.0.0.0';db_1 | 150301 14:54:43 [Note] Server socket created on IP: '0.0.0.0'.

14

Docker compose

Docker registry & start by docker-compose

실행

[root@docker188 wordpress]# docker imagesREPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZEoscinfra_web latest 4ea4c8b5cb7f 2 minutes ago 348.9 MBosc_mysql latest 967e48b02b99 3 hours ago 282.9 MBlocalhost:5000/osc/osc_mysql latest 967e48b02b99 3 hours ago 282.9 MBosc_wordpress latest d590073094fd 3 hours ago 511.5 MBlocalhost:5000/osc/osc_wordpress latest d590073094fd 3 hours ago 511.5 MBdocker.io/wordpress latest 2bed879f48b4 2 days ago 511.5 MBdocker.io/mysql latest ff78d9bb5f46 4 days ago 282.9 MBdocker.io/orchardup/php5 latest 7113324d9d9e 12 months ago 330.1 MB[root@docker188 wordpress]# docker tag oscinfra_web localhost:5000/oscweb/oscinfra_web[root@docker188 wordpress]# docker push localhost:5000/oscweb/oscinfra_webThe push refers to a repository [localhost:5000/oscweb/oscinfra_web] (len: 1)Sending image listPushing repository localhost:5000/oscweb/oscinfra_web (1 tags)511136ea3c5a: Image successfully pushed e2aa6665d371: Image successfully pushed f0ee64c4df74: Image successfully pushed 2209cbf9dcd3: Image successfully pushed 5e019ab7bf6d: Image successfully pushed 7113324d9d9e: Image successfully pushed 4ea4c8b5cb7f: Image successfully pushed Pushing tag for rev [4ea4c8b5cb7f] on {http://localhost:5000/v1/repositories/oscweb/oscinfra_web/tags/latest}

15

Docker compose


실행

[root@docker188 wordpress]# docker-compose upCreating wordpress_db_1...Pulling image orchardup/mysql:latest...

ab3b99429ab1: Download completeab3b99429ab1: Pulling image (latest) from docker.io/orchardup/mysqlStatus: Downloaded newer image for docker.io/orchardup/mysql:latestCreating wordpress_web_1...Building web...Step 0 : FROM orchardup/php5 ---> 7113324d9d9eStep 1 : ADD . /code ---> 64a9cfc3a30aRemoving intermediate container 4c5bf7fbd928Successfully built 64a9cfc3a30aAttaching to wordpress_db_1, wordpress_web_1

……

db_1 | 150529 6:06:55 InnoDB: Waiting for the background threads to startdb_1 | 150529 6:06:56 InnoDB: 5.5.38 started; log sequence number 1595675db_1 | 150529 6:06:56 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306db_1 | 150529 6:06:56 [Note] - '0.0.0.0' resolves to '0.0.0.0';db_1 | 150529 6:06:56 [Note] Server socket created on IP: '0.0.0.0'.db_1 | 150529 6:06:56 [Note] Event Scheduler: Loaded 0 eventsdb_1 | 150529 6:06:56 [Note] /usr/sbin/mysqld: ready for connections.db_1 | Version: '5.5.38-0ubuntu0.12.04.1-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu)

16

Docker compose


실행

[root@docker188 ~]# docker commit -m "0528_status" -a "[email protected]" 0358aa2a848f wordpress_web_1_0528Cd34d5a8b091dcf5b95f3eba1d702f55f591698d15d67b26762c0314b198db31[root@docker188 ~]# docker commit -m "0528_status" -a "[email protected]" 2d20222e6afb mysql_1_05285c3f4110a1864995bd5c90287f0e8ccc5453b3cd5600ae532ab5ca322cf217d1

[root@docker188 ~]# docker imagesREPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZEmysql_1_0528 latest 5c3f4110a186 20 seconds ago 292.3 MBwordpress_web_1_0528 latest cd34d5a8b091 2 minutes ago 348.9 MBwordpress_web latest 64a9cfc3a30a 18 minutes ago 348.9 MBoscinfra_web latest 4ea4c8b5cb7f 25 minutes ago 348.9 MBlocalhost:5000/oscweb/oscinfra_web latest 4ea4c8b5cb7f 25 minutes ago 348.9 MBlocalhost:5000/osc/osc_mysql latest 967e48b02b99 4 hours ago 282.9 MBosc_mysql latest 967e48b02b99 4 hours ago 282.9 MBosc_wordpress latest d590073094fd 4 hours ago 511.5 MBlocalhost:5000/osc/osc_wordpress latest d590073094fd 4 hours ago 511.5 MBdocker.io/wordpress latest 2bed879f48b4 2 days ago 511.5 MBdocker.io/mysql latest ff78d9bb5f46 4 days ago 282.9 MBdocker.io/orchardup/mysql latest ab3b99429ab1 9 months ago 292.3 MBdocker.io/orchardup/php5 latest 7113324d9d9e 12 months ago 330.1 MB

17

Docker Swarm

소개

Docker Native Clustering System

클러스터 / 컨테이너 / 이미지를 컨트롤

클러스터 그룹에서 비교적 쉬운 WorkLoad 제어

Docker API 지원

Swarm 은 매니저 Docker Node 는 클러스터의 노드

특별히 필요로 하는 인프라 조건은 없음

18

Docker Swarm

Swarm Host Install (Swarm Host 에서 실행 )

# yum install mercurial golang git

# mkdir ~/gocode; export GOPATH=~/gocode

# cd ~/gocode

# go get -u github.com/docker/swarm

Swarm Install Test

# ~/gocode/bin/swarm –help

19

Docker Swarm

Docker Node Install (Docker Node 에서 실행 )

# yum install docker

# vi /etc/sysconfig/docker 파일의 OPTIONS 항목에 -H 0.0.0.0:< 포트 > 를 추가# systemctl restart docker

/etc/sysconfig/docker 의 옵션

20

Docker Swarm

Setup Swarm Host (Swarm Host 에서 실행 )

# ~/gocode/bin/swarm create (Swarm Cluster 를 생성합니다 . 생성되는 ID 는 따로 잘 보관하여야 합니다 )

# ~/gocode/bin/swarm join --addr=<ip-of-node>:2375 token://<swarm_id> (IP 가 한번만 나타나면 바로 Ctrl + C 를 입력하여 취소하여도 상관없습니다 )

# ~/gocode/bin/swarm list token://<swarm_id> (Swarm에 등록된 노드들의 리스트를 확인합니다 .)

# ~/gocode/bin/swarm --debug manage --host=0.0.0.0:2375 token://<swarm_id> (Swarm Manager 시작 )

Swarm Manager 를 실행하는 도중 Duplicate Error 발생시는 /etc/docker/key.json 파일 or ~/.docker/key.json 파일을 삭제한 후 데몬을 재시작합니다 .

클러스터 ID

클러스터에 참가되어 있는 Node

21

Docker Swarm

Docker 컨테이너 실행 ( 아무 노드에서나 수행 )

1 번 Host 에서 ubuntu bash 쉘 수행

2 번 Host 에 컨테이너 기동

1 번 Host 에 컨테이너 파일 없음

2 번 Host 에 컨테이너 파일 확인

22

Docker Swarm

[root@Docker-Host2 ~]# docker -H 192.168.0.154:6644 run -d -t -i centos /bin/bash9fc314043a7f6960a98e1fd594c047b36aa8240da5a3bd341417aff5dcf2f648[root@Docker-Host2 ~]# [root@Docker-Host2 ~]# [root@Docker-Host2 ~]# docker -H 192.168.0.154:6644 run -d -t -i centos /bin/bash6249d9581761bcd7b33fac05eee3ba62fea93e779558a690c23ebe73a73f0532[root@Docker-Host2 ~]# docker -H 192.168.0.154:6644 run -d -t -i centos /bin/bashfdeea5649fccc1ef2786d12d08f38b58f0dc6ad0c45fe11b97385bb8eb4e93d1[root@Docker-Host2 ~]# docker -H 192.168.0.154:6644 run -d -t -i centos /bin/bash080a90c0ca8f647455a69e2d0951d802008c5078c091a9940c2dd5267c574358[root@Docker-Host2 ~]# docker -H 192.168.0.154:6644 run -d -t -i centos /bin/bash06cf2a484ea6ed893b5af7e1ac1163314bbe6ed8a55f439fe694e9353ba1054b[root@Docker-Host2 ~]# docker -H 192.168.0.154:6644 run -d -t -i centos /bin/bashfb0e410202fd552529396418ac13279c68fc08a8e2e701898dcd07e221b4a92c[root@Docker-Host2 ~]# docker -H 192.168.0.154:6644 run -d -t -i centos /bin/bash3817a06c550f9350db2bcf349342be8d0fa65010642844c455b725515cc2f694[root@Docker-Host2 ~]# docker -H 192.168.0.154:6644 run -d -t -i centos /bin/basha2d5a13ce9f7e240f96ab06d838fb72b22f5e59664756de215ff9090e3199d25[root@Docker-Host2 ~]#

Every 2.0s: docker -H 192.168.0.154:6644 ps Fri May 29 00:23:49 2015

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESa2d5a13ce9f7 centos:latest "/bin/bash" About a minute ago Up About a minute Docker-Host2/boring_pike3817a06c550f centos:latest "/bin/bash" About a minute ago Up About a minute Docker-Host1/admiring_bellfb0e410202fd centos:latest "/bin/bash" About a minute ago Up About a minute Docker-Host2/goofy_yalow06cf2a484ea6 centos:latest "/bin/bash" 2 minutes ago Up About a minute Docker-Host1/stoic_colden080a90c0ca8f centos:latest "/bin/bash" 2 minutes ago Up 2 minutes Docker-Host2/determined_hodgkinfdeea5649fcc centos:latest "/bin/bash" 4 minutes ago Up 4 minutes Docker-Host2/thirsty_davinci6249d9581761 centos:latest "/bin/bash" 4 minutes ago Up 4 minutes Docker-Host1/grave_mcclintock9fc314043a7f centos:latest"/bin/bash" 5 minutes ago Up 4 minutes Docker-Host1/happy_shockley

2 번 Host 에서 bash 쉘 수행

1 번 Host 에서 모니터링 결과

23

Docker Swarm

Docker Swarm Architecture

Swarm Docker-Host1

Docker-Host2

Swarm-Cluster

Swarm 중앙 집중식 관리각 노드의 커맨드는 클러스터를 커져 Swarm 을 거쳐 최종 노드에서 수행클러스터의 전략과 필터를 통하여 컨테이너 수행 제어

Listening / SendingPort 6644

SendingPort 6644

SendingPort 6644

24

Docker Kubernetes

소개

Kubernetes 는 여러 호스트에서 컨테이너 응용 프로그램을 관리하는 응용 프로그램의 배포 , 유지 보수 및 확장을위한 기본 메커니즘을 제공하는 시스템이다 .

Kubernetes 는 single master 가 여러 node(minions) 를 관리한다 .

출처 : http://www.cyworld.com/ruo91/8923965

- Etcd

kubernetes 의 관련 된 정보를 담고 있는 Key/Value 저장소

Master

- Service proxy

설정한 Pod 의 RR(Round Robin)을 담당하는 역할

Minion

- Kubelet

Minion 을 제어하는 agent

- pod

Control 하는 단위

docker orchestration

Documents