mysql on docker: sep 2017 - percona on docker... · sep 2017 mysql on docker: containerizing the...
TRANSCRIPT
Sep 2017
MySQL on Docker: Containerizing the Dolphin
Ashraf Sharif, Support EngineerPresenter
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB
About Severalnines and ClusterControl
Agenda
Copyright 2012 Severalnines AB
● Introduction● Image● Container● Volume● Network● Orchestration● MySQL Container Management
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB
Introduction
What is Docker?
Copyright 2012 Severalnines AB
● Lightweight application container platform.● Each container has its own:
○ Filesystem○ Process space○ Network stack
● Use pre-built image.● Advantages:
○ Rapid deployment○ Greater density per host○ Isolation
Docker Architecture
Docker Components
Copyright 2012 Severalnines AB
Docker Components
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB
Docker Image
Docker Image - Introduction
Copyright 2012 Severalnines AB
● A template for Docker container.● Create an image:
○ Commit the container's changes○ Build from Dockerfile
● Share the image:○ Save as tarball.○ Push to registry:
■ Public - Docker Hub, Docker Store■ Private, hosted - Quay, ECR, GCR, Bintray,
Artifactory■ Private, self-hosted - Your own registry
# Save image to tarball$ docker save -o mysql.tar abc
# Start your own Docker registry service$ docker run -d -p 5000:5000 --name registry registry:2
# Push image to Docker Hub$ docker push myimage/mysql:5.7
Docker Image - Commit a Container
Copyright 2012 Severalnines AB
Host
# Save image to tarball$ docker run -d --name=abc ubuntu:16.04
# Commit the changes (in another window)$ docker commit abc myimage/mysql:5.7
# Push image to Docker Hub$ docker exec -it abc /bin/bashroot@abc:/# apt-get updateroot@abc:/# DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server root@abc:/# vi /etc/mysql/my.cnf
# Ubuntu 16.04 host$ apt-get update$ DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server$ vi /etc/mysql/my.cnf
# Run MySQL server as a process$ mysqld --socket=/var/lib/mysql/mysqld.sock --pid-file=/var/lib/mysql/mysqld.pid
Docker
# Run MySQL server as a container$ docker run -d myimage/mysql:5.7 mysqld --socket=/var/lib/mysql/mysqld.sock --pid-file=/var/lib/mysql/mysqld.pid
Docker Image - Using Dockerfile
Copyright 2012 Severalnines AB
Host
# Build as image$ docker build -t myimage/mysql:5.7 .
# vi DockerfileFROM ubuntu:16.04RUN apt-get updateRUN DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-serverCOPY my.cnf /etc/mysql/my.cnfEXPOSE 3306
# Ubuntu 16.04 host$ apt-get update$ DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server$ vi /etc/mysql/my.cnf
# Run MySQL server as a process$ mysqld --socket=/var/lib/mysql/mysqld.sock --pid-file=/var/lib/mysql/mysqld.pid
Docker
# Run MySQL server as a container$ docker run -d myimage/mysql:5.7 mysqld --socket=/var/lib/mysql/mysqld.sock --pid-file=/var/lib/mysql/mysqld.pid
# Create MySQL config file$ vi my.cnf
Docker Image - Build an Image
Copyright 2012 Severalnines AB
# Get image size$ docker imagesREPOSITORY TAG SIZEmyimage/mysql 5.7 529MB
# Show created layers$ docker history myimage/mysql:5.7CREATED BY SIZE/bin/sh -c #(nop) COPY file:e8163ef656b6da... 101B/bin/sh -c DEBIAN_FRONTEND=noninteractive ... 370MB/bin/sh -c apt-get update 39.1MB/bin/sh -c #(nop) CMD ["/bin/bash"] 0B/bin/sh -c mkdir -p /run/systemd && echo '... 7B/bin/sh -c sed -i 's/^#\s*\(deb.*universe\... 2.76kB/bin/sh -c rm -rf /var/lib/apt/lists/* 0B/bin/sh -c set -xe && echo '#!/bin/sh' >... 745B/bin/sh -c #(nop) ADD file:39d3593ea220e68... 120MB
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
724179e94999
f6016aab25f1
d51f459239fb
Image: myimage/mysql:5.7
ubuntu:16.04
ad74af05f5a2
Dockerfile
Docker Image - Best Practice for MySQL
Copyright 2012 Severalnines AB
# Get image size$ docker imagesREPOSITORY TAG SIZEmysql 5.6 299MBmysql 5.7 412MB
# tail entrypoint.sh......exec mysqld "$@"
● Extend an existing image available in the registry, instead of building a new one from scratch.
● The smaller the image size, the faster the deployment. (Hint: alpine).
● If using an entrypoint script, ensure mysqld command runs with "exec", indicating it's the primary process that holds PID 1 inside container.
● General guidelines: https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/
# vi DockerfileFROM mysql:5.7
# vi DockerfileFROM ubuntu:16.04RUN apt-get install {all MySQL stuff}
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB
Docker Container
Docker Container - Run a Single Container
Copyright 2012 Severalnines AB
# Run a MySQL container$ docker run -d \--name my-test \--env MYSQL_ROOT_PASSWORD=mypassword \mysql
# Get the container layer size$ docker ps -sCONTAINER ID IMAGE NAMES SIZEd51f459239fb mysql my-test 4B (virtual 412MB)
Image: docker.io/mysql:latest
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
d51f459239fb
0639788facc8
35369f9634e1
937bbdd4305a
my-test
Image layers (RO)
412 MB
Container layers (RW)4 B
Docker Container - Run another Container
Copyright 2012 Severalnines AB
# Run a MySQL container$ docker run -d \--name my-test2 \--env MYSQL_ROOT_PASSWORD=mypassword \mysql
# Get the container layer size$ docker ps -sCONTAINER ID IMAGE NAMES SIZEd51f459239fb mysql my-test 4B (virtual 412MB)e9de9ed50ced mysql my-test2 4B (virtual 412MB)
Image: docker.io/mysql
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
d51f459239fb
0639788facc8
35369f9634e1
937bbdd4305a
my-test
Image layers (RO)
412 MB
Container layers (RW)
4 B + 4 B
e9de9ed50ced
my-test2
Docker Container - Container Layer Changes
Copyright 2012 Severalnines AB
# Operation 2 - Create a schema and table$ docker exec -it my-test \mysql -uroot -pmypassword -e \'CREATE SCHEMA testdb; CREATE TABLE t1 (id INT PRIMARY KEY AUTO_INCREMENT, data TEXT);'
Image: docker.io/mysql
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
d51f459239fb
0639788facc8
35369f9634e1
937bbdd4305a
my-test
c33a2b88395d
Copy-on-Write (CoW)
416f2a97355f
Supported CoW file system & devices: AUFS, Btrfs, Device Mapper, OverlayFS/Overlay2, ZFS, VFS
# Operation 1 - Update /etc/mysql/my.cnf$ docker exec -it my-test vi /etc/mysql/my.cnf
1
2New Branch
Docker Container - Application vs Data
Copyright 2012 Severalnines AB
Application Data
Stateless Stateful
Ephemeral Persistent
Independant Dependant
Frontend Backend
Web, API Database, File Storage
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB
Docker Volume
Docker Volume - Type
Copyright 2012 Severalnines AB
# Mount a named volume into container-v mysql1-data:/var/lib/mysql
# Mount tmpfs into container--tmpfs /tmp
● Bind mount:○ A file or directory on the host machine is
mounted into a container.○ You manage the directory’s contents.
● Named volume:○ A new directory is created within Docker’s
storage directory.○ Docker manages the directory’s contents.○ Extensible via volume plugin.
● tmpfs:○ Stored on host machine’s memory (or swap).○ Linux only.
# Bind mount a directory into container-v /data/mysql1/data:/var/lib/mysql
# Bind mount a file into container-v /data/mysql1/my.cnf:/etc/mysql/my.cnf
Docker Volume - Persistent Volumes
Copyright 2012 Severalnines AB
# Run with persistent named volume, using -v or --volume$ docker run -d \--name mysql-local \--env MYSQL_ROOT_PASSWORD=mypassword \--volume local-datadir:/var/lib/mysql \mysql:5.7
Image: docker.io/mysql:5.7
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-local
416f2a92355f
/var/lib/mysql
local-datadir
# Run with persistent named volume, using --mount$ docker run -d \--name mysql-local \--env MYSQL_ROOT_PASSWORD=mypassword \--mount source=local-datadir,target=/var/lib/mysql \mysql:5.7
Docker Volume - Changes on Volume
Copyright 2012 Severalnines AB
# Run a container with port 3308$ docker run -d \--name mysql-local \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v local-datadir:/var/lib/mysql \mysql
Image: docker.io/mysql:latest
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-local
416f2a92355f
/var/lib/mysql
local-datadir
# Run with persistent named volume, using --mount$ mysql -uroot -p -h127.0.0.1 -P3308 < dump.sql
dump.sql
3308
Docker Volume - Remote Volume
Copyright 2012 Severalnines AB
# Run a container with NFS bind mount$ docker run -d \--name mysql-nfs \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v /nfs/mysql-nfs:/var/lib/mysql \mysql
Image: docker.io/mysql
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-nfs
416f2a92355f
/var/lib/mysql
192.168.1.100/storage/docker
# Mount NFS with extra options for MySQL$ mount 192.168.1.100:/storage/docker /nfs -o noatime,nodiratime,hard,intr
3308
/nfs/mysql-nfs
Further Reading - Using NFS with MySQL:https://dev.mysql.com/doc/refman/5.7/en/disk-issues.html#disk-issues-nfs
Docker Volume - MySQL Persistency
Copyright 2012 Severalnines AB
Description Variable Name Default Value (5.7)
MySQL data directory datadir /var/lib/mysql
MySQL plugin directory plugin_dir {basedir}/lib/plugin
MySQL configuration directory config_dir /etc/my.cnf.d
MySQL binary log log_bin {datadir}/{hostname}-bin
Slow query log slow_query_log_file {datadir}/{hostname}-slow.log
Error log log_error {datadir}/{hostname}.err
General log general_log_file {datadir}/{hostname}.log
Docker Volume - Non-Persistent Volume
Copyright 2012 Severalnines AB
# Run a container with NFS bind mount and tmpfs volumes$ docker run -d \--name mysql-nfs-tmp \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v /nfs/mysql-nfs:/var/lib/mysql \--tmpfs /tmp:rw,size=1g,mode=177 \mysql
Image: docker.io/mysql:latest
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-nfs-tmp
416f2a92355f
/var/lib/mysql
192.168.1.100/storage/docker
3308
/nfs/mysql-nfs
/tmp
Docker Volume - Drivers
Copyright 2012 Severalnines AB
● Default is local.● You can use other different volume driver (or plugin).
○ Install the plugin.○ Use --volume-driver option.
● Volume drivers allow you to extend Docker's volume capabilities:
○ Snapshot○ Backup○ Encryption
# Install Rexray EBS docker plugin$ docker plugin install rexray/ebs EBS_ACCESSKEY=XXXX EBS_SECRETKEY=YYYY
# Verify if the plugin is loaded$ docker info -f '{{json .Plugins.Volume}}' | jq[ "local", "rexray"]
# Run a container with volume plugin$ docker run -d \--name=mysql-ebs \--volume-driver=rexray/ebs \-v ebs-datadir:/var/lib/mysql \mysql:5.7
# List volumes$ docker volume lsDRIVER VOLUME NAMElocal local-datadir1local local-datadir2rexray ebs-datadir
Docker Volume - Drivers and Storage Platforms
Copyright 2012 Severalnines AB
Further Reading - Volume Plugins List: https://docs.docker.com/engine/extend/legacy_plugins/#volume-plugins
Contiv
Fuxi Netshare
REX-ray
Horcrux
Convoy
OpenStorage
Amazon S3Azure FS
Google PD
Amazon EBS DigitalOcean BS
Flocker
EMC
FUSE
vSphere
iSCSI
CIFS NetApp OpenStack Cinder Minio CephFS
BeeGFS
NFS
SMB
Infinit
Docker Volume - Simple Benchmark
Copyright 2012 Severalnines AB
Local vs NFS vs NFS-tweaked vs EBS-standard
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB
Docker Network
Docker Network - Types
Copyright 2012 Severalnines AB
● Single-host:○ Host○ None○ Bridge
■ Default bridge (docker0)■ User-defined bridge
● Multi-host:○ Default overlay○ User-defined overlay
● Extensible via Docker network plugins.
# List all networks$ docker network lsNAME DRIVER SCOPEbridge bridge localdb_multi overlay swarmdb_single bridge localdocker_gwbridge bridge localhost host localingress overlay swarmnone null local
Docker Network - Host Network
Copyright 2012 Severalnines AB
# Run a container on host network$ docker run -d \--name mysql-host \--net host \-e MYSQL_ROOT_PASSWORD=mypassword \mysql:5.7
Container 1
eth0
eth0
Container 2
eth0
Docker
Host
# Display active TCP connections$ netstat -tulpn | grep mysqltcp6 0 0 :::3306 :::* LISTEN 24601/mysqld
● Container’s network interfaces will be identical with the machine host.
● Only one host network per machine host. ● Container linking, --link mysql-test:mysql is not
supported.● Port mapping, --publish 3307:3306 is not
supported.
Docker Network - Default Bridge Network
Copyright 2012 Severalnines AB
# Run a container on docker0 bridge network$ docker run -d \--name mysql-bridge \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \mysql:5.7
Container 1
eth0
eth0
Container 2
eth0
Docker
Host
# Display active TCP connections$ netstat -tulpn | grep 3308tcp6 0 0 :::3308 :::* LISTEN 24601/docker-proxy
● Packet forwarding by iptables to the bridge network.● --publish is supported. Run multiple containers with
same image through different ports.● --link is supported. Link with other containers to
expose environment variables.● docker-proxy redirects connection to the correct
container through NAT.
docker0(virtual Ethernet bridge)
vethxxx vethyyy
Docker Network - User-Defined Bridge Network
Copyright 2012 Severalnines AB
# Run a container on user bridge network$ docker run -d \--name mysql1 \--net db_single \-p 3308:3306 \--ip 192.168.10.10 \--hostname mysql1 \-e MYSQL_ROOT_PASSWORD=mypassword \mysql:5.7
Container 1
eth0
eth0
Container 2
eth0
Docker
Host
# Create a new bridge network$ docker network create subnet=192.168.10.0/24 db_single
Similar to the default docker0 bridge, plus:● --hostname is supported for sticky hostname.● --ip is supported for sticky IP address.● --link is not supported. Use embedded DNS to
resolve container's name in the same network.
db_single(virtual Ethernet bridge)
vethxxx vethyyy
DNS
Docker Network - Plugins
Copyright 2012 Severalnines AB
● Similar to volume, Docker network can be extended through plugins:
○ Multi-host networking (must use outside of Swarm)
○ Name resolver and service discovery.○ Encryption.
● Popular network plugins:○ Contiv○ Calico○ Weave○ Flannel
Further Reading - Multi-host Networking with Calico:https://severalnines.com/blog/mysql-docker-multi-host-networking-mysql-containers-part-2-calico
Docker Network - Default Overlay Network
Copyright 2012 Severalnines AB
# Run a service on default overlay network$ docker service create \--name mysql-service \--replicas 1 \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v nfs-datadir:/var/lib/mysql \mysql:5.7
Container 1
eth0
eth0 eth1
Docker
Host
# Initialize Docker Swarm$ docker swarm init
● Docker Swarm must be enabled.● Each task (container) has:
○ eth0 - Interface to external network via docker_gwbridge.
○ eth1 - Interface to overlay network via VXLAN.
docker_gwbridge
vethxxx vethyyy
br0
External
Container 2
eth0 eth1
Docker
docker_gwbridge
vethxxx vethyyy
br0
eth0Host
VXLANudp/4789
Docker Network - User-Defined Overlay Network
Copyright 2012 Severalnines AB
# Run a service on default overlay network$ docker service create \--name mysql-service \--replicas 1 \--net db_multi \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v nfs-datadir:/var/lib/mysql \mysql:5.7
Container 1
eth0
eth0 eth1
Docker
Host
# Create an overlay network$ docker network create -d overlay db_multi
Similar with the default overlay, plus:● Embedded DNS resolver:
○ {service_name} → Service VIP○ tasks.{service_name} → Container's IP
docker_gwbridge
vethxxx vethyyy
br0
External
Container 2
eth0 eth1
Docker
docker_gwbridge
vethxxx vethyyy
br0
eth0Host
VXLANudp/4789
DNS
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB
Docker Orchestration
Docker Orchestration - Container Orchestration
Copyright 2012 Severalnines AB
● Simplify container's management at scale.● Concept:
○ Multiple containers as one entity.○ Structural YAML formatting.
● Official Docker comes with:○ Docker Compose:
■ single-host, small scale, separate package.○ Docker Swarm:
■ multi-host, big scale, out-of-the-box.■ support compose formatting through
docker stack.
Docker Orchestration - Docker Compose
Copyright 2012 Severalnines AB
Docker CLI
version: '2'services: mysql-prod: image: mysql:5.7.16 command: - --innodb_buffer_pool_size=2G - --max_connections=50 ports: - "3307:3306" environment: MYSQL_ROOT_PASSWORD: r00tP4ssw0rD MYSQL_USER: myshop MYSQL_PASSWORD: MyS3creTP4s5 MYSQL_DATABASE: myshop hostname: mysql-prod networks: db-prod: ipv4_address: 192.168.10.101 volumes: - mysql-datadir:/var/lib/mysql - mysql-tmp:/tmp ulimits: nofile: soft: 16824 hard: 16824 mem_limit: 4g memswap_limit: 4gvolumes: mysql-datadir: mysql-tmp:networks: db-prod: driver: bridge ipam: driver: default config: - subnet: 192.168.10.0/24
# Production MySQL container - full command$ docker network create --subnet=192.168.10.0/24 db-prod$ docker run -d \--name mysql-prod \--net db-production \--publish 3307:3306 \--ip 192.168.10.101 \--hostname mysql-prod \--memory 4g \--memory-swap 4g \--ulimit nofile=16824:16824 \--volume mysql-datadir:/var/lib/mysql \--volume mysql-tmp:/tmp \--env MYSQL_ROOT_PASSWORD=r00tP4ssw0rD \--env MYSQL_USER=myshop \--env MYSQL_PASSWORD=MyS3creTP4s5 \--env MYSQL_DATABASE=myshop \mysql:5.7.16 \--innodb_buffer_pool_size=2G \--max_connections=50
Docker Orchestration - Run with Docker Compose
Copyright 2012 Severalnines AB
# Bring up the production MySQL container in background$ docker-compose up -dCreating network "compose_db-prod" with driver "bridge"Creating compose_mysql-prod_1
Image: docker.io/mysql:5.7.18
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-prod
416f2a92355f
/var/lib/mysql
mysql-datadir
/tmp
mysql-tmp
3307
# Stop the container$ docker-compose stop
# Destroy everything$ docker-compose down
# Start the container$ docker-compose start
Scaling is supported via docker-compose scale, however conflicting options must be omitted:hostname, ports, ipv4_address, volume
Docker Orchestration - Docker Swarm
Copyright 2012 Severalnines AB
● Multi-host deployment, through overlay network.● Concepts:
○ Node■ Manager - Dispatches tasks■ Worker - Executes tasks by manager
○ Service■ Replicated - Number of replicas■ Global - One task per available node
○ Task■ Container
○ Mesh routing■ All Swarm nodes route to the running tasks■ Ingress load balancing (round-robin only)
○ Scheduler■ Resources availability■ Label and constraints
Docker Orchestration - Simple Auto Failover using Swarm
Copyright 2012 Severalnines AB
# Run a replica of MySQL service$ docker service create \--name mysql-service \--replicas 1 \--net db_multi \-p 3307:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \--mount type=volume,source=ebs-datadir,destination=/var/lib/mysql,volume-driver=rexray \--mount type=volume,source=ebs-conf,destination=/etc/my.cnf.d,volume-driver=rexray \mysql:5.7
mysql-service
Docker
Host 1
ebs-datadir
mysql-service
Docker
Host 2
3307
ebs-conf
Docker Orchestration - Galera Cluster on Swarm
Copyright 2012 Severalnines AB
# Bootstrap-only container$ docker service create \--name pxc-bs \--replicas 1 \--net db-multi-hosts \-e MYSQL_ROOT_PASSWORD=mypassword123 \-e CLUSTER_NAME=my_galera \-e CLUSTER_JOIN= \percona/percona-xtradb-cluster
DockerHost 1
tasks.pxc1
# 3-node Galera$ for i in {1..3}; do docker service create \--name pxc${i} \--replicas 1 \--net db-multi-hosts \-p 330${i}:3306 \-v MYSQL_ROOT_PASSWORD=mypassword123 \-e CLUSTER_NAME=my_galera \-e CLUSTER_JOIN=tasks.pxc-bs,tasks.pxc1,tasks.pxc2,tasks.pxc3 \--volume pxc{i}:/var/lib/mysql \percona/percona-xtradb-cluster; \sleep 1m; \done
tasks.pxc-b
s
# Remove bootstrap-only container$ docker service rm pxc-bs
DockerHost 2
tasks.pxc2
DockerHost 3
tasks.pxc3
HAproxy
330133023303
330133023303
330133023303
3306
Docker Orchestration - Other Orchestration Tools
Copyright 2012 Severalnines AB
Docker Orchestration - Best Practice for MySQL
Copyright 2012 Severalnines AB
● Add label to node and use placement constraints:○ Avoid overcommit resources.○ Manageability and predictability.
● Use Service's name and virtual IP address to distinguish container's role.
● Understand the scheduling strategy:○ Spread○ Binpack○ Random○ Custom
● Rolling update:○ Only perform update one container at a time.○ Add delay before move to the next container.
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB
MySQL Container Management
MySQL Container - Service Control
Copyright 2012 Severalnines AB
# Get container's log$ docker logs -f mysql-staging...[Note] mysqld: Shutdown complete
# List processes inside container$ docker exec -it mysql-staging ps -e PID TTY TIME CMD 1 ? 00:00:00 mysqld 62 pts/0 00:00:00 bash 86 pts/0 00:00:00 ps
● mysqld must hold PID 1 inside container to receive the signal.
● docker stop might be dangerous. Default is 10s grace period. If still running, SIGKILL.
● Use docker kill with proper signal (SIGTERM). No grace period.
● Verify the shutdown process through docker log.● Stopping won’t delete the container.
# Stop MySQL container using kill$ docker kill --signal=TERM mysql-staging
# Stop MySQL container using stop$ docker stop mysql-staging
MySQL Container - Resource Control
Copyright 2012 Severalnines AB
# Limit memory and open file descriptors$ docker run -d \--name mysql-staging \--memory 4g \--memory-swap 4g \--ulimit nofile=16824:16824 \mysql:5.7.6
● By default, no resource constraints, use as much of a given resource as the host’s kernel will allow.
● Important options!○ --memory & --memory-swap
■ Must be equal to disable swap.■ If --memory-swap is not set, container
swap is default to --memory * 2○ --ulimit nofile={soft:hard}
■ Maximum number of files MySQL server can open simultaneously.
● Some of the container resources can be controlled dynamically using docker update:
○ Tune my.cnf accordingly.
# Increase container's memory.$ docker update \--memory 6g \--memory-swap 6g \mysql-staging
MySQL Container - Resource Monitoring
Copyright 2012 Severalnines AB
# Monitor containers$ docker stats --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}"NAME CPU % MEM USAGE / LIMITminio1 0.00% 6.078MiB / 7.78GiBmysql2 0.12% 194.1MiB / 7.78GiBmysql-local 0.14% 220MiB / 7.78GiB
● Use docker stats.● Inside container, free and top commands are not
accurate:○ Memory - /sys/fs/cgroup/memory/memory.*○ CPU - /sys/fs/cgroup/cpu/cpu.*○ IO - /sys/fs/cgroup/blkio/blkio.*
● External open-source monitoring tools:○ CLI: sysdig, dockviz○ UI: Portainer, Shipyard, Rancher, Prometheus
Portainer
MySQL Container - Configuration Management
Copyright 2012 Severalnines AB
# Bind mount my.cnf$ docker run -d \--name mysql1 \-e MYSQL_ROOT_PASSWORD=mypassword \-v /storage/mysql1/my.cnf:/etc/mysql/my.cnf \mysql:5.7.6
● Most MySQL configuration parameters can be changed during runtime. (Hint: SET GLOBAL)
● You can either:○ Use persistent storage:
■ Bind mount your my.cnf file.■ Create a named volume for /etc/my.cnf.d
○ Append the configuration as a "CMD" (right after image name), passes as flags to mysqld inside container.
● Standard MySQL and InnoDB optimization applies.
# Named volume for MySQL configuration directory$ docker run -d \--name mysql2 \-e MYSQL_ROOT_PASSWORD=mypassword \-v mysql2-conf:/etc/my.cnf.d \mysql:5.7.6
# Append configuration options$ docker run -d \--name mysql3 \-e MYSQL_ROOT_PASSWORD=mypassword \mysql:5.7.6 \--innodb_buffer_pool_size=1G \--max_connections=100
MySQL Container - Security
Copyright 2012 Severalnines AB
# Create a secret$ echo 'MyP2s$w0rD' | docker secret mysql_root_passwd -
● Docker Secrets○ Only for Swarm service.○ Accessible under /run/secrets/ inside
container.● Runtime privileges:
○ Unprivileged is default and recommended. Container is not allowed to access other devices.
○ Use --privileged only if you really need to modify kernel parameters (sysctl, /proc,/sys) or you want to run container inside container.
# Create a service and use the secret as root password$ docker service create \--name mysql-container \--publish 4406:3306 \--secret mysql_root_passwd \-e MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysql_root_passwd \-v local-datadir:/var/lib/mysql \mysql:5.7
MySQL Container - Backup
Copyright 2012 Severalnines AB
# mysqldump and store inside physical host$ docker exec -it mysql-prod mysqldump -uroot -p --single-transaction > /path/in/physical/host/dump.sql
● Logical backup:○ Mysqldump○ mysqlpump
● Xtrabackup (hot-backup):○ Install inside machine host.○ Install inside the MySQL container.○ Use perconalab/percona-xtrabackup image.
● Snapshot:○ MyISAM only - FTWRL + copy over○ Stop container + copy over
# FTWRL$ docker exec -it mysql-prod /bin/bashroot@mysql-prod:# mysql -uroot -pmysql> FLUSH TABLE WITH READ LOCK;
# Copy volume to another location (another terminal)$ cp /var/lib/docker/volumes/mysql-datadir/_data /destination/in/physical/host
# Release lock (same terminal as FTWRL)mysql> 'UNLOCK TABLES'
# Use perconalab/percona-xtrabackup image to backup$ docker run --rm -it \--net db-prod \-v mysql-datadir:/var/lib/mysql \-v /tmp/backup:/xtrabackup_backupfiles \ perconalab/percona-xtrabackup \--backup --host=mysql-prod --user=root --password=mypassword
MySQL Container - Restore
Copyright 2012 Severalnines AB
# Restore a MySQL dump$ docker exec -it mysql-staging mysql -uroot -p < /path/in/physical/host/dump.sql
● Logical backup:○ Fairly straightforward. Use standard procedure.
● Xtrabackup (hot-backup):○ Have to be prepared first○ Once prepared, simply starts a new container by
mounting the prepared directory as datadir.● Snapshot & Copy:
○ Bind mount:i. Direct mount the directory as a volume.
○ Named volume:i. Create a volume.ii. Copy the content to the volume's
directory.iii. Start a new container by mounting the
volume.
# Run a test container from a copied datadir (snapshot) or from a prepared xtrabackup directory.$ docker run -d \--name mysql-restored \-v /tmp/backup:/var/lib/mysql \mysql:5.7.16
# Prepare the backup created by xtrabackup$ docker run --rm -it \-v mysql-datadir:/var/lib/mysql \-v /tmp/backup:/xtrabackup_backupfiles \ perconalab/percona-xtrabackup \--prepare --target-dir /xtrabackup_backupfiles
MySQL Container - Logical Upgrade
Copyright 2012 Severalnines AB
1. Pull the new image, M.2. Start new container, B with new image, M.3. Export MySQL on A.4. Stop A.5. Import into B.6. Run mysql_upgrade inside B.7. OK? Remove A.8. Fallback? Stop B, start A.
# 1$ docker pull mysql:5.7.18# 2$ docker run -d \--name=mysql-new \-e MYSQL_ROOT_PASSWORD=mypassword \-v local-datadir:/var/lib/mysql \mysql:5.7.18# 3$ docker exec -it mysql-old mysqldump -uroot -p > dump.sql# 4$ docker kill --signal=TERM mysql-old# 5$ docker exec -it mysql-new mysql -uroot -p < dump.sql# 6$ docker exec -it mysql-new mysql_upgrade -uroot -p# 7$ docker rm -f mysqld-old# 8$ docker stop mysqld-new$ docker start mysqld-old
MySQL Container - In-Place Upgrade
Copyright 2012 Severalnines AB
# 1$ docker pull mysql:5.7.18# 2$ docker exec -it mysql-old mysql -uroot -p -e 'SET GLOBAL innodb_fast_shutdown = 0'# 3$ docker kill --signal=TERM mysqld-old# 4$ docker run -d \--name mysql-new \--publish 3307:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v local-datadir:/var/lib/mysql \mysql:5.7.18# 5$ docker exec -it mysql-new mysql_upgrade -uroot -p# 6$ docker rm -f mysqld-old# 7$ docker stop mysqld-new$ docker start mysqld-old
1. Pull the new image, M.2. Set innodb_fast_shutdown=0 inside container.3. Stop the container, A.4. Start a new container, B with new image, M.5. Run mysql_upgrade inside B.6. OK? Remove A.7. Fallback? Stop B, start A.
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB
What's Next?
Copyright 2017 Severalnines AB
● Visit our booth for a free T-shirt and other goodies. While stock lasts!
● Follow our MySQL on Docker blog series at severalnines.com/blog
● Attend the MySQL on Docker webinar, Wednesday Sept 27th, 2017:○ 09:00 BST / 10:00 CEST○ 09:00 PST (US) / 12:00 EST (US)
● Check out ClusterControl Docker image.
What's Next
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB
Q & A