pgconf.eu, tallinn, november 2016 honza horak
TRANSCRIPT
![Page 1: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/1.jpg)
Database Containers: Made by Distributions
Honza Horak <[email protected]>PgConf.eu, Tallinn, November 2016
![Page 2: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/2.jpg)
Database Containers Made by Distributions
The goal today
● Get feedback● Show how we think about containers● Convince you that you shouldn’t ignore them● Introduce technologies that we care about in Red Hat
![Page 3: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/3.jpg)
Database Containers Made by Distributions
Honza HorakBrno, Czech Republic
● Red Hat, Platform Engineering● Databases, Python, Ruby● RHEL, Fedora, CentOS● Software Collections
![Page 4: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/4.jpg)
Database Containers Made by Distributions
Honza HorakBrno, Czech Republic
● Red Hat, Platform Engineering● Databases, Python, Ruby● RHEL, Fedora, CentOS● Software Collections● AND CONTAINERS
![Page 5: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/5.jpg)
Database Containers Made by Distributions
What this talk includes
1. Containers basics2. Why containers matter3. PostgreSQL Docker container4. System containers5. Tools containers6. GUI apps in containers7. OS containers8. Ansible Containers9. OCI
![Page 6: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/6.jpg)
Database Containers Made by Distributions
1. CONTAINERS BASICS
![Page 7: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/7.jpg)
![Page 8: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/8.jpg)
Database Containers Made by Distributions
Containers and images
● Container○ Process isolated by Linux kernel features○ Virtualization technology
● Image○ Static container used to run containers
● Container is an instance of a (container) image
![Page 9: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/9.jpg)
Database Containers Made by Distributions
Container is not a virtual machine
Infrastructure
Host OS (kernel)
Container Engine (Docker daemon)
Bin/Libs Bin/Libs Bin/Libs
App App App
Linux Containers (e.g. Docker)
Infrastructure
Host OS (kernel)
Hypervisor
Bin/Libs Bin/Libs Bin/Libs
App App App
Guest OS Guest OS Guest OS
Traditional Virtual Machine
![Page 10: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/10.jpg)
Database Containers Made by Distributions
Container is not a virtual machine
Infrastructure
Host OS (kernel)
Docker Engine
Bin/Libs Bin/Libs Bin/Libs
App App App
Linux Containers (e.g. Docker)
Infrastructure
Host OS (kernel)
Hypervisor
Bin/Libs Bin/Libs Bin/Libs
App App App
Guest OS Guest OS Guest OS
Traditional Virtual Machine
![Page 11: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/11.jpg)
Database Containers Made by Distributions
Container is not a virtual machine
Infrastructure
Host OS (kernel)
Docker Engine
Bin/Libs Bin/Libs Bin/Libs
App App App
Linux Containers (e.g. Docker)
Infrastructure
Host OS (kernel)
Hypervisor
Bin/Libs Bin/Libs Bin/Libs
App App App
Guest OS Guest OS Guest OS
Traditional Virtual Machine
![Page 12: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/12.jpg)
Database Containers Made by Distributions
Container is not a virtual machine
Infrastructure
Host OS (kernel)
Docker Engine
Bin/Libs Bin/Libs Bin/Libs
App App App
Linux Containers (e.g. Docker)
Infrastructure
Host OS (kernel)
Hypervisor
Bin/Libs Bin/Libs Bin/Libs
App App App
Guest OS Guest OS Guest OS
Traditional Virtual Machine
![Page 13: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/13.jpg)
Database Containers Made by Distributions
Container is not a virtual machine
Infrastructure
Host OS (kernel)
Docker Engine
Bin/Libs Bin/Libs Bin/Libs
App App App
Linux Containers (e.g. Docker)
Infrastructure
Host OS (kernel)
Hypervisor
Bin/Libs Bin/Libs Bin/Libs
App App App
Guest OS Guest OS Guest OS
Traditional Virtual Machine
![Page 14: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/14.jpg)
Use only content you trust.
![Page 15: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/15.jpg)
It is sitting in the corner, it is green and very dangerous.
What is it?
![Page 16: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/16.jpg)
![Page 17: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/17.jpg)
Avoid root inside container or use user namespaces.
![Page 18: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/18.jpg)
Database Containers Made by Distributions
2. WHY CONTAINERS MATTER?
![Page 19: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/19.jpg)
Database Containers Made by Distributions
Containers gain popularity
● New kernel features (namespaces, cgroups, …)● Clouds love containers● Easy app installation on Android● Docker introduced simple packaging format● DevOps● Kubernetes (k8s)
○ open source container cluster manager
![Page 20: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/20.jpg)
Database Containers Made by Distributions
3. POSTGRESQL DOCKER CONTAINER
![Page 21: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/21.jpg)
Database Containers Made by Distributions
#> cat DockerfileFROM rhel7:7.2RUN yum -y install postgresql-server && \ yum clean all
#> docker build .
Installing RPMs in container
21
![Page 22: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/22.jpg)
Database Containers Made by Distributions
#> docker build .Sending build context to Docker daemon 2.048 kBStep 1 : FROM rhel7:7.2 ---> c453594215e4
...
Installed: postgresql-server.x86_64 0:9.2.15-1.el7_2
Dependency Installed: postgresql.x86_64 0:9.2.15-1.el7_2 postgresql-libs.x86_64 0:9.2.15-1.el7_2 systemd-sysv.x86_64 0:219-19.el7_2.9
Complete! ---> 1ff2f2d0bc66Removing intermediate container 2a86d8a78b75Successfully built 1ff2f2d0bc66
Installing RPMs in container
22
![Page 23: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/23.jpg)
Database Containers Made by Distributions
Docker and LayersWhat is gonna be part of distribution and what is beyond.
MyApp DB Master
RHEL Base Image 7.2
Python 3.5
Django App
PHP + Apache
Wordpress
PostgreSQL
MyApp DB Slave
Dis
tribu
tion
Dev
Ops
MyApp DB Backup NodeBlog
![Page 24: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/24.jpg)
Database Containers Made by Distributions
Docker and LayersWhat is gonna be part of distribution and what is beyond.
MyApp DB Master
RHEL Base Image 7.2
Python 3.5
Django App
PHP + Apache
Wordpress
PostgreSQL
MyApp DB Slave
Dis
tribu
tion
Dev
Ops
MyApp DB Backup NodeBlog
glibc fix
![Page 25: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/25.jpg)
Database Containers Made by Distributions
Docker and LayersWhat is gonna be part of distribution and what is beyond.
MyApp DB Master
RHEL Base Image 7.2
Python 3.5
Django App
PHP + Apache
Wordpress
PostgreSQL
MyApp DB Slave
Dis
tribu
tion
Dev
Ops
MyApp DB Backup NodeBlog
glibc fix
![Page 26: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/26.jpg)
Database Containers Made by Distributions
Docker and LayersWhat is gonna be part of distribution and what is beyond.
MyApp DB Master
RHEL Base Image 7.2
Python 3.5
Django App
PHP + Apache
Wordpress
PostgreSQL
MyApp DB Slave
Dis
tribu
tion
Dev
Ops
MyApp DB Backup NodeBlog
glibc fix
![Page 27: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/27.jpg)
Database Containers Made by Distributions
Docker and LayersWhat is gonna be part of distribution and what is beyond.
MyApp DB Master
RHEL Base Image 7.2
Python 3.5
Django App
PHP + Apache
Wordpress
PostgreSQL
MyApp DB Slave
Dis
tribu
tion
Dev
Ops
MyApp DB Backup NodeBlog
glibc fix
![Page 28: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/28.jpg)
Automate.
![Page 29: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/29.jpg)
Automate.Everything.
![Page 30: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/30.jpg)
Database Containers Made by Distributions
Containers in Red Hat...end in OpenShift
● PaaS on top of k8s● Building included● Whole app lifecycle● Aims on DevOps● It scales!● OpenShift v3.3 GA
https://blog.openshift.com/red-hat-container-platform-3-3-general-availability/
![Page 31: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/31.jpg)
Database Containers Made by Distributions
#> docker build -t hhorak/postgresql .…
#> docker run -ti hhorak/postgresqlbash-4.2$ rpm -q postgresql-serverpostgresql-server-9.2.15-1.el7_2.x86_64
Correct RPMs are in a container
31
![Page 32: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/32.jpg)
So we have PostgreSQL in a Linux container.
Are we there yet?
![Page 33: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/33.jpg)
Database Containers Made by Distributions
#> cat Dockerfile
FROM rhel7:7.2
RUN yum -y install postgresql-server && yum clean all
ENV HOME=/var/lib/pgsqlENV PGDATA=/var/lib/pgsql/dataENV PGUSER=postgresUSER 26
ADD run-postgresql /usr/bin/CMD [ "/usr/bin/run-postgresql" ]
Make container do something
33
![Page 34: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/34.jpg)
Database Containers Made by Distributions
#> cat run-postgresql
#!/bin/bash
initdb
echo "host all all 0.0.0.0/0 md5" >${PGDATA}/pg_hba.confecho "listen_addresses = '*'" >${PGDATA}/postgresql.conf
exec postgres "$@"
Make container do something
34
Who said microservice?
![Page 35: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/35.jpg)
Database Containers Made by Distributions
#> docker run -ti -p 5432:5432 --name p1 postgresql
#> docker inspect --format='{{.NetworkSettings.IPAddress}}' p1172.17.0.2
#> psql -h 172.17.0.2Password: _
Connecting to PostgreSQL container
35
![Page 36: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/36.jpg)
Database Containers Made by Distributions
#> docker run -ti -p 5432:5432 --name p1 postgresql
#> docker inspect --format='{{.NetworkSettings.IPAddress}}' p1172.17.0.2
#> psql -h 172.17.0.2Password: _
Connecting to PostgreSQL container
36
![Page 37: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/37.jpg)
Do not use default passwords.
![Page 38: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/38.jpg)
Database Containers Made by Distributions
#> cat run-postgresql
...echo "host all all 0.0.0.0/0 md5" >${PGDATA}/pg_hba.confecho "local all postgres peer" >>${PGDATA}/pg_hba.confecho "listen_addresses = '*'" >${PGDATA}/postgresql.conf
pg_ctl -w start -o "-h ''"psql --command "ALTER USER \"postgres\" WITH ENCRYPTED PASSWORD '${POSTGRESQL_ADMIN_PASSWORD}';"pg_ctl stop
exec postgres "$@"
Connecting to PostgreSQL container
38
![Page 39: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/39.jpg)
Database Containers Made by Distributions
#> docker run -ti -d -p 5432:5432 --name p1 \ -e POSTGRESQL_ADMIN_PASSWORD=pass postgresqlb1e23c844346d2788d7b7891d8f78244788f71b19dcf291b05cdf1d7685ef556
#> psql -h 172.17.0.2 -U postgresPassword for user postgres:psql (9.2.15, server 9.2.15)Type "help" for help.
postgres=# _
Connecting to PostgreSQL container
39
![Page 40: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/40.jpg)
Database Containers Made by Distributions
if [ -e "/run/secrets/pgusers/user/username" ] ; then POSTGRESQL_USER="$(</run/secrets/pgusers/user/username)" POSTGRESQL_PASSWORD="$(</run/secrets/pgusers/user/password)"fi
Or use k8s secrets
40
http://kubernetes.io/docs/user-guide/secrets/
![Page 41: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/41.jpg)
How to configure such a database?
![Page 42: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/42.jpg)
Database Containers Made by Distributions
#> cat run-postgresql...echo "max_connections = ${POSTGRESQL_MAX_CONNECTIONS}" >>${PGDATA}/postgresql.conf...
Configuring PostgreSQL container
42
![Page 43: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/43.jpg)
Database Containers Made by Distributions
Example of PostgreSQL 9.5 container
#> docker run -d \ -p 5432:5432 \ -e POSTGRESQL_ADMIN_PASSWORD=secret \ -e POSTGRESQL_MAX_CONNECTIONS=10 \ -e POSTGRESQL_USER=guestbook \ -e POSTGRESQL_PASSWORD=pass \ -e POSTGRESQL_DATABASE=guestbook \ -v /db:/var/lib/pgsql/data:Z \ rhscl/postgresql-95-rhel7
43
![Page 44: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/44.jpg)
Support the most common configuration (let users to change them in OpenShift)
and allow users to build their own specific layered container images easily.
![Page 45: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/45.jpg)
How?
![Page 46: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/46.jpg)
Database Containers Made by Distributions
Extending PostgreSQL container
#> cat DockerfileFROM rhscl/postgresql-95-rhel7COPY post-init-hooks ${CONTAINER_SCRIPTS_PATH}/hooks.d
#> cat post-init-hooks/post-initecho max_wal_senders=10" >> "${POSTGRESQL_CONFIG_FILE}"echo "lock_timeout=${POSTGRESQL_LOCK_TIMEOUT}" >> "${POSTGRESQL_CONFIG_FILE}"psql -f initdb.sql
#> docker build -t hhorak/postgresql-95-rhel7 .
46
![Page 47: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/47.jpg)
Database Containers Made by Distributions
Extending PostgreSQL container
#> cat DockerfileFROM rhscl/postgresql-95-rhel7COPY post-init-hooks ${CONTAINER_SCRIPTS_PATH}/hooks.d
#> cat post-init-hooks/post-initecho max_wal_senders=10" >> "${POSTGRESQL_CONFIG_FILE}"echo "lock_timeout=${POSTGRESQL_LOCK_TIMEOUT}" >> "${POSTGRESQL_CONFIG_FILE}"psql -f initdb.sql
#> docker build -t hhorak/postgresql-95-rhel7 .
47
![Page 48: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/48.jpg)
Database Containers Made by Distributions
Extending PostgreSQL container
#> cat DockerfileFROM rhscl/postgresql-95-rhel7COPY post-init-hooks ${CONTAINER_SCRIPTS_PATH}/hooks.d
#> cat post-init-hooks/post-initecho max_wal_senders=10" >> "${POSTGRESQL_CONFIG_FILE}"echo "lock_timeout=${POSTGRESQL_LOCK_TIMEOUT}" >> "${POSTGRESQL_CONFIG_FILE}"psql -f initdb.sql
#> docker build -t hhorak/postgresql-95-rhel7 .
48
![Page 49: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/49.jpg)
Database Containers Made by Distributions
Extending PostgreSQL container
#> cat DockerfileFROM rhscl/postgresql-95-rhel7COPY post-init-hooks ${CONTAINER_SCRIPTS_PATH}/hooks.d
#> cat post-init-hooks/post-initecho max_wal_senders=10" >> "${POSTGRESQL_CONFIG_FILE}"echo "lock_timeout=${POSTGRESQL_LOCK_TIMEOUT}" >> "${POSTGRESQL_CONFIG_FILE}"psql -f initdb.sql
#> docker build -t hhorak/postgresql-95-rhel7 .
49
![Page 50: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/50.jpg)
Database Containers Made by Distributions
Extending PostgreSQL container
#> cat DockerfileFROM rhscl/postgresql-95-rhel7COPY post-init-hooks ${CONTAINER_SCRIPTS_PATH}/hooks.d
#> cat post-init-hooks/post-initecho max_wal_senders=10" >> "${POSTGRESQL_CONFIG_FILE}"echo "lock_timeout=${POSTGRESQL_LOCK_TIMEOUT}" >> "${POSTGRESQL_CONFIG_FILE}"psql -f initdb.sql
#> docker build -t hhorak/postgresql-95-rhel7 .
50
![Page 51: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/51.jpg)
So we build micro-services that will be run in OpenShift.
![Page 52: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/52.jpg)
So we build micro-services that will be run in OpenShift.
OpenShift likes scaling.
![Page 53: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/53.jpg)
So we build micro-services that will be run in OpenShift.
OpenShift likes scaling.How to make databases to scale?
![Page 54: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/54.jpg)
Database Containers Made by Distributions
Scalable PostgreSQL container
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 run-postgresql-master
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 -e POSTGRESQL_MASTER_IP=172.16.12.2 run-postgresql-slave
54
![Page 55: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/55.jpg)
Database Containers Made by Distributions
Scalable PostgreSQL container
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 run-postgresql-master
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 -e POSTGRESQL_MASTER_IP=172.16.12.2 run-postgresql-slave
55
![Page 56: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/56.jpg)
Database Containers Made by Distributions
Scalable PostgreSQL container
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 run-postgresql-master
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 -e POSTGRESQL_MASTER_IP=172.16.12.2 run-postgresql-slave
56
![Page 57: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/57.jpg)
Database Containers Made by Distributions
Scalable PostgreSQL container
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 run-postgresql-master
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 -e POSTGRESQL_MASTER_IP=172.16.12.2 run-postgresql-slave
57
![Page 58: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/58.jpg)
Database Containers Made by Distributions
Scalable PostgreSQL container
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 run-postgresql-master
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 -e POSTGRESQL_MASTER_IP=172.16.12.2 run-postgresql-slave
58
![Page 59: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/59.jpg)
Database Containers Made by Distributions
Scalable PostgreSQL container
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 run-postgresql-master
$ docker run -d --name pg_master -e POSTGRESQL_ADMIN_PASSWORD=pass -e POSTGRESQL_MASTER_USER=user -e POSTGRESQL_MASTER_PASSWORD=pass -p 5432:5432 rhscl/postgresql-95-rhel7 -e POSTGRESQL_MASTER_IP=172.16.12.2 run-postgresql-slave
59
![Page 60: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/60.jpg)
For more complex replication users are expected to build own layer.
![Page 61: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/61.jpg)
Database Containers Made by Distributions
Auto-tuning in PostgreSQL containerContainers may be slim or fat.
# Get available memory for containerMEMORY_IN_BYTES=$(cat /sys/fs/cgroup/memory/memory.limit_in_bytes)
# Use 1/4 of given memory for shared buffersPOSTGRESQL_SHARED_BUFFERS="$(($MEMORY_IN_BYTES/1024/1024/4))MB" # Setting effective_cache_size to 1/2 of total memoryPOSTGRESQL_EFFECTIVE_CACHE_SIZE="$(($MEMORY_IN_BYTES/1024/1024/2))MB"
# postgresql.conf is later generated as:shared_buffers = ${POSTGRESQL_SHARED_BUFFERS}effective_cache_size = ${POSTGRESQL_EFFECTIVE_CACHE_SIZE}
61
![Page 62: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/62.jpg)
For working code, see:https://github.com/sclorg/postgresql-container
![Page 63: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/63.jpg)
Database Containers Made by Distributions
Or play with container directly
#> docker pull centos/postgresql-94-centos7#> docker pull centos/postgresql-95-centos7
#> docker pull registry.access.redhat.com/rhscl/postgresql-94-rhel7#> docker pull registry.access.redhat.com/rhscl/postgresql-95-rhel7
63
![Page 64: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/64.jpg)
Database Containers Made by Distributions
4. SYSTEM CONTAINERS
![Page 65: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/65.jpg)
Database Containers Made by Distributions
Run container as systemd servicei.e. replace daemon with container
We need to:
1. Create Docker container2. Create systemd unit file for the service3. Work with the systemd unit as usually
![Page 66: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/66.jpg)
Database Containers Made by Distributions
#> docker create --name postgresql-service -e … -v /var/lib/pgsql:/var/lib/pgsql:Z fedora/postgresql
Run container as systemd service1. Create Docker container (but not run)
![Page 67: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/67.jpg)
Database Containers Made by Distributions
# cat /etc/systemd/system/postgresql-cont.service[Unit]Description=PostgreSQL service as a docker containerAfter=docker.service
[Service]ExecStart=/usr/bin/docker start postgresql-contExecStop=/usr/bin/docker stop postgresql-cont
[Install]WantedBy=multi-user.target
Run container as systemd service2. Create systemd unit file for the service
![Page 68: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/68.jpg)
Database Containers Made by Distributions
#> systemctl enable postgresql-cont.service#> systemctl start postgresql-cont.service
Run container as systemd service3. Work with the systemd service as usually
![Page 69: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/69.jpg)
Not every container is a daemon
![Page 70: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/70.jpg)
Database Containers Made by Distributions
5. TOOLS CONTAINERS
![Page 71: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/71.jpg)
Database Containers Made by Distributions
#> docker run -ti hhorak/postgresql-tools pgbench …
#> docker run -ti hhorak/postgresql-tools pg_standby …
Tools to manage daemons(that are not part of the daemon image)
Interaction is easy, we can use network socket to work with daemon.
![Page 72: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/72.jpg)
Database Containers Made by Distributions
#> docker exec -ti postgresql1 pg_dump >/home/hhorak/dump.sql
#> docker run -ti -v /:/host hhorak/postgresql bashbash-4.2$ pg_dump >/host/home/hhorak/dump.sqlbash-4.2$ ...
How to interact with host
![Page 73: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/73.jpg)
Database Containers Made by Distributions
6. GUI APPS IN CONTAINERS
![Page 74: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/74.jpg)
Database Containers Made by Distributions
Why GUI in containers
● Some level of isolation○ Filesystem, Cgroups, Namespaces
● Deps bundling →one app running on any Linux● Android-like app store for all distros● Clean system underneath
![Page 75: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/75.jpg)
Database Containers Made by Distributions
GUI in DockerWell, it sometimes even works, but..
docker run -ti --rm \ -e DISPLAY=$DISPLAY \ -v /tmp/.X11-unix:/tmp/.X11-unix \ -u 1001 \ hhorak/pgadmin3
![Page 76: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/76.jpg)
Database Containers Made by Distributions
GUI in Flatpak
● Designed for GUI apps● Sandbox rather than full container● Runtimes define common environment to build on● Runs in desktop session
aka xdg-app
![Page 77: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/77.jpg)
Database Containers Made by Distributions
Flatpak design
http://flatpak.org/#developers
77
![Page 78: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/78.jpg)
Database Containers Made by Distributions
Working with Flatpak applicationhttp://flatpak.org/developer.html
$ tar xvf pgadmin3-1.22.1.tar.gz$ cd pgadmin3-1.22.1
$ flatpak build ../dictionary ./configure --prefix=/app$ flatpak build ../dictionary make$ flatpak build ../dictionary make install
78
$ flatpak install gnome org.gnome.Platform 3.20$ flatpak install gnome-apps org.postgresql.pgadmin3 stable
$ flatpak run org.postgresql.pgadmin3
![Page 79: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/79.jpg)
Database Containers Made by Distributions
7. OS CONTAINERS
![Page 80: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/80.jpg)
Database Containers Made by Distributions
MultiContainer vs. OS Container
VS
![Page 81: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/81.jpg)
Database Containers Made by Distributions
Operating System Container
● Some applications work better when running on one machine● Handy for transition period
○ Part of the services as containers, rest as standard services
● Safe zombie handling● Container’s journald integration
Running whole OS (systemd or other init process) inside container.
![Page 82: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/82.jpg)
Database Containers Made by Distributions
Running systemd inside Dockerhttp://developers.redhat.com/blog/2016/09/13/running-systemd-in-a-non-privileged-container
docker run -ti --tmpfs /run --tmpfs /tmp -v /sys/fs/cgroup:/sys/fs/cgroup:ro <your-image>
82
FROM fedora:24RUN dnf -y install postgresql-server; dnf clean allRUN systemctl enable postgresqlSTOPSIGNAL SIGRTMIN+3EXPOSE 80CMD [ "/sbin/init" ]
![Page 83: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/83.jpg)
Database Containers Made by Distributions
OS Container using systemd-nspawn
● better connection with host (logging, machinectl, ...)● no image management, no containers linking● good enough for testing or debugging something at container level
Running whole OS (including systemd) inside container.
![Page 84: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/84.jpg)
Database Containers Made by Distributions
Running systemd inside nspawnhttps://www.variantweb.net/blog/using-systemd-nspwan-for-lightweight-container-in-fedora-21/
dnf -y --releasever=24 --nogpg --installroot=/var/tmp/testnspawn --disablerepo='*' --enablerepo=fedora --enablerepo=updates install systemd passwd yum fedora-release postgresql-server @standardchcon -R -t svirt_sandbox_file_t /var/tmp/testnspawnrestorecon -R /var/tmp/testnspawn
84
systemd-nspawn -D /var/tmp/testnspawnpasswdpostgresql-setup --initsystemctl enable postgresql
systemd-nspawn -bD /var/tmp/testnspawn
![Page 85: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/85.jpg)
Who likes Ansible?
![Page 86: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/86.jpg)
Database Containers Made by Distributions
8. Ansible containers
![Page 87: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/87.jpg)
Database Containers Made by Distributions
Ansible containers
● Building containers using popular Ansible● Orchestrating containers like other services● Especially useful for transition to containers
![Page 88: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/88.jpg)
Database Containers Made by Distributions
Ansible Container https://github.com/ansible/ansible-container-examples/tree/master/wordpress
88
services: db: image: rhel:7 ports: - "5432:5432" command: ['/usr/bin/pg_ctl', '-w', 'start'] wordpress: image: rhel:7 ports: - "80:80" links: - dbcommand: bash -c "bash /tmp/a.sh ; usr/sbin/apachectl -D FOREGROUND"
![Page 89: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/89.jpg)
Database Containers Made by Distributions
Ansible container - name: postgresql-init shell: postgresql-setup --upgrade
- name: run postgres shell: pg_ctl -w start - name: Create postgres database pgsql_db: name: "{{ wp_pgsql_db }}" state: present - name: Create postgres user pgsql_user: name: "{{ wp_pgsql_user }}" password: "{{ wp_pgsql_password }}" state: present priv: "*.*:ALL,GRANT" host: "%"
- hosts: db vars: - wp_pgsql_db: wordpress - wp_pgsql_user: wordpress - wp_pgsql_password: password tasks: - name: Install postgresql-server yum: name: "{{ item }}" state: latest with_items: - postgresql-server
- name: Update the repository shell: yum -y erase vim-minimal && \ yum -y update && \ yum clean all
![Page 90: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/90.jpg)
Database Containers Made by Distributions
Ansible Container https://github.com/ansible/ansible-container-examples/tree/master/wordpress
90
$> ansible-container init -- do changes -- $> ansible-container build $> ansible-container run $> docker login $> ansible-container push $> ansible-container shipit openshift
![Page 91: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/91.jpg)
Database Containers Made by Distributions
9. OCI
![Page 92: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/92.jpg)
Database Containers Made by Distributions
9. OCIOpen Container Initiative
![Page 93: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/93.jpg)
Database Containers Made by Distributions
Docker
rktAnsiblecontainer
nspawn
![Page 94: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/94.jpg)
Database Containers Made by Distributions
"The mission of the OCI is to promote and promulgate
a set of common, minimal, open standards
and specifications around container technology."
https://www.opencontainers.org
![Page 95: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/95.jpg)
Database Containers Made by Distributions
Open Container InitiativeCollaboration of Red Hat, Google, Docker, and others big players in containers world.
● libcontainer - https://github.com/docker/libcontainer ● runc - https://github.com/opencontainers/runc ● Open Container Format
![Page 96: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/96.jpg)
Database Containers Made by Distributions
DevConf.czBrno, Czech RepublicJanuary 27 - 29, 2017
#DEFINEfuture
A free annual community conference for developers, admins, and users of
Linux & JBoss.
![Page 97: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/97.jpg)
Thanks.
OpenShift: https://docs.openshift.com Sources of Docker images: https://github.com/sclorg/Project Atomic: https://www.projectatomic.io
Honza Horak <[email protected]>@HonzaHorak
![Page 98: PgConf.eu, Tallinn, November 2016 Honza Horak](https://reader030.vdocuments.site/reader030/viewer/2022041100/5ed73dd5d37f9f58ca6a880c/html5/thumbnails/98.jpg)
Do not forget,content does matter.
Honza Horak <[email protected]>@HonzaHorak
https://hhorak.fedorapeople.org/2016/161031_pgconf_databases_containers.pdf