docker concepts for oracle/mysql dbas and devops
TRANSCRIPT
Docker Concepts for Oracle/MySQL DBAs and DevOps
Zohar Elkayam CTO, Brillix
Twitter: @realmgic
2
• Zohar Elkayam, CTO at Brillix
• Programmer, DBA, team leader, database trainer, public speaker, and a senior consultant for over 19 years
• Oracle ACE Since 2014
• ilOUG Board Member – Israel Oracle User Group
• Blogger – www.realdbamagic.com and www.ilDBA.co.il
Who am I?
3
About Brillix
• We offer complete, integrated end-to-end solutions based on best-of-breed innovations in database, security and big data technologies
• We provide complete end-to-end 24x7 expert remote database services
• We offer professional customized on-site trainings, delivered by our top-notch world recognized instructors
4
Some of Our Customers
5
• Founded in 1999, over 7000 members at its peak
• Divided into Technologies (Databases) and Applications groups
• Free Bi-monthly meetups for DBAs and Developers at Oracle Israel offices
• Yearly conference with guests speakers from all over the world:• January 22-23, 2018• Over 30 Speakers from all over the world: Ask TOM team, Oracle product mangers
and Oracle ACEs from 9 different countries!
• You can join our community: meetup group, facebook group or follow us on twitter (@ilOUG_org)!
• Our website: www.ilOUG.org
ilOUG – Israel Oracle User Group
6
Agenda
• Docker overview – why do we even need containers?
• Installing Docker and getting started
• Images and Containers
• Docker Networks
• Docker Storage and Volumes
• Oracle and Docker
• Docker tools, GUI and Swarm
7
Our Goal Today
• Learn what Docker is and how to use it
• See how Docker containers might be helpful with Oracle
• We will not expert everything
• Did you hear about Oracle 18c?
• This is a starting point – don’t be afraid to try!
8
.בסיום יום הסמינר יחולק טופס משוב ונשמח לקבל את חוות דעתכם•
!היא זו המאפשרת לנו לעשות את הסמינרים טובים יותר:דעתכם חשובה•
Our REAL Agenda
הפסקה10:30-10:45
י הכנס בגן המלוןארוחת צהריים לכל משתתפ12:30-13:30
הפסקה מתוקה במתחם קבלת הפנים15:00-15:15
הולכים הביתה16:30
Docker
10
What is Docker?
• Docker is an open-source project that automates the deployment of applications inside software containers, by providing an additional layer of abstraction and automation of operating system–level virtualization on Linux
11
What’s the Buzz Around Containers?
• The software industry has changed dramatically
• 5-10 years ago:• Monolithic applications• Long development cycles• Slowly scaling up
• Now:• Decoupled services, Microservices• Fast, iterative improvements• Quickly scaling out
• Google claims all of its services now run in containers
12
It’s All About CI/CD
• CI/CD: Continuous Integration and Continuous Deployment
• We want to always integrate new things into our applications and have a fast deployment
• The challenge: development and deployment became complex: multiple stacks, multiple targets. Keeping everything in sync and deploying a reliable environment in short spans to production is hard
• Docker Goal: Provide a uniformed wrapper around a software package: «Build, Ship and Run Any App, Anywhere»
• Using Docker to achieve CI/CD is the most common usage for Docker today
13
The Analogy: Cargo Transport
14
The Analogy: Cargo Transport Solution
15
Docker Solution: Containers
16
Docker Containers
• The concept: One Application per Container
• Self contained - everything you need is in an isolated environment
• Isolation for security - containers not aware of each other
• Isolation for performance – hard limits on CPU, RAM and IO per container
17
VMs vs. Containers
• Less overhead but It’s not a lightweight VM – it’s a Software Delivery Platform
• It's about isolation - Containers run on Linux kernel of host and are visible on host
18
Why Use Docker Containers
• It’s highly portable
• It Runs everywhere: bare metal, VM, and even the cloud (AWS, GCP, Oracle Cloud, Microsoft Azure)
• Small, lightweight and highly scalable
• Better performance than VMs, smaller foot print
• Very cost effective
19
Why Does IT Cares About Docker?
Developers care because:
• Quickly create ready-to-run packaged applications
• A clean, safe, hygienic, portable runtime environment
• No missing/conflicting dependencies or packages
• Each app runs in an isolated container
• Automate testing, integration, packaging
• Reduce/eliminate platform compatibility issues
• Cheap/zero cost deployment, with instant replay and reset
20
Why Does IT Cares About Docker? (2)
Administrators care because:
• Configure once, run many times
• Makes app lifecycle efficient, consistent and repeatable
• Eliminate environment inconsistencies between development, test, production
• Supports segregation of duties
• Improve speed and reliability of continuous integration and deployment
• Lightweight containers address performance, costs, deployment and portability issues
21
Common Use Cases
• Development and Test Environment
• Micro-services
• Application Deployment without “dependency hell” of multiple applications
• It Solves the “It Worked For Me!” issue
• Continuous Integration
• Platform-as-a-Service (PaaS)
22
Docker Inc.
• Founded in 2009
• Primary sponsor of the Docker Project• Hires maintainers and contributors• Provides infrastructure for the project• Runs the Docker Hub and Store
• How does Docker Inc. make money?• Docker Hub has free and paid services• DTR (Docker Trusted Registry)• Enterprise support for Engine and other products• Training and professional services
23
Docker Adoption Graph
24
Docker Adoption
Installing Docker
26
Let’s Begin: Installing Docker
• Docker is easy to install• Runs on practically every Linux distribution (Kernel must be 2.6.32+ or
3.10+)• Runs on OS X via a virtual machine• Microsoft Windows (10) via virtual machine
• Community Edition (CE) and Enterprise Edition are available• EE has support, and some certifications to hardware/software• CE has the community support and is way more common
• Latest version is 17.09 (released on September, 2017), I still use 17.06.2…
27
Docker on Windows 10
• Make sure your virtualization is on:
28
Docker on Windows
• Download the Docker Toolbox from https://www.docker.com/products/docker-toolbox
• That will install:• Docker Client for Windows
• Docker Toolbox management tool and ISO
• Oracle VM VirtualBox
• Git MSYS-git UNIX tools
• Click the “QuickStart Terminal” on your desktop and enable the User Account Control permission
29
Installing Docker: Oracle Linux 7
• Unlike other distros, Oracle Linux doesn’t enable the yum repository holding Docker by default, so we need to first enable it:
• Use yum to install Docker engine:
[root@lnx7-oracle-1 ~]# vi /etc/yum.repos.d/public-yum-ol7.repo
...
[ol7_addons]
name=Oracle Linux $releasever Add ons ($basearch)
baseurl=http://yum.oracle.com/repo/OracleLinux/OL7/addons/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1
[root@lnx7-oracle-1 ~]# yum install docker-engine
30
Installing Docker: Oracle Linux 7
• Create a btrfs mount point for the docker file system
• This filesystem will provide better performance and copy-on-write support
• More on copy-on-write and btrfs - later
mkfs.btrfs -L btrfs1 /dev/sdb
echo “LABEL=btrfs1 /var/lib/docker btrfs defaults 0 1” >>/etc/fstab
mkdir -p /var/lib/docker
mount /var/lib/docker
31
Installing Docker: Oracle Linux 7
• Start the Docker daemon and enable it to start automatically:[root@lnx7-oracle-1 ~]# systemctl start docker
[root@lnx7-oracle-1 ~]# systemctl enable docker
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service
to /usr/lib/systemd/system/docker.service.
[root@lnx7-oracle-1 ~]# systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor
preset: disabled)
Drop-In: /etc/systemd/system/docker.service.d
└─docker-sysconfig.conf
Active: active (running) since Thu 2017-11-16 21:30:31 IST; 13min ago
Docs: https://docs.docker.com
Main PID: 881 (dockerd)
Memory: 18.6M
CGroup: /system.slice/docker.service
├─881 /usr/bin/dockerd --selinux-enabled
32
Check Docker Version
• After install, verify the docker version and docker info[root@lnx7-oracle-1 ~]# docker version
Client:
Version: 17.06.2-ol
API version: 1.30
Go version: go1.8.3
Git commit: d02b7ab
Built: Fri Oct 6 00:02:23 2017
OS/Arch: linux/amd64
Server:
Version: 17.06.2-ol
API version: 1.30 (minimum version 1.12)
Go version: go1.8.3
Git commit: d02b7ab
Built: Fri Oct 6 00:03:48 2017
OS/Arch: linux/amd64
Experimental: false
33
The docker Group
• Add a docker group
• Add a user to the docker group
• Restart the docker deamon
• Relogin to the use the user, without using sudo to run things from root
$ sudo groupadd docker
$ sudo gpasswd -a $USER docker
$ sudo systemctl restart docker.service
34
Docker and Security
• The docker user is root equivalent
• It provides root-level access to the host
• You should restrict access to it like you would protect root
• If you give somebody the ability to access the Docker API, you are giving them full access on the machine
• Therefore, the Docker control socket is (by default) owned by the docker group, to avoid unauthorized access on multi-user machines
Getting Started:Working With Docker
36
Docker Components
37
Docker Architecture
• The Docker daemon (or "Engine"): receives and processes incoming Docker API requests
• The Docker client talks to the Docker daemon via the Docker API• We'll use mostly the CLI embedded within the docker binary
• GUI tools available, we will talk about them later
• Docker Hub Registry (and Docker Store): Collection of public images• The Docker daemon talks to it via the registry API
38
Hello World: Busybox
• Let’s run a container that echo “hello world”
• This started a very small (>1 MB) container called Busybox, and ran the single command we provided
• Busybox is usually used for embedded system (routers, switches, etc.)
[zohar@lnx7-oracle-1 ~]$ docker run --rm busybox echo hello world
Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
0ffadd58f2a6: Pull complete
Digest: sha256:bbc3a03235220b170ba48a157dd097dd1379299370e1ed99ce976df0355d24f0
Status: Downloaded newer image for busybox:latest
hello world
39
Using Shell to Busybox
• Let’s connect to that container and have a look around
• We used -it option to indicate “interactive” and “tty” (terminal)
[zohar@lnx7-oracle-1 ~]$ docker run --rm -it busybox sh
/ # ps -ef
PID USER TIME COMMAND
1 root 0:00 sh
5 root 0:00 ps -ef
/ # uname -a
Linux a451342a146a 4.1.12-103.9.2.el7uek.x86_64 #2 SMP Tue Oct 31 16:43:46 PDT
2017 x86_64 GNU/Linux
/ #
40
Which Containers are Up?
• We can check which containers are up using docker ps
• We can check which container was the last to start by using docker ps -l
41
Detaching an Interactive Container
• We can detach the interactive container by using CTRL+P,Q
• When checking on the host what containers are up, we can see it’s running in the background
• We can re-attach back to it by using docker attach command
• We can use the container name or ID in order to re-attach
[zohar@lnx7-oracle-1 ~]$ docker attach vibrant_borg
/ #
42
Running Ubuntu in Container
• Let’s try a more useful container: Ubuntu Linux
• Here we can actually install things on it (it’s connected to the internet!)
[zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu
root@502f2202f0bc:/#
[zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu
root@502f2202f0bc:/# apt-get update
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [52.0
kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
[...]
43
Working in the Container
• Let’s have some fun: let’s install figlet
• This only install the program on our current container, it does not change our image or our host system
root@502f2202f0bc:/# apt-get install figlet
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
figlet
[…]
root@502f2202f0bc:/# figlet hello
_ _ _
| |__ ___| | | ___
| '_ \ / _ \ | |/ _ \
| | | | __/ | | (_) |
|_| |_|\___|_|_|\___/
44
Running Containers in the Background
• We can start containers and run them in the background using the run -d flag – but what happens to their console output?
• Console output is being logged – we can attach to the full output by using the docker logs command
[zohar@lnx7-oracle-1 ~]$ docker run -d jpetazzo/clock
70e4be1a0c7ece5831632d169f21974ef9945234c6f48223a24ac92f8e73517f
[zohar@lnx7-oracle-1 ~]$ docker ps -l
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
70e4be1a0c7e jpetazzo/clock "/bin/sh -c 'while..." 12 seconds ago
Up 10 seconds tender_murdock
[zohar@lnx7-oracle-1 ~]$ docker logs 70e
Fri Nov 17 11:23:50 UTC 2017
Fri Nov 17 11:23:51 UTC 2017
Fri Nov 17 11:23:52 UTC 2017
Fri Nov 17 11:23:53 UTC 2017
45
View Tail of the Log
• To view only the tail of the log, we use docker logs --tailcommand (with or without –f)
[zohar@lnx7-oracle-1 ~]$ docker logs 70e --tail 3
Fri Nov 17 11:27:30 UTC 2017
Fri Nov 17 11:27:31 UTC 2017
Fri Nov 17 11:27:32 UTC 2017
46
Stopping the Container
• Stopping our container gracefully:
• We can also use the kill command which will not wait for the container to stop gracefully for 10 seconds
• When using interactive mode, CTRL+C will send SIGINIT and close the container
[zohar@lnx7-oracle-1 ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
502f2202f0bc ubuntu "/bin/bash" 5 minutes ago
Up 5 minutes tender_keller
[zohar@lnx7-oracle-1 ~]$ docker stop 502
502
[zohar@lnx7-oracle-1 ~]$
47
One-Off Containers
• Using the --rm flag when running the container, will cause the container to be removed completely when it is taken down
• This is useful for “one-off” containers that will do an action and then disappear (like the “hello world” we did before)
• We can also use the docker rm command to delete unused containers
48
Showing Stopped Containers
• Stopped containers still exist – they are not automatically removed
• We can list all containers (running and stopped) by using docker ps –a
[zohar@lnx7-oracle-1 ~]$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS
70e4be1a0c7e jpetazzo/clock "/bin/sh -c 'while..." 16 minutes ago Exited (137) 1 second ago
502f2202f0bc ubuntu "/bin/bash" 2 hours ago Exited (0) 16 minutes ago
262ebcab83c8 ubuntu "/bin/bash" 2 hours ago Exited (0) 2 hours ago
49
Restarting and Attaching to Our Container
• If we restart our container, the figlet command will be available (but we still can’t duplicate our container)
• If we start another container (using run), the figlet will not be available because it’s not in our base image
[zohar@lnx7-oracle-1 ~]$ docker start 502
502
[zohar@lnx7-oracle-1 ~]$ docker attach 502
root@502f2202f0bc:/# figlet hello again
_ _ _ _
| |__ ___| | | ___ __ _ __ _ __ _(_)_ __
| '_ \ / _ \ | |/ _ \ / _` |/ _` |/ _` | | '_ \
| | | | __/ | | (_) | | (_| | (_| | (_| | | | | |
|_| |_|\___|_|_|\___/ \__,_|\__, |\__,_|_|_| |_|
|___/
50
docker exec Command
• In some cases, attaching to a container will not allow us to interact with it (we’ll talk about CMD and ENTRYPOINT later)
• In those cases, we will want to execute a command in the container to interact with it:
• We can run whatever command we want, but bash is one of the more common…
[zohar@lnx7-oracle-1 ~]$ docker exec -ti plugpdb bash
51
Summary for the Basic Commands
• docker ps [-a] [-l] [-q]
• docker run [-it|-d] [--rm]
• docker attach
• docker start
• docker stop
• docker kill
• docker rm (docker rmi)
Under the Hood: Images and Layers
53
Images and Layers
• An image is a collection of files + some meta data (technically: those files form the root filesystem of a container)
• Images are made of layers, conceptually stacked on top of each other
• Each layer can add, change, and remove files
• Images can share layers to optimize diskusage, transfer times, and memory use
• Layer default location: /var/lib/docker
[Source: https://docs.docker.com/terms/layer/]
54
Differences Between Containers and Images
• An image is a read-only filesystem - we can’t change them after they are created
• A container is an encapsulated set of processes running in a read-write copy of that filesystem
• Images are like templates/stencils we create Containers from
• To optimize container boot time, copy-on-write is used instead of regular copy
• The command docker run starts a container from a given image
55
Copy-On-Write
• Copy-on-write, sometimes referred to as implicit sharing or shadowing is a resource-management technique to efficiently implement a "duplicate" or "copy" operation on modifiable resources
• If a resource is duplicated but not modified, it is not necessary to create a new resource
• Modifications must still create a copy, but the copy operation is deferred to the first write
• By sharing resources in this way, it is possible to significantly reduce the resource consumption of unmodified copies, while adding a small overhead to resource-modifying operations
56
UnionFS and BTRFS
• AUFS is a union filesystem. The aufs storage driver was previously the default storage driver used for managing images and layers on Docker
• Btrfs is a next generation copy-on-write filesystem that supports many advanced storage technologies that make it a good fit for Docker
• Btrfs is included in the mainline Linux kernel
• Docker’s btrfs storage driver leverages many btrfs features for image and container management: block-level operations, thin provisioning, copy-on-write snapshots, and ease of administration
57
Where Do We Get a Base Image?
• Build our own from “Scratch”
• Docker Hub and Docker Hub Store• Some images might require approve in the store before pulling it
• Get recipes off GitHub or BitBucket
• Remember: Images are read-only, we can’t change them after they are created but we can evolve them
58
Wait! Images Are Read-Only?!
• Question: If images are read-only, how do we change them?
• Answer: We don’t!• We create a new container from that image
• We make changes to that container
• When we are satisfied with those changes, we evolve them into a new layer
• A new image is created by stacking the new layer on top of the old image
59
The Chicken and Egg Problem
60
Creating the First Image
• There is a special empty image called scratch which means the first image is built from “scratch”
• The docker import command loads a tarball into Docker• The imported tarball becomes a standalone image
• That new image has a single layer
• We probably don’t need this – we will use more “common” base images
61
The Docker Hub Repository
• Docker Hub is a public repository of pre-built images
• We can upload (push) images, or download (pull) other people/organizations images to base our containers/layers on
• If we don’t want to use the public Hub, we can create our own local repositories (and namespaces)
62
Downloading an Image
• Explicitly: we can download an image without initiating a container using the docker pull command
• Download the exact image only
• Image kept on the Docker host ready to use
• Implicitly: when executing docker run command, it will download all the images needed that are not locally stored
• Download the entire dependency layers
• Only being done once, on the first run of that image
63
Pulling Image Example
• Using the run command, we tried to start a container from the Ubuntu image. Some layers did not exist so they were automatically pulled and downloaded:
[zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
ae79f2514705: Downloading [===================> ]
18.19MB/47.62MB
c59d01a7e4ca: Download complete
41ba73a9054d: Download complete
f1bbfd495cc1: Download complete
0c346f7223e2: Download complete
64
Listing The Local Docker Images
• List all the local images by using the docker images command[zohar@lnx7-oracle-1 ~]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
figlet latest 564ca4364896 5 days ago 162MB
mysql latest 5709795eeffa 12 days ago 408MB
ubuntu latest dd6f76d9cc90 13 days ago 122MB
busybox latest 6ad733544a63 13 days ago 1.13MB
jpetazzo/namer master f1557ed35146 2 years ago 723MB
jpetazzo/clock latest 12068b93616f 2 years ago 2.43MB
65
But How Do We Find Images?
• Finding an image in the repository is easy :• Find it online using hub.docker.com or store.docker.com
• Use the shell interface:$ docker search mysql
66
Finding an Image: Shell
• Name is the name and namespace of the image we are searching for• Root-like (i.e. mysql or mariadb)• User/organizaion (i.e. zabbix or realmgic)• Self hosted (e.g. registry.example.com:5000/my-private-image)
• Description is an optional description for your images
67
Namespaces
• Root: official images. They are put there by Docker Inc., but they are generally authored and maintained by third parties
• Small, "swiss-army-knife" images like busybox• Distro images to be used as bases for your builds, like ubuntu, fedora• Ready-to-use components and services, like redis, postgresql, mysql
• User: The user namespace holds images for Docker Hub users and organizations
• Self-Hosted: Images which are not hosted on Docker Hub, but on third party registries. They contain the hostname (or IP address), and optionally the port, of the registry server.
68
Finding an Image (cont.)
• Stars indicate how poplar an image is
• Official indicate it is part of the root namespace
• Automated images are built automatically by the Docker Hub using submitted Dockerfiles in GitHub - this means that their build recipe is always available
69
A Word of Warning!
• Images on the Hub are not being verified and might be harmful
• Use trusted images from the store to make sure legitimate source
• You can also use images with a known Dockerfile – for example, Automated Docker Images where you can verify their build process
70
Image Tags
• Images can have tags to define image versions or variants
• docker pull ubuntu will refer to ubuntu:latest
• The :latest tag is generally updated often
• We only specify tags when:• We want to ensure that the same version will be used everywhere
• Recording a procedure into a script
• Going to production
• Ensure repeatability later
71
Saving a New Image
• docker commit• Saves all the changes made to a container into a new layer
• Creates a new image (effectively a copy of the container)
• Be careful, it saves everything you did to the container!
• docker build• Performs a repeatable build sequence in a Dockerfile (recipe)
• This is the preferred method!
72
Pushing an Image to the Hub
• Once the image is ready, pushing it to the hub is simple - using the docker push command
• Make sure your image is tagged before uploading[zohar@lnx7-oracle-1 ~]$ export DOCKER_ID_USER=“realmgic“
[zohar@lnx7-oracle-1 ~]$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't
have a Docker ID, head over to https://hub.docker.com to create one.
Username (realmgic):
Password:
Login Succeeded
[zohar@lnx7-oracle-1 ~]$ docker push $DOCKER_ID_USER/my_image
73
Building a New Image Example
• Let’s use the example from earlier and build an image that has the figlet tool in it
• Our base image will be the Ubuntu Image
• The base package might not be updated, so let’s update its Linux packages and install figlet
[zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu
root@502f2202f0bc:/#
root@502f2202f0bc:/# apt-get update && apt-get install figlet
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
[...]
74
Check Container Differences
• We can check what changed between this container and the base image using the docker diff command
• Since Docker uses copy-on-write, these are the only files kept for this container on disk. Multiple containers share the base image files (due to UnionFS/btrfs for example)
[zohar@lnx7-oracle-1 ~]$ docker diff 502
C /root
A /root/.bash_history
C /etc
C /etc/alternatives
A /etc/alternatives/figlet
A /etc/alternatives/figlet.6.gz
[...]
75
Commit The Changes
• Create a new image from the newly built container using the docker commit command
• The new image will look like this since we didn’t tag it with a name:
• Tag the new image:
[zohar@lnx7-oracle-1 ~]$ docker commit 502
sha256:8cc5680434bb94b272e59624cd44667eaab191a55a08c38a90c472f9f9b45b3f
[zohar@lnx7-oracle-1 ~]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
<none> <none> 8cc5680434bb About a minute ago 162MB
[zohar@lnx7-oracle-1 ~]$ docker tag 8cc5680434bb figlet
[zohar@lnx7-oracle-1 ~]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
figlet latest 8cc5680434bb 5 minutes ago 162MB
76
Manual Bad, Automatic Good
77
Automated Docker Images: Dockerfile
• Dockerfile is a build recipe to customize the base (r/o) image and make it into a new image
Base Docker Image
Instruction
• Run commands• Add files/directories• Define ENV variables• Define processes to run• Set Entrypoint
New Docker Image
78
Automated Docker Images on Hub
• Docker Hub can automatically build images based on Dockerfilesplaced on Github
• These Images can be built automatically by the Hub or we can download the recipe and build it ourselves
• Advantages:• Images built this way are built exactly as specified
• The Dockerfile is available to anyone with access to your Docker Hub repository
• Your repository is kept up-to-date with code changes automatically
79
Creating a Dockerfile
• Our new Dockerfile must be in an empty directory
• In the Dockerfile, we specify the base Image we start from using the FROM command
• The RUN commands are never interactive, so we’ll need to sort it out when we build the script
[zohar@lnx7-oracle-1 ~]$ mkdir figlet
[zohar@lnx7-oracle-1 ~]$ cd figlet/
[zohar@lnx7-oracle-1 figlet]$ vi Dockerfile
FROM ubuntu
RUN apt-get update -y
RUN apt-get install –y figlet
80
Build The Image
• Create a new image from the Dockerfile we createdusing the docker build command
[zohar@lnx7-oracle-1 figlet]$ docker build -t figlet_auto .
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM ubuntu
---> dd6f76d9cc90
Step 2/3 : RUN apt-get update -y
---> Running in 174cec778baa
[...]
---> 035ad14c7312
Removing intermediate container 174cec778baa
Step 3/3 : RUN apt-get install figlet -y
---> Running in e9d2c2870c5d
[...]
---> e6e0b459c15c
Removing intermediate container e9d2c2870c5d
Successfully built e6e0b459c15c
Successfully tagged figlet_auto:latest
81
Reading the Output: Step 1
• Our base image is Ubuntu – so we start a new container for it (dd6f76d9cc90) and make it our base image
• The -t indicates the tag for the name of our new image
[zohar@lnx7-oracle-1 figlet]$ docker build -t figlet_auto .
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM ubuntu
---> dd6f76d9cc90
82
Reading the Output: Step 2
• We start a new container from our base image (174cec778baa) and modify it using our RUN command
• At the end of this step, we’ll have a new base image: 035ad14c7312
• The intermediate base image will be removed
Step 2/3 : RUN apt-get update -y
---> Running in 174cec778baa
[...]
---> 035ad14c7312
Removing intermediate container 174cec778baa
83
Reading the Output: Step 3
• We repeat the process for step 3
• The final container becomes our base image (e6e0b459c15c) and it is tagged to the name we chose (figlet_auto:latest)
Step 3/3 : RUN apt-get install figlet -y
---> Running in e9d2c2870c5d
[...]
---> e6e0b459c15c
Removing intermediate container e9d2c2870c5d
Successfully built e6e0b459c15c
Successfully tagged figlet_auto:latest
84
Image History
• The history command lists all the layers composing an image
• For each layer, it shows its creation time, size, and creation command
• When using Dockerfile, each line indicates a single command
[zohar@lnx7-oracle-1 figlet]$ docker history figlet_auto
IMAGE CREATED CREATED BY SIZE
COMMENT
4cbdf48e2076 About a minute ago /bin/sh -c apt-get install figlet -y 1.02MB
c12b1ebe20b8 2 minutes ago /bin/sh -c apt-get update -y 39.2MB
dd6f76d9cc90 13 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 13 days ago /bin/sh -c mkdir -p /run/systemd && echo '... 7B
<missing> 13 days ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\... 2.76kB
<missing> 13 days ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B
<missing> 13 days ago /bin/sh -c set -xe && echo '#!/bin/sh' >... 745B
<missing> 13 days ago /bin/sh -c #(nop) ADD file:5b334adf9d9a225... 122MB
85
Image History (cont.)
• When NOT using Dockerfile, some lines will not explain what really happened:
[zohar@lnx7-oracle-1 figlet]$ docker history figlet
IMAGE CREATED CREATED BY SIZE
COMMENT
8cc5680434bb 43 minutes ago /bin/bash 40.3MB
dd6f76d9cc90 13 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 13 days ago /bin/sh -c mkdir -p /run/systemd && echo '... 7B
<missing> 13 days ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\... 2.76kB
<missing> 13 days ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B
<missing> 13 days ago /bin/sh -c set -xe && echo '#!/bin/sh' >... 745B
<missing> 13 days ago /bin/sh -c #(nop) ADD file:5b334adf9d9a225... 122MB
86
The Caching System
• Building the same image again will be instantaneous
• After each step docker takes a snapshot of the resulting image
• Before executing the step, Docker checks if it has already built that sequence (based on the text in the Dockerfile)
• Example: • Changing the apt-get install figlet and add a space, it will run again
• The apt-get update will not run again even though the apt mirrors updated (same command)
• We can force a rebuild with docker build --no-cache
87
CMD or ENTRYPOINT?
• Both ENTRYPOINT and CMD give you a way to identify which command should be run when a container is started from your image
• For example, most Linux container will use /bin/bash or /bin/shas their default CMD to allow shell access
• Running a container without a CMD or ENTRYPOINT will result with an error (either built in or via docker run command)
• Each image can have only one CMD and one ENTRYPOINT. Specifying more than one in the Dockerfile will override the previous command
88
Using CMD
• Example: let’s say we want to change or figlet image to display “hello” message.
• We need to modify the Dockerfile and build it:
• When running a new image:
FROM ubuntu
RUN apt-get update -y
RUN apt-get install figlet -y
CMD figlet hello
[zohar@lnx7-oracle
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto
_ _ _
| |__ ___| | | ___
| '_ \ / _ \ | |/ _ \
| | | | __/ | | (_) |
|_| |_|\___|_|_|\___/
89
Override CMD
• If we want to override the CMD, we just send in a new command in the run command
• The bash canceled our figlet command – but what happens if we don’t want it to override? What if we want to send in a parameter?
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto bash
root@388ac4ed1d28:/#
90
Using ENTRYPOINT
• We want to be able to specify a different message on the command line (parameter)
• When running a new image:
FROM ubuntu
RUN apt-get update -y
RUN apt-get install figlet -y
ENTRYPOINT [“figlet”, “-f”, “banner”]
[zohar@lnx7-oracle
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto banner
##### ## # # # # ###### #####
# # # # ## # ## # # # #
##### # # # # # # # # ##### # #
# # ###### # # # # # # # #####
# # # # # ## # ## # # #
##### # # # # # # ###### # #
91
Override ENTRYPOINT
• If we want to override the ENTRYPOINT, we can’t just send in a new command in the run command because it will be printed…
• We need to override it with the --entrypoint flag:
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto bash
##### ## #### # #
# # # # # # #
##### # # #### ######
# # ###### # # #
# # # # # # # #
##### # # #### # #
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti --entrypoint bash figlet_auto
root@5dd5da9a0825:/#
92
Using CMD and ENTRYPOINT Together
• In our example, figlet needs and extra parameter to work (the message). How can we specify a default value for it?
• We can combine using CMD and ENTRYPOINT• ENTRYPOINT will specify the command to run
• CMD will specify the default parameter
FROM ubuntu
RUN apt-get update -y
RUN apt-get install figlet -y
ENTRYPOINT ["figlet", "-f", "banner"]
CMD ["CMD"]
93
Using ENTRYPOINT and CMD Example
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto
##### # # ######
# # ## ## # #
# # # # # # #
# # # # # #
# # # # #
# # # # # #
##### # # ######
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto test
##### ###### #### #####
# # # #
# ##### #### #
# # # #
# # # # #
# ###### #### #
94
Copying Files From Host to Image
• We can copy files from our host to the image when we build it
• We can see the file on our image’s container
• Changing the file will invalidate the cached layer – so rebuilding the image with a new file will generate a new image
FROM ubuntu
RUN apt-get update -y
RUN apt-get install figlet -y
COPY zohar.txt /
ENTRYPOINT ["figlet", "-f", "banner"]
CMD ["CMD"]
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti --entrypoint bash figlet_auto
root@b65dd0df7597:/# cat /zohar.txt
Testing a copy file...
95
The ADD Command
• Since the COPY command does not support copying file from a URL, we can use the ADD command for that
• ADD also allows us to copy tar files and automatically open them
• It is best practice to use COPY unless you must use ADD
96
The WORKDIR Instruction
• The WORKDIR instruction sets the working directory for subsequent instructions
• It also affects CMD and ENTRYPOINT, since it sets the working directory used when starting the container
• You can specify WORKDIR again to change the working directory for further operations
WORKDIR /src
97
The ENV Instruction
• The ENV instruction specifies environment variables that should be set in any container launched from the image
• This is the same as exporting in the OS level
• ENV variables can be overridden by the -e flag in the docker runcommand
ENV ORACLE_SID orclcdb
docker run -p 1521:1521 -e ORACLE_SID=orcl store/oracle/database-enterprise:12.2.0.1
98
The USER Instruction
• The USER instruction sets the user name or UID to use when running the image
• It can be used multiple times to change back to root or to another user
Docker Networks
100
Docker Network Basics
• Containers cannot have public IPv4 addresses
• They do have private addresses
• Services have to be exposed port by port
• Ports have to be mapped to different ports on the host to avoid conflicts
• Containers can see the network by default – it uses our default bridge network
• Docker --link only worked on a single host – does not work in swarm/cluster and considered deprecated
101
Example: NGINX Server
• Starting a NGINX (web) server
• The -P flag (publish all) indicates every exposed port to be mapped automatically to a port in the host server.
• In this case, port 80 will be mapped to port 9000
• We can also ask the container for his mapped ports:
[zohar@lnx7-oracle-1 ~]$ docker run -d -P nginx
[zohar@lnx7-oracle-1 ~]$ docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS
PORTS NAMES
47733cd2d419 nginx "nginx -g 'daemon ..." About a minute ago Up About a minute
0.0.0.0:9000->80/tcp gallant_poincare
[zohar@lnx7-oracle-1 ~]$ docker port 477
80/tcp -> 0.0.0.0:9000
102
Example: NGINX Server (cont.)
• When we try to reach the container port, we use the localhost port for it:
[zohar@lnx7-oracle-1 ~]$ curl -l localhost:9000
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
103
Manual Binding of Ports
• Setting the ports manually can be done during the run command
• The convention is host-port:container-port
• Trying to start another container with the same port will result with an error
[zohar@lnx7-oracle-1 ~]$ docker run --rm -d -p 80:80 nginx
[zohar@lnx7-oracle-1 ~]$ docker run --rm -d -p 80:80 nginx
6abc743b8dbaa46b0af1d6f4d72f6313bd41baa9c74524649ac7fe489e387709
docker: Error response from daemon: driver failed programming external connectivity
on endpoint friendly_bohr
(dde052b96b19300988e0c4ea389fe5d0a91eab3e91bafd0dfd9d1d454869dc27): Bind for
0.0.0.0:80 failed: port is already allocated.
104
Exposing Ports in the Image using Dockerfile
• The EXPOSE instruction tells Docker what ports are to be published in this image
• The Dockerfile doesn’t actually map the port to a port on the host –we still need to make it public in our run command
EXPOSE 1521
105
Finding Container Private IP
• We can use docker inspect command to find the private IP of a container
• The docker inspect is an advanced command, that can retrieve tons of information about our containers
• We can ping our container
[zohar@lnx7-oracle-1 ~]$ docker inspect --format '{{ .NetworkSettings.IPAddress }}'
keen_neumann
172.17.0.3
[zohar@lnx7-oracle-1 ~]$ ping 172.17.0.3
PING 172.17.0.3 (172.17.0.3) 56(84) bytes of data.
64 bytes from 172.17.0.3: icmp_seq=1 ttl=64 time=0.109 ms
64 bytes from 172.17.0.3: icmp_seq=2 ttl=64 time=0.148 ms
106
Docker Networks
• Conceptually, a Docker network is a virtual switch
• It can be local (to a single Engine) or global (across multiple hosts)
• Each network has a dedicated subnet
• Containers with explicit names are discoverable via DNS
• A network is managed by drivers – some of them are built in and some can be added by plugins
[zohar@lnx7-oracle-1 ~]$ docker network ls
NETWORK ID NAME DRIVER SCOPE
03d9803daff0 bridge bridge local
3eddac377534 host host local
e733f2a48ed2 none null local
107
The Default Bridge Driver
• By default, the container gets a virtual eth0 interface
• It is connected to the Docker bridge network
• Addresses are allocated on a private, internal subnet
• Outbound traffic goes through an iptables MASQUERADE rule
• Inbound traffic goes through an iptables DNAT rule
• The container can have its own routes, iptables rules, etc.
108
The Host Driver
• The host network is internal network between the host and the container
• A container with a host driver sees (and can access) the network interfaces of the host
• Network traffic doesn't have to go through NAT, bridge, or veth
• Performance = native!
109
The null Driver
• It only gets the lo loopback interface. No eth0
• It can't send or receive network traffic
• Useful for isolated/untrusted workloads
110
Creating a Network
• Create a network
• Use network for a container
[zohar@lnx7-oracle-1 ~]$ docker network create OracleWeek
72635c667b4c3e54e350925339cef73cf480528cef6368d6f59954d2358c210d
[zohar@lnx7-oracle-1 ~]$ docker network ls
NETWORK ID NAME DRIVER SCOPE
72635c667b4c OracleWeek bridge local
03d9803daff0 bridge bridge local
3eddac377534 host host local
e733f2a48ed2 none null local
[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont1 --net OracleWeek busybox
/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
144: eth0@if145: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff
inet 172.18.0.2/16 scope global eth0
valid_lft forever preferred_lft forever
111
Containers DNS
• All containers with specific name in the same network share the same dynamic DNS (placed at 127.0.0.11):
[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont2 --net OracleWeek busybox
/ # ping cont1
PING cont1 (172.18.0.2): 56 data bytes
64 bytes from 172.18.0.2: seq=0 ttl=64 time=0.176 ms
64 bytes from 172.18.0.2: seq=1 ttl=64 time=0.332 ms
/ # nslookup cont1
Server: 127.0.0.11
Address 1: 127.0.0.11
Name: cont1
Address 1: 172.18.0.2 cont1.OracleWeek
/ # nslookup cont2
Server: 127.0.0.11
Address 1: 127.0.0.11
Name: cont2
Address 1: 172.18.0.3 05efb3eb1863
112
Connecting Multiple Networks
• Networks are independent from each other but a container can be connected to multiple networks
• We can attach a network to a running container
• Example:• Create a new network and a new container in that network
[zohar@lnx7-oracle-1 ~]$ docker network create ilOUG
41244791088e6970dca741b54fa54a8e0f0a08eeb1b184a78220a9cf03d2528a
[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont3 --net ilOUG busybox
/ # ping cont1
ping: bad address 'cont1‘
113
Connecting Multiple Networks (cont.)[zohar@lnx7-oracle-1 docker network connect ilOUG cont1
[zohar@lnx7-oracle-1 ~]$ docker attach cont1
/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
144: eth0@if145: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff
inet 172.18.0.2/16 scope global eth0
valid_lft forever preferred_lft forever
153: eth1@if154: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
link/ether 02:42:ac:13:00:03 brd ff:ff:ff:ff:ff:ff
inet 172.19.0.3/16 scope global eth1
valid_lft forever preferred_lft forever
/ # ping cont2
PING cont2 (172.18.0.3): 56 data bytes
64 bytes from 172.18.0.3: seq=0 ttl=64 time=0.203 ms
/ # ping cont3
PING cont3 (172.19.0.2): 56 data bytes
64 bytes from 172.19.0.2: seq=0 ttl=64 time=0.108 ms
114
Multi-host Networking: Overlay Driver
• When using cluster, overlay network will interact between nodes
115
Overlay Driver
• The overlay network driver can support a clustered (swarm mode) environment
• When you create a service that uses the overlay network, the manager node automatically extends the overlay network to nodes that run service tasks
• Only swarm services can connect to overlay networks, not standalone containers
• The overlay network without swarm requires a valid key-value store service (Consul, Etcd, and ZooKeeper (Distributed store)) and might be deprecated in the future
Docker Storage
Volumes, Bind Mounts and tmpfs
117
Docker “External” Storage
• Our containers are independent – they can’t see the host filesystem , but sometimes we want to share files/directories with the host or between containers
• The solution: map a volume to the container so it will be available to both the host and all the using containers
118
Named Volume
• Created and managed by Docker
• We can create a volume explicitly using the docker volumecreate command, or Docker can create a volume during container or service creation
• When creating a volume, it is stored within a directory on the Docker host (empty)
• When you mount the volume into a container, this directory is what is mounted into the container
• A given volume can be mounted into multiple containers simultaneously
119
Named Volumes Example
• We can create volumes, list them and inspect for actual location:[zohar@lnx7-oracle-1 ~]$ docker volume create test
Test
[zohar@lnx7-oracle-1 ~]$ docker volume ls
DRIVER VOLUME NAME
local fc7d0d5897f2090f51a3ee7589ba360841344dc34c605626337debff4dfa9860
local test
[zohar@lnx7-oracle-1 ~]$ docker inspect test
[
{
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/test/_data",
"Name": "test",
"Options": {},
"Scope": "local"
}
]
120
Volumes Functionality
• Volumes act as passthroughs to the host filesystem
• Volumes are easier to back up or migrate than bind mounts
• Volumes can be more safely shared among multiple containers
• When you docker commit, the content of volumes is not brought into the resulting image
• Volume drivers allow you to store volumes on remote hosts or cloud providers, to encrypt the contents of volumes, or to add other functionality
121
Bind Mounts
• Bind mounts have limited functionality compared to volumes
• When we use a bind mount, a file or directory on the host machine is mounted into a container
• The file or directory is referenced by its full path on the host machine
• The file or directory does not need to exist on the Docker host already, if it does not exist – it will be automatically created (when using –v)
• We use bind mounts when we want to mount a non-empty directory from the host to the container
122
Binding the Mount Volumes
• Volumes can be mounted in two ways:• on the docker run command
• Within a Dockerfile, with a VOLUME instruction
docker run -it -v $(pwd):/zohar --rm --name cont1 --net ilOUG busybox
VOLUME /zohar
123
Mounting a Volume Manually
• We can map volumes on the docker run command
• The –v flag mounts a directory from your host into your Docker container. The flag structure is:
• If [host-path] or [container-path] doesn't exist it is created
• You can control the write status of the volume with the ro and rw options
• If you don't specify rw or ro, it will be rw by default
docker run -it -v $(pwd):/zohar --rm --name cont1 --net ilOUG busybox
[host-path]:[container-path]:[rw|ro|z|Z]
124
Volumes and SElinux
• The permission on the volume will be the same as on the host
• That might be a problem if the files are owned by a user the container doesn’t know
• We can “help” Docker and SElinux play nice together by specifying the “z” (r/o) or “Z” (r/w) flag:
[zohar@lnx7-oracle-1 ~]$ docker run -it -v $(pwd):/zohar --rm --name cont1 --net
ilOUG busybox
/ # ls -l /zohar/
ls: can't open '/zohar/': Permission denied
/ # [zohar@lnx7-oracle-1 docker run -it -v $(pwd):/zohar:Z --rm --name cont1 --net
ilOUG busybox
/ # ls /zohar/
Desktop Documents Downloads Music Pictures Public Templates Videos
figlet myimage namer oradata
125
tmpfs Mounts
• A tmpfs mount is not persisted on disk, either on the Docker host or within a container
• It can be used by a container during the lifetime of the container, to store non-persistent state or sensitive information
[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --tmpfs /zohar/tmpfs busybox
/ # df -h /zohar/tmpfs/
Filesystem Size Used Available Use% Mounted on
tmpfs 1.8G 0 1.8G 0% /zohar/tmpfs
126
Orphaned Volumes
• What happens when you remove containers with volumes?
• They stay on disk – even if no one will ever use them again
• Use the –v flag to remove the container with the volumes attached
• We can prune the unused volumes using the docker volumeprune command
• Ultimately, you are the one responsible for logging, monitoring, and backup of your volumes!
Oracle RDBMS, MySQLand Docker
128
Oracle RDBMS Official Image
• Oracle provides their own Oracle RDBMS official image with database (12.2.0.1 EE) included (free):
• https://store.docker.com/images/oracle-database-enterprise-edition
• In order to pull this image, we need a Docker repository account
• We also need to accept the conditions on the store itself
• Official Image from Oracle Store:• https://container-registry.oracle.com/
docker login
docker pull store/oracle/database-enterprise:12.2.0.1
129
Oracle Official Image Limitations
• Supports a single instance database,
• No multiple homes
• No Grid Infrastructure/ASM
• No multiple instances on the same container
• No AMM
• Dataguard is not supported
• Database options and patching are not supported
130
Oracle Official Slim Image
• Since the official image is quite big (~4GB) there is also a slim version:
• This image does not support the following features - Analytics, Oracle R, Oracle Label Security, Oracle Text, Oracle Application Express and Oracle DataVault.
• The slim version is around 1.4GB
• Oracle support does not require you to use the provided Docker files – we can build our own images!
docker pull store/oracle/database-enterprise:12.2.0.1-slim
131
Building Our Own Images
• Oracle official Docker team github account with different Dockerfilesand samples: https://github.com/oracle/docker-images
• On the repository:• Oracle 11.2.0.2 (XE) Dockerfile
• Oracle 12.1.0.2 Dockerfile and scripts (SE2, EE)
• Oracle 12.2.0.1 Dockerfile and scripts (SE2, EE)
• Lots of other Oracle stuff: Oracle Java, Weblogic, GoldenGate and NoSQL…
• The build recipes build our own images, so we can customize it
• Recipes require ZIP files of the software for deployment on the image
132
How It’s Going to Work
• The official image based on Oracle Linux 7
• The image contains an instance running inside the container
• The ORACLE_HOME and datafiles are kept on a volume (host machine)
• The image also runs the listener on port 1521 and exposes this port
• The instance runs inside the container is a root container (CDB) in our multitenant database. It also has a single plugged database
• Persistence: • There is a volume plug we can use to map our local pluggable database files
to the database and then we can plug our database to the instance
133
Creating the Container
• Creating the container (might take some time):[zohar@lnx7-oracle-1 ~]$ docker volume create OracleDBData
[zohar@lnx7-oracle-1 ~]$ docker run --name plugpdb -p 1521:1521 -p 5500:5500 -e
ORACLE_SID=MYCDB -v OracleDBData:/ORCL /home/zohar/oradata:/opt/oracle/oradata
store/oracle/database-enterprise:12.2.0.1
Setup Oracle Database
Oracle Database 12.2.0.1 Setup
Sat Nov 18 16:09:47 UTC 2017
Check parameters ......
log file is : /home/oracle/setup/log/paramChk.log
paramChk.sh is done at 0 sec
untar DB bits ......
log file is : /home/oracle/setup/log/untarDB.log
untarDB.sh is done at 166 sec
config DB ......
log file is : /home/oracle/setup/log/configDB.log
[…]
134
Inside the Container
• The entire database home and database file are stored in the same location: /ORCL, divided into u01-u04 for different usages
• We can mount these areas into our own disks or volume
• In our example, the entire /ORCL is mounted into one volume: OracleDBData
• We can mount other mount points or volumes to keep our pluggable databases
135
Connecting to the Database
• Connecting to the database, via the container:
• The initial password for SYS is “Oradoc_db1”
• We can also connect using bash command and work like any other “server”
docker exec -it plugdb bash -c "source /home/oracle/.bashrc; sqlplus /nolog"
136
Building Our Own Image
• Download the appropriate directory from and unzip it on the server• https://github.com/oracle/docker-
images/tree/master/OracleDatabase/dockerfiles
• Download the Oracle Software from the Oracle site and place them in the same directory
• Modify whatever you need – scripts, Dockerfile, keep some of the options manually removed, etc.
• Build the image
docker build --force-rm=true --no-cache=true -t
oracle/database:12.2.0.1-ee -f Dockerfile.ee .
137
Start the Container
• Running the container is just like before – we can pass some envvariables but most of them already has default values
• This database doesn’t have to use containers (like the official image): ORACLE_CONTAINERDB=false
docker run --name orcl1 \
-p 1522:1521 -p 5502:5500 \
-e ORACLE_SID=orclcdb \
-e ORACLE_PWD=manager1 \
-e ORACLE_CHARACTERSET=WE8MSWIN1252 \
-e ORACLE_CONTAINERDB=false \
oracle/database:12.2.0.1-ee
138
MySQL
• MySQL database is ideal for Docker Containers – its containers exist since forever
• There are actually two MySQL main images:• mysql (also available on the store)
• mysql/mysql-server (automated, connected to mysql GitHub)
• Both images are tagged with multiple version: 5.5, 5.6, 5.7 (latest) and 8.0 (preview). They also run sub version (e.g. 5.6.38, 5.7.20, etc.)
139
Starting the MySQL Container
• Starting the container
• Connecting to the database using root:
docker run --name my-container-name -e MYSQL_ROOT_PASSWORD=my-secret-pw -d
mysql/mysql-server:tag
docker exec -it my-container-name mysql -uroot -p
140
Where to Store Data
• There are several ways to store data used by applications that run in Docker containers:
• Let Docker manage the storage of your database data by writing the database files to disk on the host system using its own internal volume management. This is the default and is easy and fairly transparent to the user
• Create a data directory on the host system (outside the container) and mount this to a directory visible from inside the container. This places the database files in a known location on the host system, and makes it easy for tools and applications on the host system to access the files
141
Connecting to Existing Database
• If you start your mysql container instance with a data directory that already contains a database (specifically, a mysql subdirectory), it will use that as its database
• Note: the $MYSQL_ROOT_PASSWORD variable should be omitted from the run command line; it will in any case be ignored, and the pre-existing database will not be changed in any way
• If there is no database initialized when the container starts, then a default database will be created.
142
Creating Database Dumps
• Most of the normal tools will work, although their usage might be a little convoluted in some cases to ensure they have access to the mysqld server. A simple way to ensure this is to use docker exec and run the tool from the same container, similar to the following
$ docker exec some-mysql sh -c 'exec mysqldump --all-databases -uroot \
-p"$MYSQL_ROOT_PASSWORD"' > /some/path/on/your/host/all-databases.sql
143
Overriding Options and MySQL.cnf
• We can override options by using the --option flag
• To provide a different mysql.cnf, use the –v flag:
• Remember SELinux? We need to allow the file for the container:
docker run --name my-container-name -d mysql/mysql-server --option1=value --
option2=value
docker run --name my-container-name -d mysql/mysql-server --character-set-
server=utf8 --collation-server=utf8_general_ci
docker exec -it my-container-name cat /etc/my.cnf > /my/custom/config-file
docker run --name my-new-container-name -v /my/custom/config-file:/etc/my.cnf -e
MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql/mysql-server:tag
chcon -Rt svirt_sandbox_file_t /my/custom/config-file
Docker Compose
145
Docker Compose Tool
• Dockerfiles are great to build a single container
• But when you want to start a complex stack made of multiple containers, you need a different tool. This tool is Docker Compose
• Docker Compose is optional (needs to be installed separately)
• The general idea of Compose is to enable a very simple, powerful onboarding workflow:
• Clone your code• Run docker-compose up
• Your app is up and running!
146
Compose overview
• You describe a set (or stack) of containers in a YAML file called docker-compose.yml
• You run docker-compose up
• Compose automatically pulls images, builds containers, and starts them
• Compose can set up networks, volumes, and other Docker options
• Compose can run the containers in the background, or in the foreground
• When containers are running in the foreground, their aggregated output is shown
147
Installing Compose
• We can download and install Docker Compose right off its github project: https://github.com/docker/compose/releases
• Make sure to use the latest version (current version: 1.17.1):curl -L https://github.com/docker/compose/releases/download/1.17.1/docker-compose-
`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
148
docker-compose.yml
• Example for a Docker compose yaml file:
• This file will start two containers: one based on the local Dockerfile(www) and one based on the redis image. It will also mount a volume and map a port.
version: '2'
services:
web:
build: .
ports:
- "5000:5000"
volumes:
- .:/code
redis:
image: redis
149
docker-compose.yml for Oracle DBMS
• Another example for a Docker compose yaml file:
• In this example, we will start a database server using the 12.2.0.1-eeimage, map some ports. Our Oracle SID is also predefined.
version: '2'
services:
database:
image: oracle/database:12.2.0.1-ee
volumes:
- /home/oracle/oradata:/opt/oracle/oradata # persistent oracle database data.
ports:
- 1521:1521
- 8080:8080
- 5500:5500
environment:
-ORACLE_SID=DEVOPSUATCDB Example by Gerald Venzl
150
Docker Compose Usages
• Starting a Docker Compose environment:
• A network and two containers were created
• Check what is up (in our work directory only):
[zohar@lnx7-oracle-1 wordpress]$ docker-compose up -d
Creating network "wordpress_default" with the default driver
Creating wordpress_db_1 ...
Creating wordpress_db_1 ... done
Creating wordpress_wordpress_1 ...
Creating wordpress_wordpress_1 ... done
[zohar@lnx7-oracle-1 wordpress]$ docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------------
wordpress_db_1 docker-entrypoint.sh mysqld Up 3306/tcp
wordpress_wordpress_1 docker-entrypoint.sh apach ... Up 0.0.0.0:8000->80/tcp
151
Docker and Docker Compose
• We can still use our original docker commands[zohar@lnx7-oracle-1 wordpress]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
ce8847b28d67 wordpress:latest "docker-entrypoint..." 2 minutes ago
Up 2 minutes 0.0.0.0:8000->80/tcp wordpress_wordpress_1
daff10f9d8b9 mysql:5.7 "docker-entrypoint..." 2 minutes ago
Up 2 minutes 3306/tcp wordpress_db_1
[zohar@lnx7-oracle-1 wordpress]$ docker network ls
NETWORK ID NAME DRIVER SCOPE
[...]
4f395cc553ad wordpress_default bridge local
152
Stopping the Service
• We can stop the entire service by using docker-compose stop
• We can remove the service by using the docker-compose down command. This will delete all containers as well (but not the volumes)
[zohar@lnx7-oracle-1 wordpress]$ docker-compose stop
Stopping wordpress_wordpress_1 ... done
Stopping wordpress_db_1 ... done
[zohar@lnx7-oracle-1 wordpress]$ docker-compose down
Stopping wordpress_wordpress_1 ... done
Stopping wordpress_db_1 ... done
Removing wordpress_wordpress_1 ... done
Removing wordpress_db_1 ... done
Removing network wordpress_default
Misc.
154
Docker Containers After Restart
• When the host shut down, all the containers will automatically stop
• When the host comes back up, all containers will be DOWN
• In order to automatically start the container after boot, we can use the --restart option on the run command
$ docker run -dit --restart unless-stopped redis
Flag Description
no Do not automatically restart the container. (the default)
on-failure Restart the container if it exits due to an error, which manifests as a non-zero exit code.
unless-stoppedRestart the container unless it is explicitly stopped or Docker itself is stopped or restarted.
always Always restart the container if it stops.
155
Limiting Container Performance
• We can limit the performance on a container:• CPU: number of cpu (which cpus to use), number of cycles, and different
quotas
• Memory: amount of memory, amount of virtual memory
docker run --rm --cpuset-cpus=0 busybox echo hello world
docker run --rm --cpuset-cpus=0 --cpu-period=50000 busybox echo hello world
156
HEALTHCHECK
• HEALTHCHECK instruction in DockerFile
• Tells Docker how to test a container to check that it is still working properly (not just “up”) – i.e. the database/listener are up
• New status added to container lists
• Adds “(healthy)” to Status column in a “docker ps response”
[zohar@lnx7-oracle-1 ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS
d2926bc4b490 portainer/portainer "/portainer" 39 minutes ago Up 39 minutes
6a204203d9e7 store/oracle/database-ente... "/bin/sh -c '/bin/..." 3 hours ago Up 3 hours (healthy)
157
Docker Secrets
• Stored encrypted in swarm mode only
• Exposed only to nodes that run services that need them
• Presented in container via RAM only tmpfs files• never persisted to disk in encrypted format
• when container stops, secret is no longer present
• All communications between swam nodes via TLS, so secret never in the clear on the wire either
• Different secret values per environment using tags
158
Docker Security
• Use trusted images / with a known Dockerfile
• Think (twice) about pulling images from public repos / Docker hub
• Docker images can be signed
• Docker can use TLS (client to daemon)
• Do NOT disable SELinux or set enforce off (avoid setenforce 0)
• Do not keep secrets in images
• Remember: The host can always access container
GUI, Orchestration and Clusters
Swarm, GUI Interfaces
160
Docker GUI
• Kitematic – the default (now obsolete) interface from Docker themselves
• Portainer - very simple, very robust GUI interface for Docker
• Shipyard – Built on Docker Swarm, Shipyard gives you the ability to manage Docker resources including containers, images, private registries and more
• Rancher – high end solution that interfaces nicely with cloud solutions
• There are others – but I suggest going with Portainer (for beginners) or Rancher or Shipyard (more advanced)
161
Portainer
• Portainer is a simple but robust GUI interface for Docker
• It runs from within a Docker container
• It’s easier to see what is going on:• Containers• Volumes• Networks• Logs• Etc.
docker run -d --privileged -p 9000:9000 -v
/var/run/docker.sock:/var/run/docker.sock -v /opt/portainer:/data
portainer/portainer
162
Portainer Deshboard
163
Portainer Containers
164
Using Console From Web
165
Monitoring Performance
166
Kubernetes: Container Orchestration
• Kubernetes is an open source container orchestration tool developed by Google to allow a wider usage of Docker with more control and better scaling and availability
• It is one of the most commonly used container orchestration tool and has a very large community
• Kubernetes allows you to completely control your container deployment across the cluster with custom storage and network solutions and well defined deployment rules
• Kubernetes allows for easy integration with public (AWS, GCE, Azure) and private (VMWare) cloud solutions
167
Kubernetes Concepts
• While it provides a useful, scalable, and powerful tool, Kubernetes also gets a lot of clout because it’s an intricate and sometimes overly-complicated system
• Installing Kubernetes is a slightly more complex endeavor since you will need additional tools to provide network (i.e. flannel) and store the cluster data (i.e. etcd).
• Basic Concepts:• Pod: The basic deployment unit, One or more containers (usually one) that share a
common function.• ReplicaSet: Controls the replication and availability of a pod.• Deployment: Controls the pod’s deployment by creating and managing ReplicaSets.
Allows for rolling updates.• Service: Advertises a set of pods to other pods or to the outside world. Allows easy
discovery with the DNS add-on.• StatefulSet: Manages the deployment and scaling of a set of Pods, and provides
guarantees about the ordering and uniqueness of these Pods.
168
Kubernetes
Source: wikipedia
169
Docker Swarm
• Native Docker cluster same API as a single engine
• Fast provisioning, about 500 msec
• Scheduling Algo: spread, binpack, rand
• Features are optional, you can continue useKubernetes etc.
• There is NO insecure mode
170
Docker Swarm Overview
171
Docker Swarm
• Since Docker 1.12
• Swarm is merged with Docker engine:• Load balancer included
• Service discovery
• Cluster scheduler
• Swarm has many features like Google's Kubernetes• Easier to get started
• Better performance
172
Swarm Services
• To deploy an application image when Docker Engine is in swarm mode, you create a service
• Frequently a service will be the image for a microservice within the context of some larger application
• Examples of services might include an HTTP server, a database, or any other type of executable program that you wish to run in a distributed environment
173
Setup Swarm and Machine
• Create Swarm ID
• Create Machine with Swarm master
• Create Machine with Swarm agent01 / 02
• Set Docker env for Swarm master
• Deploy service to the swarm
• If needed, scale the service to run more container on more nodes
174
Learn More
More information:
• https://docs.docker.com/• Reference, information, how-to
• https://training.docker.com/• Training, self paced and trainer
• Blog: https://blog.docker.com/
• Videos: https://www.youtube.com/user/dockerrun
175
Oracle Support
• Support for Docker Running on Oracle Linux (Doc ID 1921163.1)• https://support.oracle.com/epmos/faces/DocumentDisplay?id=1921163.1
• Oracle Support Document 2216342.1 (Oracle Support for Database Running on Docker):
• https://support.oracle.com/epmos/faces/DocumentDisplay?id=2216342.1
• Gerald Venzl (will be here in ilOUG Tech Days 2018 (January 2018), speaking about Docker!)
• https://geraldonit.com/2017/08/21/creating-an-oracle-database-docker-image/
Did You Hear AboutOracle 18c?
177
• Oracle is changing the way they number products• Products will be release more often• Version names will be based on last two digits of the year and a
subversion• There is no more Release 1-Release 2 for major versions, we can’t tell
when we jumped to the next major version and when for the next minor• Examples:
• 12.2.0.1 will be followed by 18.1 (and not 12.2.0.2)• SQL Developer 4.2.1 was followed by 17.3
• Read more at the updated MOS Note 742060.1 – Release Schedule of Current Database Releases
Wait! 18c?! What happened to 13c?
178
Summary
• We talked about Docker and Docker containers use cases
• We learned how to use Docker (slides will be available soon)
• We saw how Docker works with Oracle and MySQL
• We overviewed swarm mode (not for databases)
• We talked about Docker in the cloud environment
Q&AAny Questions? Now will be the time!
