docker meets python a look on the docker sdk for python · - docker sdk might be a better choice...

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 1

Docker meets Python –

A look on the Docker SDK for Python

“pip install docker”

Jan Wagner

Data Science Consultant

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 2

Agenda

1. Who am I?

2. The Docker Daemon/Service and Ways to communicate with it

3. Docker SDK for Python!

a. Where and how to get it

b. Code Examples

4. Ideas for Usecases

a. Using Python as Container-Starting-Script… end extend it

b. Get Container-Logs for further processing

c. pyTest with different Python Versions

5. Wrap Up

6. Q&A

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 3

About me – Hi, my name is Jan!

• 32 Years old, Not married yet, but engaged

• Data Science Consultant @ accantec consulting AG

• We focus on Data

– BI, Data Engineering & Data Science

• https://accantec.de

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 4

About me – Get in touch

j.wagner@accantec.com

https://twitter.com/wgnrjn

https://www.instagram.com/wgnrjn

https://github.com/wgnrjn

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 5

Docker

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 6

Your friendly house guest – The Whale!

The Docker Daemon/Service:

▪ Reachable per default from within your System (localhost)

▪ Reachable per configuration from the outside

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 7

Ways to communicate with it

GUI **

CLI * WebGUI ***

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 8

Ways to communicate with it

* Docker CLI ($docker run hello-world)

https://docs.docker.com/engine/reference/commandline/cli/

GUI **

CLI * WebGUI ***

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 9

Ways to communicate with it

* Docker CLI ($docker run hello-world)

https://docs.docker.com/engine/reference/commandline/cli/

** Kitematic

https://kitematic.com/

** Docker Extension for VS Code

https://github.com/microsoft/vscode-docker

GUI **

CLI * WebGUI ***

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 10

Ways to communicate with it

* Docker CLI ($docker run hello-world)

https://docs.docker.com/engine/reference/commandline/cli/

** Kitematic

https://kitematic.com/

** Docker Extension for VS Code

https://github.com/microsoft/vscode-docker

*** Portainer

https://www.portainer.io/

GUI **

CLI * WebGUI ***

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 11

GUI **

CLI * WebGUI ***

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 12

GUI **

CLI * WebGUI ***

Docker SDK

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 13

Get the Docker SDK for Python

• Officially available for Python and Golang

https://docs.docker.com/develop/sdk/

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 14

Get the Docker SDK for Python

• Officially available for Python and Golang

https://docs.docker.com/develop/sdk/

• pip/pip3 install docker

• Package on conda-forge available too

https://anaconda.org/conda-forge/docker-py

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 15

Code Examples

Taken from https://docs.docker.com/develop/sdk/examples/ *

* Code Examples on Website are in Python 2 Syntax … I don´t know why ☺

Docker CLI Docker SDK – Python

$ docker run hello-world import dockerclient = docker.from_env()print (client.containers.run('hello-world‘))

$ docker pull alpine import dockerclient = docker.from_env()image = client.images.pull("alpine")print (image.id)

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 16

Hint!

Docker CLI Docker SDK – Python

$ docker image […]

$ docker images

import dockerclient = docker.from_env()client.images.[…]

$ docker container […] import dockerclient = docker.from_env()container = client.containers.[…]

https://docker-py.readthedocs.io/en/stable/client.htmlhttps://docs.docker.com/engine/reference/commandline/container/https://docs.docker.com/engine/reference/commandline/image/https://docs.docker.com/engine/reference/commandline/images/

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 17

Ideas for Usecases

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 18

Using Python as Container-Starting-Script

https://hub.docker.com/_/postgres

$ docker run –name test-db -e POSTGRES_PASSWORD=EuroPython -d postgres

$ docker run --name test-db –e POSTGRES_PASSWORD=EuroPython -v /some/where/data:/var/lib/postgresql/data […….] postgres

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 19

Using Python as Container-Starting-Script

import os

os.system ("Do-Stuff -parameterXYZ")

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 20

Using Python as Container-Starting-Script

import docker

var1 = "..."

var2 = ["...", "123"]

var3 = "..."

client = docker.from_env()

client.containers.run('postgres', name=var1, environment=var2, mounts=var3, detach=True)

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 21

Extend it to …

[...]

client = docker.from_env()

myContainer = client.containers.get('myTestContainer')

myContainer.stop()

client.containers.prune()

newImage = client.images.build('path/to/Dockerfile')

newImageID = newImage.id()

[...]

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 22

Get Container-Logs

import docker

client = docker.from_env()

container = client.containers.run('ubuntu', detach=True)

f = open("myContainerLog", "w")

for line in container.logs(stream=True):

f.write(line)

f.close()

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 23

pyTest with different Python Versions

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 24

pyTest with different Python Versions

…. maybe just build a CI/CD Pipeline in this case ….

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 25

Wrap Up

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 26

Wrap Up

Depending on your Task at Hand:

- Python might be a better choice then e.g. bash Scripts

- Docker SDK might be a better choice then „import os“ or equivalents

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 27

Wrap Up

Depending on your Task at Hand:

- Python might be a better choice then e.g. bash Scripts

- Docker SDK might be a better choice then „import os“ or equivalents

Hope you now have:

- An (high level) Idea about the Docker SDK for Python

- Some inspiration for your Day-to-Day work with Docker and Python

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 28

Wrap Up

Depending on your Task at Hand:

- Python might be a better choice then e.g. bash Scripts

- Docker SDK might be a better choice then „import os“ or equivalents

Hope you now have:

- An (high level) Idea about the Docker SDK for Python

- Some inspiration for your Day-to-Day work with Docker and Python

Last Hint:

- Read the Docker Docs

accantec group | Alstertor 17 | 20095 Hamburg | Tel.: +49 (40) 67 59 59 0 | Fax.: +49 (40) 67 59 59 29 | www.accantec.de | Mail: info@accantec.de 29

Thank you!

---------------------

Questions?