docker swarm: light-weight orchestrator - qnibqnib.org/data/hpcw19/4_sched_1_swarm.pdfdocker swarm:...

1 www.prace-ri.eu

Docker SWARM: Light-weight orchestrator

HPCW @ ISC2019

Abdulrahman Azab

Dept. of Research Computing

University of Oslo, Norway

2 www.prace-ri.eu

PRACE 6IP WP6.2.3: The deployment of containers and full virtualised tools into HPC infrastructures

3 www.prace-ri.eu

Docker UCP

4 www.prace-ri.eu

5 www.prace-ri.eu

6 www.prace-ri.eu

[centos@ucp-manager-0 ~]$ UCP_IP=<ucp-manager-0 IP>[centos@ucp-manager-0 ~]$ UCP_FQDN=<ucp-manager-0 FQDN>[centos@ucp-manager-0 ~]$ docker container run --rm -it --nameucp \-v /var/run/docker.sock:/var/run/docker.sock \docker/ucp:3.1.0 install \--admin-username admin \--admin-password adminadmin \--san ${UCP_IP} \--san ${UCP_FQDN}

7 www.prace-ri.eu

8 www.prace-ri.eu

Orchestration

9 www.prace-ri.eu

10 www.prace-ri.eu

11 www.prace-ri.eu

Container NetworkOperations

12 www.prace-ri.eu

13 www.prace-ri.eu

14 www.prace-ri.eu

15 www.prace-ri.eu

Use case:Containerised HPC

With HTConcor and swarm

16 www.prace-ri.eu

HTCondor

17 www.prace-ri.eu

HTCondor VM universe

universe = vmexecutable = vmware_sample_joblog = simple.vm.log.txtvm_type = vmwarevm_memory = 64 vmware_dir = C:\condor-testvm_checkpoint = true queue

18 www.prace-ri.eu

HTCondor Docker universe

universe = docker docker_image = debianexecutable = /bin/cat arguments = /etc/hosts output = out.$(Process) error = err.$(Process) request_memory = 100M queue 10

19 www.prace-ri.eu

Containerised HTCondor on Swarm

20 www.prace-ri.eu

21 www.prace-ri.eu

22 www.prace-ri.eu

23 www.prace-ri.eu

24 www.prace-ri.eu