ELC 2017 – Randy Witt – Intel Open Source Technology Center

Cross platform enablement for the yocto project with containers

Why’d I even do this?My personal problems

3

• Yocto Project QA (Autobuilder) builds on multiple distros

• On failure what are the options?

• ssh in, clone repo, set up build dir

• Create a virtual machine and try there

• Way too much overhead for me

THE multiple distro Problem

4

• Ctrl-c may not clean up all processes

• Must manually find the processes and kill them

• May not even know the processes are there

Bitbake SOMETIMES LEAKS processes

Quick overviewcontainers As a SOlution

6

• Leverage Linux® kernel features

• namespaces for isolation (pid, network, mount)

• Running a container can be as simple as using “unshare”

• cgroups for process encapsulation and resource management

• Restrict number of cores, amount of memory, …

• All processes for the container run in a cgroup so can kill at cgroup level

• Most things that run containers, leverage these kernel features

• docker, lxc, …

Containers aren’t magic

7

Inside container

Pid namespace examplepokyuser@6325b7c8feaf:~$ sleep 5000 &[1] 32pokyuser@6325b7c8feaf:~$ ps -C sleep -o pid,start,args

PID STARTED COMMAND32 15:24:03 sleep 5000

~% ps -C sleep -o pid,start,argsPID STARTED COMMAND

8257 15:24:03 sleep 5000

Outside container

8

• Use a Dockerfile to create an image

• Dockerfile used to install software to the image and configure it

• A container uses a temporary instance of the image

• Modifications to the filesystem instance aren’t preserved

Docker

9

sample dockerfileFROM ubuntu-16.04

RUN apt-get install python

CMD echo “Hello from inside the container!”

10

• --rm: Remove the container after it exits

• -it: Interactive terminal with a tty

• -v: bind mount /foo to the container as /bar

• c1: Name of the image to run

Docker rundocker run --rm -it -v /foo:/bar c1

11

Linux/foo

12

Linux/foo

docker run –v /foo:/bar c1

13

Linux/foo

docker run –v /foo:/bar c1

Docker

14

Linux

c1

/foo

docker run –v /foo:/bar c1

Docker

15

Linux

/barc1

/foo

docker run –v /foo:/bar c1

Docker

16

Linux

/barc1

/foo

docker run –v /foo:/bar c1

Docker

docker run –v /foo:/baz c2

17

Linux

/barc1 c2

/foo

docker run –v /foo:/bar c1

docker run –v /foo:/baz c2

Docker

/baz

Yocto Project containersWhat’s available?

19

• Drops to a shell where you follow normal Yocto Project instructions

• Default based on Ubuntu 14.04

• Can use a different distro

Poky container

docker run --rm -it -v /home/myuser/mystuff:/workdir crops/poky--workdir=/workdir

docker run --rm -it -v /home/myuser/mystuff:/workdircrops/poky:fedora-24 --workdir=/workdir

20

• --workdir: The working directory when dropped to the shell

Poky containerdocker run --rm -it -v /home/myuser/mystuff:/workdir crops/poky--workdir=/workdir

21

• debian-8• fedora-22• fedora-23• fedora-24• fedora-25• opensuse-13.2• opensuse-42.1

• opensuse-42.2• ubuntu-14.04• ubuntu-16.04• ubuntu-16.10

POKY CONTAINER distrosdocker run --rm -it -v /home/myuser/mystuff:/workdircrops/poky:opensuse-42.2 --workdir=/workdir

22

https://www.youtube.com/watch?v=vt18U5twrgw

Poky container screencast

23

• Downloads an extensible sdk and drops to a shell ready to run sdkcommands

• If the sdk has already been installed and setup, just leave off the url

Extensible sdk container

docker run --rm -it -v /home/myuser/sdkstuff:/workdircrops/extsdk-container--url http://someserver/extensible_sdk_installer.sh

docker run --rm -it -v /home/myuser/sdkstuff:/workdircrops/extsdk-container

24

https://www.youtube.com/watch?v=L-sXqUoU49Y

extsdk container screencast

25

• Runs toaster

• -p: Forwards port 8000 in the container to 127.0.0.1:18000

toaster container

docker run -it --rm -p 127.0.0.1:18000:8000-v /home/myuser/toasterstuff:/workdir crops/toaster

26

https://www.youtube.com/watch?v=LJ9TBsuMwFA

Toaster container screencast

OTHER PLATFORMSRunning the containers on the macOS™ operating system

28

• Setup

• Instructions at https://github.com/crops/docker-win-mac-docs/wiki

• Runs in a hypervisor (intended to be transparent)

• Uses a Docker volume rather than bind mount

Differences

29

Linux

/barc1 c2

/foo

docker run –v /foo:/bar c1

docker run –v /foo:/baz c2

Docker

/baz

30

Linux

/barc1 c2

/foo

Docker

/bazdocker run –v vol:/bar c1

docker run –v vol:/baz c2

31

Linux

/barc1 c2

/foo

Docker

/baz

macOS

docker run –v vol:/bar c1

docker run –v vol:/baz c2

32

Linux

/barc1 c2

/foo

Docker

/baz

macOS

Hypervisor

docker run –v vol:/bar c1

docker run –v vol:/baz c2

33

Linux

/barc1 c2docker run –v vol:/bar c1

docker run –v vol:/baz c2

Docker

/baz

Hypervisor

macOS

vol

34

Linux

/barc1 c2docker run –v vol:/bar c1

docker run –v vol:/baz c2

Docker

/baz

Hypervisor

macOS

vol

docker start samba

35

Linux

/barc1 c2docker run –v vol:/bar c1

docker run –v vol:/baz c2

Docker

Hypervisor

macOS

samba

docker start samba

/baz /workdir

vol

36

https://www.youtube.com/watch?v=w9_Wt6iQK3g

Other platform screencast

Questions?

37

38

• poky container• https://github.com/crops/poky-container

• https://hub.docker.com/r/crops/poky/

• extsdk container• https://github.com/crops/extsdk-container

• https://hub.docker.com/r/crops/extsdk-container/• toaster container• https://github.com/crops/toaster-container

• https://hub.docker.com/r/crops/toaster/

More info

39

macOS™ are registered trademarks of Apple Inc.

Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.

“Docker and the Docker logo are trademarks or registered trademarks of Docker, Inc. in the United States and/or other countries. Docker, Inc. and other parties may also have trademark rights in other terms used herein.”

Credits