linux containers

Slide 1

Linux ContainersJason ArneilSolutions Architecte-DBA LtdA RedStack Technology CompanyOracle EMEA Database Partner of the Year Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] ContainersTypes of Virtualisation Linux ContainersDemoResource ManagementOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

Much shorter session 2

jarneil.wordpress.com@jarneilOracle EMEA Database Partner of the Year

Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

My name is Jason Arneil

Been in IT for around 15 years

SA/DBA

Written quite a few blog articles

also on twitter

now an oracle AceAbout Red Stack Technology


4VirtualisationHardwareVirtualisation O/SPartitionsSoftwareVirtualisationOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

5


Quite a range of virtualisation options from even just Oracle

6containers can just be thought of as a collection of processes, separated from the main host processes via a set of resource namespaces and constrained via control groups resource tunablesOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

provide a means to isolate individual services or applications

as well as of a complete Linux operating system from other services running on the same host

To accomplish this, each container gets its own directory structure, network devices, IP addresses and process table.

The processes running in other containers or the host system are not visible from inside a container

Linux Containers allow for fine granular control of resources like RAM, CPU or disk I/O.

This allows a very direct and fair distribution of CPU and I/O-resources

LXC is a userspace interface for the Linux kernel containment features.7HardwareStandard Host O/SO/S VirtulisationContainer 1Container 2HardwareDriver SupportHypervisorVirtual Machine MonitorVM2Guest O/SVM1Guest O/SOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

Containers share the same kernel

Communicate with the host via standard system calls

Are as isolating as hypervisors

You can think of containers as an abstraction at the OS kernel level

Anything you can do with the OS you can do with a containerie. Resource management becomes easy an OS is used to doing that

Hypervisors run separate kernels you essentially need a 2nd operating system one in the host one in guest

Communicate with host via virtual machine emulation

You can think of hypervisors as an abstraction the hardware level

8


You can think of jails as the first O/S virtualisation technology

They go way back in time 30+ years!!!

My first introduction to virtualisation came in the early 2000s working for the company running the UK DNS syatem

Some of the NAMESERVERS used freebsd and something called jails

This is a virtualisation technique that allows you partition a system into independent mini-systems essentially you can sandbox these mini-systems from each other but they all run under one O/S image

This is O/S virtualisation

The big driver for us at the time was security9


I next encountered Operating System virtualisation with Solaris Zones or containers.

They first appeared around 2005

An application running inside a container will be executed directly on the operating system kernel of the host system, shielded from all other running processes in a sandbox-like environment.

10


linux containers are very similar to these

They first appeared in 2008

An application running inside a container will be executed directly on the operating system kernel of the host system, shielded from all other running processes in a sandbox-like environment.

11Linux ContainersOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

provide a means to isolate individual services or applications

as well as of a complete Linux operating system from other services running on the same host

To accomplish this, each container gets its own directory structure, network devices, IP addresses and process table.

The processes running in other containers or the host system are not visible from inside a container

Linux Containers allow for fine granular control of resources like RAM, CPU or disk I/O.

This allows a very direct and fair distribution of CPU and I/O-resources

LXC is a userspace interface for the Linux kernel containment features.122.6.39-400.210.2.el6uekOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

the same kernel version runs on the host as well as in the containers

it is not possible to run different Linux kernel versions or other operating systems like Microsoft Windows or Oracle Solaris for x86 inside a container

it is possible to run different Linux distribution versions (e.g. Fedora Linux in a container on top of an Oracle Linux host

Because the 2.6.39 kernel is supported on OEL5 and OEL6 You can also run those different versions of OEL in separate containers.

BE AWARE if a container causes a kernel crash it will bring down ALL containers and the host kernel!

13


Since Linux Containers are fully implemented on the OS level (the Linux kernel),

they can be easily combined with other virtualization technologies.

e.g. Run on top of a linux guest sitting on top an Oracle VM server14Why?Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

Going to look at why you might use containers some of the arguments apply equally well to all types of Vritualisation

But containers also have certain advantages over hypervisors15


Consolidation of multiple separate Linux systems on one server

Not all systems are busy ALL the time its a lightweight way of doing virtualisation16


Running multiple instances of an application in parallel

Have had the situation where multiple developers want to run their application to test at the same time

Each user receives his "own" application instance, with a defined level of service/performance. NO one user can hog the system

17


Quickly creating sandbox environments for development and testing purposes

containers that have been created and configured once can be archived as templates and can be duplicated (cloned)

CLONES can be quickly discarded:

You can have a repeatable software builds and test environments always have the same starting point

This is similar to what is achievable with typical software virtualisation 18


Safe execution of an individual application

if an application running inside a container has been compromised because of a security vulnerability,

the host system and other containers remain unaffected

The potential damage can be minimized, analyzed and resolved directly from the host system.

19


One advantage containers have over using a hypervisor

is that you can run more of them with the same amount of resources you can make them denser

Containers only run 1 kernel per physical system

hypervisors run 1 kernel per guest + 1 for the host

better resource management with a container

Containers are

20


Containers are also much more elastic than full VM/hypervisor solution

It is much, much fast to boot/stop a container than a full VM

Vertical scaling is also very straightforward with containers21


Containers are also much more elastic than full VM/hypervisor solution

It is much, much fast to boot/stop a container than a full VM

Vertical scaling is also very straightforward with containers22Control GroupsOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

Control Groups provide means to manage and monitor the allocation of resources for individual processes or process groups.

Among other things, you can restrict the maximum amount of

memory, CPU cycles as well as the disk and network throughput (in MB/s or IOP/s)23cgroupOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

A cgroup associates a set of processes or tasks with a set of parameters for one or my subsystems

24subsystemOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

A subsystem is a resource controller that applies limits or acts upon a group of processes

Each subsystem has specific parameters that enable resource control and reporting mechanism25lssubsys -amOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

To list the subsystems26cpuset /cgroup/cpusetcpu /cgroup/cpucpuacct /cgroup/cpuacctmemory /cgroup/memorydevices /cgroup/devicesfreezer /cgroup/freezernet_cls /cgroup/net_clsblkio /cgroup/blkioOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

cpuset - assigns individual CPUs and memory nodes to cgroup tasks

cpu - schedules CPU access

cpuacct - reports total CPU time used.

memory - reports or limits memory use.

devices - grants or denies access to devices.

freezer - suspends or resumes tasks.

net_cls - tags outgoing network packets with an identifier.

blkio - report/control I/O bandwidth for block devices.27hierachyOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

A hierachy is a set of subsystems and cgroups arranged in a tree

so that every system process is in exactly one of the cgroups in the hierachy28Name SpacesOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

Name Spaces help to isolate process groups from each other

e.g. the visibility of other running processes or the exclusive access to a network device

It's also possible to restrict a process group's access and visibility of the entire file system hierarchy

CGroups and Name Spaces provide the foundation on which Linux containers are based on, but they can actually be used independently as well.29

LXCOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

30LXC is a userspace interface for the Linux kernel containment features.Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

We are going to use lxc31lxc-create n cont1 t oracle -- --release=6.latestOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

To create a linux container you use the lxc-create command

-n is the name of the container you are creating

-t is the template name there are quite a few to choose from stored under

/usr/share/lxc/templates

These are bash scripts

The lxc-create command runs the template script lxc-oracle

The strange double dash is passing an option to the template32lxc-create: No config file specified, using the default config /etc/lxc/default.confHost is OracleServer 6.4Create configuration file /container/cont1/configDownloading release 6.latest for x86_64Loaded plugins: refresh-packagekit, securityol6_latest | 1.4 kB 00:00 ol6_latest/primary | 33 MB 00:04 ol6_latest 22589/22589Setting up Install ProcessResolving Dependencies--> Running transaction check---> Package chkconfig.x86_64 0:1.3.49.3-2.el6_4.1 will be installed--> Processing Dependency: libc.so.6(GLIBC_2.2.5)(64bit) for package: chkconfig-1.3.49.3-2.el6_4.1.x86_64--> Processing Dependency: libc.so.6(GLIBC_2.8)(64bit) for package: chkconfig-1.3.49.3-2.el6_4.1.x86_64..Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

You can see that it actually downloads using yum a bunch of rpms when isntalling the container

but then we did ask for 6.latest!

33Complete!Rebuilding rpm databaseConfiguring container for Oracle Linux 6.4Added container user:oracle password:oracleAdded container user:root password:rootContainer : /container/cont1/rootfsConfig : /container/cont1/configNetwork : eth0 (veth) on virbr0'oracle' template installed'cont1' createdOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

Once complete it tells us

you can see where the container is /container

and the config as well

34templatesOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

The lxc-oracle template script defines system settings and resources that are assigned to a running container, including:

the default passwords for the oracle and root users, which are set to oracle and root respectively

the host name (lxc.utsname), which is set to the name of the container

the number of available terminals (lxc.tty), which is set to 4

the location of the container's root file system on the host (lxc.rootfs)

the location of the fstab mount configuration file (lxc.mount)

all system capabilities that are not available to the container (lxc.cap.drop)

the local network interface configuration (lxc.network)

all whitelisted cgroup devices (lxc.cgroup.devices.allow)35ls -lh /container/cont1/

-rw-r--r--. 1 root root 2.1K Nov 25 09:06 config-rw-r--r--. 1 root root 195 Nov 25 09:06 fstabdr-xr-xr-x. 21 root root 4.0K Nov 25 09:08 rootfsOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

we have a config file for the container

an fstab for the container lists filesystems

and we have the rootfs with everything youd expect on /36lxc-start -n cont1 dOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

This will start the container cont1

and the d will ensure its in the background, otherwise it takes control of your shell and you get a login prompt.

37lxc-lsOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

This will give you a list of the containers you have created

38lxc-info n cont1Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

This will give you a list of the containers you have created

39ps elf --forestOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

40root 2771 2748 0 4738 1164 0 10:04 pts/0 00:00:00 | \_ lxc-start -n cont1root 2775 2771 0 4819 1496 0 10:04 ? 00:00:00 | \_ /sbin/initroot 3005 2775 0 2291 604 0 10:04 ? 00:00:00 | \_ /sbin/dhclient -H cont1 -1 -q -lf /var/lib/dhclient/dhclient-eth0.leases -pf /var/run/dhclient-eth0.pid eth0root 3028 2775 0 44888 1344 0 10:04 ? 00:00:00 | \_ /sbin/rsyslogd -i /var/run/syslogd.pid -c 5root 3070 2775 0 16575 1160 0 10:04 ? 00:00:00 | \_ /usr/sbin/sshdroot 3081 2775 0 1027 576 0 10:04 pts/1 00:00:00 | \_ /sbin/mingetty /dev/tty1root 3083 2775 0 1027 572 0 10:04 pts/2 00:00:00 | \_ /sbin/mingetty /dev/tty2root 3085 2775 0 1027 572 0 10:04 pts/3 00:00:00 | \_ /sbin/mingetty /dev/tty3root 3087 2775 0 1027 576 0 10:04 pts/4 00:00:00 | \_ /sbin/mingetty /dev/tty4root 3208 2775 0 1027 568 0 10:23 pts/5 00:00:00 | \_ /sbin/mingetty /dev/consoleOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

41


By default, the lxc-oracle template script sets up networking by setting up a veth bridge

In this mode, a container obtains its IP address from the dnsmasq server that libvirtd runs on the private virtual bridge network (virbr0) between the container and the host

The host allows a container to connect to the rest of the network by using NAT rules in iptables,

However you CANt ccess the container with this outside the host

To have off-host access you can use either the MACVLAN bridge or VEPA42lxc-console -n cont1 Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

How do you connect to your container?

ssh is running and you can ssh into it, if you know the IP address

or

you can connect with this

this will give you a login prompt just like being on the console

43lxc-clone -o cont1 n cont2Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

You can very straightforwardly clone a container

and if you are using a filesystem with snapshotting technology you can create it as a snapshot

44Tweaking configurationCopying rootfs...Updating rootfs...'cont2' createdOracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

You can very straightforwardly clone a container

and if you are using a filesystem with snapshotting technology you can create it as a snapshot

45lxc-freeze -n cont1 Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

You can suspend a container

This stops all processing in container46lxc-unfreeze -n cont1 Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

You can un suspend a container

47lxc-cgroup -n cont1 state-object Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

To control resources of a container

you can use control groups

for example you could limit the container to a subset of the cpus available on the box, or a number cpu shares

or limit the amount of memory available to the container

You can even limit the I/O a container could do

This could be useful in a consolidation exercise

To make change permanent edit /container/ config file

48lxc-shutdown -n cont1 Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

This will shutdown container in an orderly fashion

49lxc-stop -n cont1 Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

This basically kills the container dead without trying to shutdown properly

50lxc-destroy -n cont1 Oracle Partner of the year 2010 & 2012 +44 (0) 844 811 [email protected] RedStack Technology CompanyOracle EMEA Database Partner of the Year

This removes the container from the host system

51Q & A


52

linux containers

Documents