microcontainers, microservices, microservers? less [linux] is more!
TRANSCRIPT
![Page 1: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/1.jpg)
1
Microcontainers, Microservices, Microservers. Less [Linux] is more!
Belfast Devops15/09/2015
Dermot [email protected]
http://uk.linkedin.com/in/bradleydermot
![Page 2: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/2.jpg)
2
My Background
1993-1997 Set up first ISP in Northern Ireland (Genesis Project Ltd)
1997-2006Senior Technical Support Engineer @ Aldiscon/APiON/Openwave Systems
2006-2011Mobile Technology Consultant @ SLA Mobile, on-site at Vodafone Global in Germany
2011-2012Contractor @ Alcatel-Lucent Network Services in Germany
2012-2015 Contractor @ Vodafone Group Services in Germany
2015 April-August Sabbatical/career break
2015 September Senior Linux Administrator @ Proofpoint
![Page 3: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/3.jpg)
3
Talk Overview
In a Microservices vision large scale services can be built by combining Application Containers like Lego bricks. However App Containers are not usually lightweight – a typical Docker container uses Ubuntu as its base. Alternatively if based on minimal Linux distributions or (in a more extreme case) applications are just statically linked, the resulting containers can be far smaller. Lightweight means faster startup & migration time between hosts and less RAM use. And as for the container hosts? Whether VM instances or bare metal, the underlying OS can also be stripped right down – CoreOS and RancherOS were designed with exactly this in mind. Together lightweight containers and hosts mean savings – fewer, smaller, cheaper cloud VM instances or bare metal servers (such as Microservers) and potentially improved security.
![Page 4: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/4.jpg)
4
The progression from physical machines to application containers
![Page 5: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/5.jpg)
5
Physical Machines
● Real machines hosting applications● Individual machines often dedicated to hosting only a single or a few applications
● Typically machines are underutilised (CPU, RAM use)
IBM HPDELL* * *
* apparently use of their logos requires prior written permission!
![Page 6: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/6.jpg)
6
Virtual Machines
● Provides a degree of application isolation allowing several physical machines to be replaced by 1 physical machine hosting multiple VMs
● Results in better physical machine utilisation
![Page 7: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/7.jpg)
7
OS Containers
● Single kernel shared between containers – better I/O (disk, network) performance than VMs but “weaker” security isolation between containers
● Faster startup time than VMs
![Page 8: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/8.jpg)
8
Application Containers
● An application plus “just enough” OS to support the app (i.e. no sshd, no syslogd, no init, etc)
● Immutable Infrastructure – no need to use Puppet/Chef/Ansible/Salt to manage them
● The same container is portable across physical hosts, VMs, and multiple cloud providers
● The exact same container can be run on a developer's laptop, in a testlab, and in production environment
![Page 9: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/9.jpg)
9
Various architectures of container use
Physical Machine
Physical Machine
Host OS
Host OSHost OS
Hypervisor
Container
Container
Container
Container
Container
Container
Container
Container
Cloud Provider Responsibility
Cloud Provider Responsibility
Bare Metal Virtualisation Cloud for VMs Cloud for Containers
![Page 10: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/10.jpg)
10
Reducing the size of App Containers
![Page 11: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/11.jpg)
11
The range of sizes
Normal “full fat” Linux distribution
Lean but generic Linux distribution
Minimal application-specific container (perhaps just a statically-linked binary)
Larger
Smaller
![Page 12: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/12.jpg)
12
Minimal App Containers (1)
An application-specific container where the application is compiled & either statically linked (resulting in just a single binary file in the container) or dynamically linked (resulting in a binary plus dependant libraries).
![Page 13: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/13.jpg)
13
Minimal App Containers (2)
● “dockerize” is a utility that analyses a dynamically-linked binary to create a container with the binary plus just the libraries it needs.
● “gockerize” is a utility for building statically linked golang binaries and creating Docker container images.
![Page 14: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/14.jpg)
14
Minimal App Containers (3)
Examples of “extreme” minimal containers:
● Nginx (with Openssl) – 3.034MB
● Python (StaticPython) – 8.5MB
● PostgreSQL – 15.82MB
http://mwcampbell.us/blog/tiny-docker-musl-images.html
https://github.com/elyase/docker/tree/master/staticpython
http://yasermartinez.com/blog/posts/creating-super-small-docker-images.html
![Page 15: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/15.jpg)
15
“Off the shelf” App Container base sizes
![Page 16: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/16.jpg)
16
Alpine Linux (1)
● Distribution based on musl and Busybox
● Official base image size in Docker Hub is only 5MB in size!
● Comes with a package manager. Large number of common applications already packaged.
● Provides a useful middle ground between normal “full fat” distributions and handcrafted minimal app-specific containers.
![Page 17: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/17.jpg)
17
Alpine Linux (2)
NGINX – 8MB
OpenJDK Java 7 JRE, 123MB
Oracle Java 8 JRE, 173MB
Example sizes of Alpine-based containers:
![Page 18: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/18.jpg)
18
Reducing the size of the Host OS
![Page 19: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/19.jpg)
19
Host OS Options (1)
● RancherOS, 20MB● Boot2Docker, 24MB● CoreOS, 190MB● Vmware Photon OS, 207MB
● Intel ClearLinux, 800MB
● Project Atomic, 800MB● Snappy Core Linux, 3.9GB??
Container-oriented distributions:
![Page 20: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/20.jpg)
20
Host OS Options (2)
● TinyCore Linux, 10MB+
● Alpine Linux, 87MB+
Generic distributions:
Size is of base distribution, does not take into account size of Docker tools etc...
![Page 21: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/21.jpg)
21
Combined small App Containers with small Host OS
![Page 22: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/22.jpg)
22
Benefits of combined reductions
● Smaller spec Cloud VMs or physical servers can be utilised
● Cost savings!
● Quicker app container and/or host VM migration
● Smaller attack surface
![Page 23: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/23.jpg)
23
Special case: App Container & slim VM merged
![Page 24: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/24.jpg)
24
Intel Clear Containers
● KVM hypervisor● Uses kvmtool (no need for BIOS/UEFI)● Stripped-down Linux kernel (only Virtio devices)● Systemd as init● Rkt containers supported (docker support soon)● Startup almost as fast (150ms) as native container● Per-container RAM overhead of 18-20MB● Container running inside a lightweight VM
![Page 25: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/25.jpg)
25
Microservices
![Page 26: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/26.jpg)
26
Microservices (1)
“Microservices is a software architecture style in which complex applications are composed of small, independent processes communicating with each other using language-agnostic APIs. These services are small, highly decoupled and focus on doing a small task, facilitating a modular approach to system-building”
Wikipedia definition:
http://en.wikipedia.org/wiki/Microservices
![Page 27: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/27.jpg)
27
Microservices (2)
● “small independent processes”
● “highly decoupled”
● “modular approach to system building”
App Containers fit the Microservices model perfectly:
![Page 28: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/28.jpg)
28
Microservices (3)
Database Database
Application
Content Server
Load Balancer
Content Server
Application Application
Lego brick approach to building services
![Page 29: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/29.jpg)
29
Microservices (4)
● Load Balancer: HAproxy● Content Server: Apache, Nginx● Application run-times: Perl, PHP, Python, Ruby,
Node.js, JVM● Database: MariaDB, PostgreSQL, MongoDB● Caching: Memcached, Varnish
Typical App Container building blocks:
![Page 30: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/30.jpg)
30
Microservices (5)
Perhaps there is a demand for a curated set of such minimal App Container building blocks?
Am I volunteering? Who knows...
![Page 31: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/31.jpg)
31
Microservers
![Page 32: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/32.jpg)
32
Microservers (1)
Dell CS5220 12 unit chassis
![Page 33: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/33.jpg)
33
Microserver (2)
Facebook/OCP Mono Lake <90W microserver for Yosemite sled
![Page 34: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/34.jpg)
34
Microservers (3)
![Page 35: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/35.jpg)
35
Microservers (4)
● Low cost, low power usage, small size physical server
● Not every task needs high-end CPUs and lots of RAM
![Page 36: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/36.jpg)
36
Micro Data Centre
![Page 37: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/37.jpg)
37
Micro Data Centre (1)
![Page 38: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/38.jpg)
38
Micro Data Centre (2)
● Small rack cabinet with multiple compute nodes, storage nodes, admin node(s), network switch, UPS, etc
● 10U cabinet capable of hosting 4000+ containers?
![Page 39: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/39.jpg)
39
Micro Data Centre (3)
● No special (physical, power, cooling) room requirements unlike a normal data centre● Easier to keep data on-site for legal or privacy reasons● Potentially economical to have multi-site redundancy with 2 MDCs in separate but geographically-close (low network latency) locations
![Page 40: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/40.jpg)
40
Personal experimentation – my current Micro Data Centre
![Page 41: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/41.jpg)
41
Testlab summary
● Multiple Compute Nodes● 3+ Storage Nodes● Admin Node● Managed Switch
![Page 42: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/42.jpg)
42
Compute Nodes (1)
● Off-the-shelf Mini-ITX motherboards with low power CPUs (with passive heatsinks or heatsink/fans)
● Small amounts of RAM, 4-8Gb. Low power (1.35V) DDR3L if possible.
● No local storage● Low wattage small power supply● Machines PXE boot the Host OS and run directly from
RAM
![Page 43: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/43.jpg)
43
Compute Nodes (2)
![Page 44: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/44.jpg)
44
Compute Nodes (3)
● My intention is to make a custom low-cost blade-type chassis for nodes.
● Chassis will be approx. 6U high and 250mm deep with 12 compute nodes per chassis.
● Individual nodes will be approx. 6Ux 35mm x 245mm
![Page 45: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/45.jpg)
45
Storage Nodes
● Low power CPUs on Mini-ITX motherboards● Mix of SSDs and HDDs● Using CEPH to provide storage services
![Page 46: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/46.jpg)
46
Admin Node
Pair of HDDs used with software mirroring● Provides DHCP/TFTP/HTTP services for PXE booting
Compute Nodes● Will run InfluxDB, ElasticSearch and Kibana for metric
and log analysis
![Page 47: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/47.jpg)
47
Network Infrastructure
● Gigabit Ethernet managed switch with VLAN, IPv6, and SSH support
● Use Ansible, via SSH, to configure the switch● Plan is for all containers & hosts to use only IPv6
addressing. The entry point to the “cloud” (i.e. Load Balancer) will handle both IPv4 and IPv6 external addresses and forward traffic onwards to internal IPv6 addresses.
![Page 48: Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://reader036.vdocuments.site/reader036/viewer/2022062412/58f0d4aa1a28ab39538b4593/html5/thumbnails/48.jpg)
48
The End!
Questions?