using ansible to core os & kubernetes clusters
TRANSCRIPT
![Page 1: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/1.jpg)
Using Ansible to CoreOs & Kubernetes ClustersAnsible Meetup Berlin 28.4.2016
![Page 2: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/2.jpg)
Agenda
● Kubernetes?● Coreos?● Components
○ Etcd○ kubernetes master○ Kubernetes nodes
● Doing it all with ansible● Demo
![Page 3: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/3.jpg)
Motivation
● GIFEE - Google Infrastructure for Everyone Else● Useful if you can't do Public Cloud for example for legal issues● Platform for microservices
![Page 4: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/4.jpg)
CoreOs
![Page 5: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/5.jpg)
What is CoreOs
● Modern Minimal Linux Os● designed for security, consistency, and reliability● No packages, additional applications come via containers● Configured via cloud config● https://coreos.com/os/docs/latest/cloud-config.html● No python !
![Page 6: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/6.jpg)
Etcd
● /etc distributed● Consistent reliable distributed key-value store● Used to keep configuration in a cluster● Backend for kubernetes state● Leader election● Uneven number of nodes to be able to form a majority even in case of
network partitions
![Page 7: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/7.jpg)
Etcd small cluster
● Etcd runs on every node● Etcd competes for resources with the actual
workloads of the cluster● Might be a problem in production when the
cluster gets high load
![Page 8: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/8.jpg)
Etcd Production deployment
![Page 9: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/9.jpg)
Kubernetes
![Page 10: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/10.jpg)
Kubernetes
● Container Orchestration Platform Initiated by Google● Automatic bin packing● Service discovery and load balancing● Automated rollouts and rollbacks● Secret and configuration management● Self-healing
![Page 11: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/11.jpg)
Kubernetes Architecture overview
![Page 12: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/12.jpg)
Deploying all this with ...
![Page 13: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/13.jpg)
The inventory - groups: etcd & coreos
[etcd-node]# nodes to run etcd and be part of the etcd cluster ( uneven # )
[etcd-proxy]# nodes to run etcd in proxy mode and proxy requests to the cluster
[coreos:children]Etcd-nodeetcd-proxy
![Page 14: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/14.jpg)
Inventory groups: kubernetes
[kubernetes-master]# api server and master elected components (scheduler & kube-controller)
[kubernetes-node]# kubernetes worker node - runs kubelet and kube-proxy
[kubernetes:children]Kubernetes-masterkubernetes-node
![Page 15: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/15.jpg)
[loadbalancer]# run a nginx-reverse-proxy (kubernetes daemonset)
[ceph-mon]# run monitor for ceph on kubernetes (daemonset) (should be 3 nodes)
[ceph-osd]# run osd for ceph on kubernetes (daemonset)
Inventory additional groups
![Page 16: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/16.jpg)
Inventory: Global Configuration Variables
Kube_master_dns_name # dns netry for kube master nodes if you have dnsKube_master_ip # the master node kubectl should talk to if you have no dns kube_cluster_name=privatkubernetes_version=1.2.3k8s_service_ip_range=10.100.0.0/16k8s_service_ip=10.100.0.1k8s_dns_service_ip=10.100.0.10k8s_dns_domain=cluster.local
![Page 17: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/17.jpg)
Bare metal Provider
● Designed to use every baremetal provider that provides an api to boot servers into a rescue system and access via ssh
● Sample implementations for:○ Hetzner○ Kimsufi ( ovh )
● Minimal hardware requirements ○ One network interface is enough
![Page 18: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/18.jpg)
Exposing Services
https://github.com/cornelius-keller/kubernetes-reverseproxy
● Nginx reverse proxy ● Runs as a daemonset on dedicated nodes● Wirtes an nginx config from specially annotated services
![Page 19: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/19.jpg)
How it works
● Generate all certificates needed for kubernetes and etcd● Boot nodes into a rescue system● Install coreos ● Reboot the system● Install python (pypi) to be able to work with ansible● Template the cloudconfig ● Reboot again
Takes about 8-10 min for a three node cluster
![Page 20: Using ansible to core os & kubernetes clusters](https://reader031.vdocuments.site/reader031/viewer/2022021813/5879e5ac1a28ab15288b57ff/html5/thumbnails/20.jpg)
Where to find
https://github.com/cornelius-keller/ansible-coroeos-kubernetes
Thanks!