kubernetes immersion
TRANSCRIPT
Kubernetes Immersion
Introduction: Who Am I
Juan Larriba
DevOps Engineer at everis innolab
@compilemymind
Introduction: Containers
Containers are gaining a lot of traction because they isolate different applications on the same physical or virtual hardware
Usually, servers are provisioned for the worst case scenario, leading to a lot of unused resources most of the time
Containerization lets us to securely share that hardware between different applications that can work a different times, optimizing the usage time
Introduction: Container Orchestrators
Currently there are 4 main container orchestrators fighting to be the market leader
Kubernetes
Mesos
Docker Swarm
Nomad
Kubernetes Architecture
Architecture
Architecture
Kubernetes is programmed as a monolithic application but deployed as a microservices application
It provides two basic components: Hyperkube and Kubelet
It relies on external services for networking and persistent storage of its own state
All comunications, both external and internal, use the HTTPS protocol
Architecture: Software Defined Networking
One of the first problems we face when working with Docker, is the manual port management issue
When deploying a number of containers on the same machine, we need to track manually which ports is exposing each container
To avoid this problem, Kubernetes uses a Software Defined Networking (commonly Flannel, but also Weave and even cloud providers native IaaS)
Each container is then automatically assigned a different IP, so all of them can expose the same port
Architecture: etcd
Kubernetes needs to persist its state in some kind of persistent storage
It uses exclusively etcd as its backend
etcd is a distributed key-value storage created by the CoreOS team
Each etcd major version breaks the previous API
As of Kubernetes 1.4, the version used is etcd2
Architecture: Kubelet
The Kubelet is a native Linux daemon that needs to be executed in each member of a cluster: masters and nodes
It communicates with its node Docker API to effectively launch the Docker containers required by other Kubernetes components
It really can work standalone, acting as a Supervisord of Docker containers
Before 1.4, it was the only Kubernetes component that did not work as a Docker container. From 1.4 onwards, it is also deployed in a Docker container
Architecture: kube-apiserver
It is deployed only in the master
It is the entrypoint for the Kubernetes cluster
It exposes a REST API
The client communicates and sends commands to the apiserver, who validates the information sent and if it is correct stores it in etcd
Architecture: kube-scheduler
It is deployed only in the master
The Scheduler is aware of the cluster status and decides where the new objects must be colocated
It is a very complex piece of software, the real “brain” of the Kubernetes cluster
As stated in Kubernetes documentation:
The scheduler needs to take into account individual and collective resource requirements, quality of service requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, deadlines, and so on
Architecture: kube-controller-manager
It is deployed only in the master
The Controller-Manager is a the control loop of the cluster
The Controller-Manager watches the shared state of the cluster stored in etcd by the API Server
It continuously compares the desired state of the cluster with the current state and notifies the other components of the cluster to perform the actions needed to move the cluster towards the desired state
Architecture: kube-proxy
It is deployed as a static pod on each node of the cluster
Implements Services capabilities
Kubernetes Addons
Addons: Ingress Controller
It provides a way to route external requests to applications in the cluster
Matches DNS names and contexts (which external clients like browsers can understand) to Kubernetes Services
One specification, multiple implementations
Currently we use the Nginx implementation, but a custom implementation is easily done
Addons: Dashboard
A web frontend for the cluster
It shows in a graphical UI all the information that can be obtained through the API or the CLI
Addons: Heapster
Reads monitoring data from the Kubelet (extracted from the Docker API and the node it lives in) and exposes it via a REST API
It can be deployed standalone and it will store all the cluster metrics for the last 15 minutes
It can be plugged to different backends, currently supporting Log, InfluxDB, Google Cloud Monitoring, Google Cloud Logging, Hawkular-Metrics, OpenTSDB, Monasca, Kafka, Riemann, Elasticsearch…
When plugged to a backend, it will store unlimited metrics (limited by the backend policies)
Addons: Kubedash
Exposes a web frontend for Heapster metrics
It can be used as a very basic monitoring tool for Kubernetes clusters
Limited functionality, but very easy to install and operate, so it can be very convenient sometimes
Addons: SkyDNS
Kubernetes uses DNS for service discovery
As each application deployed in the cluster will have its own IP, Kubernetes provides a way to resolve service names to Ips
SkyDNS is a Google implementation of the DNS protocol in Go with some improvements
DNS entries can be managed using a REST API
The DNS entries are stored in etcd, making SkyDNS a cloud-ready DNS server
Kubernetes Objects
Objects: Pod
The most basic unit of computation in Kubernetes is a Pod
A Pod can contain one or more Docker containers, but for simplification, we will only store one container in one Pod
Each Pod definition passed to the Kubelet creates, at least, two Docker containers: the user container and a special Pod container that handles the networking
A Pod has a SDN assigned IP, and thus it is only accessible from the same node
Objects: Service
Defines a “ClusterIP” so a Pod can be reached from each node of the cluster
Every replica of the same Pod share the same Service, which acts as Load Balancer
A Service is not an Nginx or an HAProxy, it does not consume resources nor it is deployed to a node. It is a kube-proxy configuration
Depending on the IaaS, a Service can aquire an external IP
Objects: Ingress
Exposes a Service with a network wide URL so it can be accessed from the outside world
Provides a much more safer and manageable way of accessing services than directly exposing IPs
The Ingress endpoint is provided by the Ingress Controller Addon
Objects: ReplicationController
Ensures that a specified number of pod “replicas” are running at any one time
If there are too many pods, it will kill some. If there are too few, the replication controller will start more
You can think of a replication controller as something similar to a process supervisor, but rather than individual processes on a single node, the replication controller supervises multiple pods across multiple nodes
Objects: ReplicaSet
It is the next-gen ReplicationController, still in beta.
The biggest difference is that ReplicaSets do not support the rolling-update command
ReplicaSets can be used standalone, but their main usage is to be used by Deployments to orchestrate pod creation, deletion and updates
When you use Deployments you don’t have to worry about managing the Replica Sets that they create
Objects: Deployment
Provides declarative updates for ReplicaSet
It provides all the capabilities of a Replication Controller, but adds other powerful features
It adds the versioning feature: a Deployment is able to track the previously deployed versions and perform easy rollbacks
Pause and Resume
Update the Deployment to recreate the pods
Objects: DaemonSet
It is a special kind of ReplicationController that ensures one replica of a pod is running on each node of the cluster
You do not specify directly how many replicas does a DaemonSet deploys
As nodes are added to the cluster, pods are added to them. As nodes are removed from the cluster, those pods are garbage collected
Objects: Namespace
Every Kubernetes Object must be unique
This can be a nightmare as the cluster grows
To avoid this problem, each Object is created inside a Namespace, and its name only needs to be unique to that Namespace.
DNS Service Discovery takes in account the Service Name and the Namespace when resolving
Kubernetes Persistence
Persistence: Volume
A Kubernetes Volume is a temporal data storage that lives while the pod is alive
It persists through container restarts, but a pod restart will erase the information
It is meant to be shared between different containers of the same Pod
As we take the approach of having just one container for each Pod, these kind of volumes do not have any usage
Persistence: Persistent Volume
When containers need to store information in a persistent way, we use Persistent Volumes
A Persistent Volume is a piece of networked storage provisioned and made available to the cluster by an administrator
It is not meant to be created during a normal Kubernetes workflow
It is an abstraction of hardware resources (disk storage) so Pods can use it without knowing what underlying technology provides the storage
Persistence: Persistent Volume Claim
When a user of the cluster wants to request storage for his Pods, he creates a Persistent Volume Claim
The Claim object will automatically search the pooled and unused Persistent Volumes to find one that matches the request
Once a Persistent Volume has been claimed, its ownership cannot be changed until the Claim is removed from the cluster
Kubernetes CLI
CLI: Frequent Commands
kubectl get namespace
kubectl get pods –namespace default
kubectl describe pod <podname>
kubectl logs <podname>
kubectl exec –it <podname> bash
kubectl create –f <filename.yml>
CLI: Frequent Commands
kubectl rolling-update dms-appointments --image=registry.pre.kube.volkswagengroup.es/everis/dms-appointments --image-pull-policy Always --namespace everis-dev
Kubectl
DEMO
KUBERNETES ADVANCED
Advanced: Secret
It is meant to hold sensitive information, such as password, in an encrypted way
Putting secret info in a Secret is safer thant putting it verbatim in a Pod definition or a Docker image
Secrets are used by Pods by mounting them in a container Volume
Advanced: ConfigMap
It is a standard way of storing generic configuration as a Kubernetes object
It is very similar to a Secret, but to work with string that do not contain sensitive information
It can be thought of a HashMap for Kubernetes.
Advanced: Horizontal Pod Autoscaler
It can automatically scale the number of Pods in a ReplicationController, Deployment or ReplicaSet based on observed CPU utilization
The user defines an autoscaling rule referencing CPU: Scale when the Pod is at 80% CPU for 2 minutes with an upper limit of 10 replicas
Then, the autoscaler polls the CPU metric and scale up or down based on that rule
Its functionality is very limited
Advanced: Resource Limits
When created without limits, a container inside a Pod can potentially demand all the node’s resources
As not all the containers peak at the same time, this beahivour is sometimes wonderful, as it cut down infrastructure costs
But for the moments we need hard limits, we can establish limits to both a Pod or a Namespace
Advanced: REST API
As stated before, the only interface the Kubernetes components expose to the world and between them, is an HTTPS one
Thus, everything can be achieved accessing directly the REST API exposed by the apiserver
An extensive API documentation can be found in the Kubernetes documentation page
Advanced: Downward API
Allows containers to consume information about themselves or the system and expose that information how they want it, without necessarily coupling to the Kubernetes client or REST API
It is a way to declarative use the Kubernetes API while writing YAML files
Examples of common information retrieved with Downward API are the Pod’s IP or its memory and CPU limits
Q&AQuestions and Answers
@compilemymind