mirantis contributions to kubernetes ecosystem

Copyright © 2017 Mirantis, Inc. All rights reserved

Contributions to Kubernetes Ecosystem

Moscow Kubernetes Meetup15.02.2017

2017 | www.mirantis.com

● Why we’re getting involved in Kubernetes community?○ Containers and Kubernetes are the future○ Need to improve the state of Kubernetes on Bare Metal○ Our customers want Kubernetes○ Open contribution process

■ Much more meritocratic approach then in case of Docker Swarm or Mesos

● It started with Kubernetes on OpenStack via Murano

The beginning of the story


● Fixing bugs, writing tests● Participating in SIG and Community meetings● Helping with project management● Trying to add features we need

Getting started with participating in k8s community


● Lack of reviewers● Complicated discussions about new functionality

○ DaemonSet upgrades and anti-affinity○ Multiple runtimes in Kubelet○ Node evacuation / maintenance (partially in)○ ConfigMap templates (rejected)

● It’s hard to land XXL patches in general● The upstream wants to keep the core small

○ They want Kubernetes to be “the Linux kernel of distributed systems”

Problems with working upstream


● Third Party Resources● Container Runtime Interface● External controllers that connect to apiserver● Provide external tooling around kubernetes,

including one used for cluster setup

Extending Kubernetes without changing the core


● k8s-AppController: managing complex deployments● k8s-externalipcontroller: external IP support for bare

metal k8s clusters● Kargo: setting up a Kubernetes cluster

○ It was not Mirantis project in the beginning, but as of now it’s mostly ours

● Virtlet: running VM workloads on Kubernetes clusters

Mirantis projects belonging to k8s ecosystem


● kubeadm-dind-cluster: running multinode development clusters locally using Docker-in-Docker

● We didn’t stop being active in k8s core○ we continue fixing bugs○ also adding new features, e.g. for example, we’re working on

DaemonSet upgrades● We lead SIG On-Prem, SIG-OpenStack and SIG-PM● Mirantis is a member of CNCF

Mirantis projects belonging to k8s ecosystem


● The very initial purpose of k8s was mostly running “cattle” workloads

● Support for stateful workloads is improving over time○ For instance, there’s support for PVs, StatefulSets, init

containers etc.● Defining dependencies between Kubernetes objects

is hard○ E.g. a web app pod may need to wait for its database to

become ready. This may be a problem for legacy apps

AppController: managing complex deployments



Kubernetes cluster

AppController pod

ThirdPartyResources

Resource Definitions Dependencies

Kubernetes Objects

Kubectl (operator)

Creates

Extends API

CreatesCreates

Creates when

dependencies are met

Reads

Starts

Retrieves status


● AppController represents k8s objects and their dependencies as TPRs

● k8s objects are created when their dependencies are satisfied○ E.g. pod can depend on a service or a job that needs to be

complete before the pod is created○ Objects can depend on objects that are created by

AppController or pre-existing k8s objects● Helm integration is WiP● Application Lifecycle Management (planned)



● Ansible-based Kubernetes installer● Supports AWS, GCE, Azure, OpenStack and BM● Supports HA● Flexible deployment options

○ A possibility to choose network plugin, load balancer, rkt support for core services etc.

● Support most popular Linux distributions● The project has extensive CI setup● Battle-tested on Scale Lab (up to 1000 nodes)● There are kubeadm integration plans

Kargo: setup a Kubernetes cluster


● Some legacy applications can’t be easily containerized○ Substantial effort may be necessary for the transition, there

can be licensing problems, there’s need for extra isolation and so on

● Virtlet runs VMs as Pods, supports QCOW2 images● VMs can communicate with other Pods and access

cluster services

Virtlet: running VM workloads on k8s clusters


● The implementation is based on Container Runtime Interface○ CRI is also being used by cri-o, hyper, rkt○ It will be also be used for Docker in Kubernetes soon

● We’re making it easy to install Virtlet on k8s clusters○ It can run as a DaemonSet and still avoid chicken-and-egg

problem thanks to multiple runtime support provided by CRI Proxy

● “Futuristic” use case: running Unikernel applications on Kubernetes cluster



● We will be speaking about more of our projects on following meetups

● Questions?

Thanks for your attention!

mirantis ￼contributions to kubernetes ecosystem

Software

mirantis contributions to kubernetes ecosystem