mesos and kubernetes ecosystem overview
TRANSCRIPT
HUAWEI TECHNOLOGIES CO., LTD.
www.huawei.comEco System Overview
Mesos & CNCF Meetup @Huawei, Bangalore, 6th August 2016Compiled by Krishna Kumar, CNCF Ambassador, Lead Architect & Delivery Head Huawei India Cloud/PaaS
HUAWEI TECHNOLOGIES CO., LTD. 2
Contents
1.Overview2.Architecture3.Open Source4.Customers & Partners5.Competitors
*Apache Mesos & Mesosphere are interchangeably referred in several places.
HUAWEI TECHNOLOGIES CO., LTD. 4
History of Apache Mesos Mesos began as a research project in the UC Berkeley RAD Lab by then PhD students Benjamin Hindman, Andy
Konwinski, and Matei Zaharia, as well as professor Ion Stoicayear 2009. It was originally named Nexus but due to a conflict with another university's project, was renamed to Mesos. Become top level Apache project from July 2013 https://www.usenix.org/legacy/events/nsdi11/tech/slides/hindman.pdf
HUAWEI TECHNOLOGIES CO., LTD. 6
HUAWEI TECHNOLOGIES CO., LTD. 8
History of Kubernetes Project Started at Google who have lots of
experience with their Borg system and "The Datacenter as a Computer" collection of papers
Kubernetes (from κυβερνήτης: Greek for "helmsman" or "pilot") was founded by Joe Beda, Brendan Burns and Craig McLuckie and first announced by Google in 2014.
Its development and design are heavily influenced by borg and many of the top contributors to the project previously worked on Borg.
Kubernetes v1.0 was released on July 21, 2015.
Along with the Kubernetes v1.0 release, Google partnered with the Linux Foundation to form the Cloud Native Computing Foundation (CNCF) and offered Kubernetes as a seed technology.
HUAWEI TECHNOLOGIES CO., LTD. 9
HUAWEI TECHNOLOGIES CO., LTD. 10
Contents
1.Overview2.Architecture3.Open Source4.Customers & Partners5.Competitors
*Apache Mesos & Mesosphere are interchangeably referred in several places.
HUAWEI TECHNOLOGIES CO., LTD. 11
HUAWEI TECHNOLOGIES CO., LTD. 12
HUAWEI TECHNOLOGIES CO., LTD. 13
Huawei Built this one!
HUAWEI TECHNOLOGIES CO., LTD. 14
Written in C++, Create a big resource pool from all resources.
Full control of the containers scheduled. Custom schedule can manage the containers, VMs and bare metal machines as desired.
Since Mesos enables you to run services such as web servers or application servers on the same cluster as Spark for analytics purposes, it increases the overall cluster utilization and accommodates the effects of data gravity. Best in the class resource scheduler in the world!Project Features• Scalability to 10,000s of nodes• Fault-tolerant replicated master and slaves using ZooKeeper• Support for Docker containers• Native isolation between tasks with Linux Containers• Multi-resource scheduling (memory, CPU, disk, and ports)• Java, Python and C++ APIs for developing new parallel applications• Web UI for viewing cluster state
HUAWEI TECHNOLOGIES CO., LTD. 15
Kubernetes architectureA running Kubernetes cluster contains node agents (kubelet) and master components (APIs, scheduler, etc), on top of a distributed storage solution.
The scheduler is configurable and at compile time which fit predicates and priority functions you want Kubernetes to apply. It has two types of policies, FitPredicate - The sum of the requested resources of the container(s) already running on the machine plus the requested resources of the new container(s) you are considering scheduling onto the machine must not be greater than the capacity of the machine& PriorityFunction - The scheduler may find that multiple machines "fit" the pod and it prefers the machine whose already-running pod consume the least resources.
Scheduling is a policy-rich, topology-aware, workload-specific function that significantly impacts availability, performance, and capacity. The scheduler needs to take into account individual and collective resource requirements, quality of service requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, deadlines, and so on. Workload-specific requirements will be exposed through the API as necessary.
HUAWEI TECHNOLOGIES CO., LTD. 16
KubernetesWritten in Golang(GO), works well with CoreOS, Red Hat Atomic. Easy to run cluster resources.Kubernetes will do the containers in Pods – a group of containers scheduled on the same host - supports composable micro services.You can deploy any services you like into Kubernetes. There are some services available now with built-in Kubernetes support. Cassandra and Spark are some of them. This makes better utilization, reduce complexity and service discovery much easier.Project Features• Primarily targeted at applications composed of multiple
containers, • such as elastic, distributed micro-services. • Support multiple platforms• Easy to get up and running & portable• Manages Docker deployment well• Scheduling provides out of the box• Monitoring Dashboard
HUAWEI TECHNOLOGIES CO., LTD. 17
Mesos Kubernetes1. Master – Enable sharing of resource2. Slave – Execute the task in it3. Cluster – a group of machines4. ZooKeeper – distributed
synchroniztion/configuration5. Framework – scheduler + executor6. Scheduler – Offer resources7. Executor – Run the Framework task8. Task – a job to run9. Containerizer – run & monitor executors
1. Master – Cluster controlling unit2. etcd – HA Key/value store3. API Server - Observing the state of the cluster4. Controller Manager – runs multiple controllers5. Scheduler Server – assigns workloads to
nodes6. Minion – server that perform work7. Kubelet - server/slave node that runs pods8. Proxy Service – host subnetting to external
parties9. Pods – One or more containers10.Services – load balancer for containers11.Replication Controller – For horizontally-scaled
pods12.Labels – A tag for work unit for grouping
More detailed explanations of the components: https://github.com/mesosphere/kubernetes-mesos/issues/285
The Components in the Architecture System
HUAWEI TECHNOLOGIES CO., LTD. 18
The new HTTP API aims to solve these problems by providing an RPC-based HTTP API for both frameworks (i.e.schedulers and executors) and operators.
With the unified containerizer, we can now use a single container runtime engine, the MesosContainerizer to run containers off the most popular image formats (Docker, Appc, and soon OCI) without dependency on any external daemons.
Now, each container gets its own network namespace, which provides network isolation and allows users to work with containers as if they were running on bare metal or VM. One of the highlights of the new CNI support is the ability to now provide a single IP per container in Mesos.
Docker volume plugin API, Mesos containers can connect with external volumes from numerous storage providers (e.g., Amazon EBS, Ceph, EMC ScaleIO).
Added foundations for multi-tenancy by adding fine-grained authorization controls. Now possible to set up ACLs so that a user can only view information about her own tasks in the WebUI and/or HTTP endpoints.
You can now run your TensorFlow jobs directly on Mesos. The custom GPU isolator handles all of the details of allocating GPUs to your application and making sure they are isolated from other jobs running on the system.
Mesos 1.0 comes with experimental support for running Mesos Agent on Windows. It is now possible to build Mesos Agent and all its dependencies on Windows using CMake and Microsoft Visual Studio compiler.
http://mesos.apache.org/blog/mesos-1-0-0-released/
HUAWEI TECHNOLOGIES CO., LTD. 19
Increased scale and automation - Autoscale clusters up and down while doubling the maximum number of nodes per cluster.
Cross-cluster federated services - Customers want their services to span one or more (possibly remote) clusters, and for them to be reachable in a consistent manner from both within and outside their clusters, introduces cross-cluster service discovery.
Stateful applications - containers for stateful workloads (such as databases or key value stores) will find a new ‘PetSet’ object
Ease of use for local development - we are introducing Minikube, where with one command a developer can start a local Kubernetes cluster on their laptop that is API compatible with a full Kubernetes cluster.
Support for rkt and container standards OCI & CNI - Container Network Interface (CNI) natively, and have already taken steps to the Open Container Initiative (OCI), which is still being ratified. We are also introducing rkt as an alternative container runtime in Kubernetes node.
Updated Kubernetes dashboard UI - Customers can now use the Kubernetes open source dashboard for the majority of interactions with their clusters, rather than having to use the CLI. The updated UI lets users control, edit and create all workload resources.
http://blog.kubernetes.io/2016/07/kubernetes-1.3-bridging-cloud-native-and-enterprise-workloads.html
HUAWEI TECHNOLOGIES CO., LTD. 20
Contents
1.Overview2.Architecture3.Open Source4.Customers & Partners5.Competitors
*Apache Mesos & Mesosphere are interchangeably referred in several places.
HUAWEI TECHNOLOGIES CO., LTD. 21
Top contributors
Code Commits
HUAWEI TECHNOLOGIES CO., LTD. 22
HUAWEI TECHNOLOGIES CO., LTD. 23
Code Commits
Top contributors
HUAWEI TECHNOLOGIES CO., LTD. 24
Contents
1.Overview2.Architecture3.Open Source4.Customers & Partners5.Competitors
*Apache Mesos & Mesosphere are interchangeably referred in several places.
HUAWEI TECHNOLOGIES CO., LTD. 25http://mesos.apache.org/documentation/latest/powered-by-mesos/
HUAWEI TECHNOLOGIES CO., LTD. 26
HUAWEI TECHNOLOGIES CO., LTD. 27
HUAWEI TECHNOLOGIES CO., LTD. 28
http://techblog.netflix.com/2016/07/distributed-resource-scheduling-with.html
HUAWEI TECHNOLOGIES CO., LTD. 29
http://www.ebaytechblog.com/2014/05/12/delivering-ebays-ci-solution-with-apache-mesos-part-ii/
HUAWEI TECHNOLOGIES CO., LTD. 30
https://www.linux.com/news/how-verizon-labs-built-600-node-bare-metal-mesos-cluster-two-weeks
HUAWEI TECHNOLOGIES CO., LTD. 31
Mantl is a modern, batteries included platform for rapidly deploying globally distributed services by Cisco
Kubernetes Mesos
HUAWEI TECHNOLOGIES CO., LTD. 32
HUAWEI TECHNOLOGIES CO., LTD. 33
“We are currently witnessing the transition from Container 1.0 to Container 2.0, which brings with it the ability to efficiently host microservices, stateful databases, storage and more inside containers on shared infrastructure. DC/OS supports these capabilities today,” said Florian Leibert, Co-founder and CEO, Mesosphere.http://www.datastax.com/2016/08/datastax-and-mesosphere-lead-databases-into-container-2-0-era-with-datastax-enterprise-on-dcos
The general availability of Confluent, the commercial Kafka distribution, and LightBend, an open source application development environment on DC/OS are a step in that direction. Last month saw the availability of Kubernetes 1.3, which introduced the concept of PetSet, the ability to run stateful workloads within the same cluster.Microsoft Azure Container Service is a managed CaaS platform powered by DC/OS. http://www.forbes.com/sites/janakirammsv/2016/08/01/mesosphere-wants-to-move-enterprises-to-the-container-2-0-era/2/#48cfa371c7d3
https://mesosphere.com/blog/2016/07/20/serverless-computing-dcos-galactic-fog/?platform=hootsuite
HUAWEI TECHNOLOGIES CO., LTD. 34
HUAWEI TECHNOLOGIES CO., LTD. 35
HUAWEI TECHNOLOGIES CO., LTD. 36
Contents
1.Overview2.Architecture3.Open Source4.Customers & Partners5.Competitors
*Apache Mesos & Mesosphere are interchangeably referred in several places.
HUAWEI TECHNOLOGIES CO., LTD. 37
The Competitors
HUAWEI TECHNOLOGIES CO., LTD. 38
http://www.nextplatform.com/2015/08/13/will-openstack-kubernetes-or-mesos-control-future-clusters/
This integration between the two worlds is being done through an effort called Magnum, which we told you about back in May and which will provide hooks between OpenStack and Google’s Kubernetes, Docker Swarm, CoreOS Tectonic, and perhaps other container management systems as they arise.
Myriad – A open source software project is both a Mesos framework and a YARN scheduler that enables Mesos to manage YARN resource requests.
HUAWEI TECHNOLOGIES CO., LTD. 39
A leading cloud expert in the world
Container World coming together…
1. Mesosphere – Kubernetes-Mesos - https://github.com/mesosphere/kubernetes-mesos2. Mesos and Kubernetes at scale - https://www.youtube.com/watch?v=pFCJ3WiVjyA3. Mesos vs. Kubernetes - http://
stackoverflow.com/questions/26705201/whats-the-difference-between-apaches-mesos-and-googles-kubernetes4. Kubernetes – Marathon - https://
www.quora.com/What-is-the-difference-between-Googles-Kubernetes-and-Mesospheres-Marathon5. Mesos - Kubernetes PaaS - http://www.slideshare.net/MiguelZuniga1/platform-as-a-service-with-kubernetes-and-mesos6. Mesos, Docker, Kubernetes - http://www.slideshare.net/timothysc/apache-coneu7. Mesos vs. Kubernetes - http://thenewstack.io/a-brief-comparison-of-mesos-and-kubernetes/
HUAWEI TECHNOLOGIES CO., LTD. 40
HUAWEI TECHNOLOGIES CO., LTD. 41
As container competition heats up………….. The Linux Foundation is definitely hoping for the meeting of minds through Cloud Native Computing Foundation!
Thank youwww.huawei.com
HUAWEI TECHNOLOGIES CO., LTD. 43
*All the materials are pulled out from web. Thanks to all for posting those