Revolutionizing WSO2 PaaS with Kubernetes & App Factory

Imesh GunaratneSenior Technical Lead, WSO2

Dmitry SotnikovVice President - Cloud, WSO2

Agenda

• An Introduction to Linux Containers• An Introduction to Docker• An Introduction to Kubernetes• Deploying WSO2 products on Kubernetes

• Carbon cluster discovery on Kubernetes• Monitoring server health • Artifact distribution• Propagating software updates • Load balancing with Nginx• Demo

Agenda

• App Factory and WSO2 App Cloud• Creating applications and uploading existing apps• Hosting web applications and API backends• Using databases in the Cloud• Coding and debugging applications with Cloud IDE or

desktop tools• Application lifecycle management• Custom URLs• Teamwork and social development• Issue and task tracking

An Introduction to Linux Containers

Linux Containers

Linux Container Brief for IEEE WG P2302, Boden Russell

Kernel features used by Linux Containers● Namespaces● cgroups

○ Limit, account & Isolate resource usage (CPU, memory, disk, etc)

● chroot○ Isolate access to disk

● AppArmor● SELinux● seccomp

- mnt (mount points, filesystems)- pid (processes)- net (NICs, routing)- ipc (inter-process communication)- uts (hostname)- user (user ids)

- Secure computing/ application sandboxing

- Security modules for controlling access to resources based on security policies

Linux Container Brief for IEEE WG P2302, Boden Russell

LXC - A Hypervisor for Linux Containers

LXC Components

https://linuxcontainers.org/lxc/introduction

● The liblxc library● Language bindings for the API:

○ python3, python2○ lua○ Go○ ruby○ Haskell

● A set of standard tools to control the containers

● Distribution container templates

More Information on LXC

https://linuxcontainers.org

An Introduction to Docker

Virtual Machines Vs Docker

https://www.docker.com/what-docker

Introduction to Docker

● Started as an internal project within dotCloud

● Initial release in March 2013● Initially used libvirt, LXC &

systemd-nspawn● Introduced libcontainer: A

standard interface for Linux containers

https://en.wikipedia.org/wiki/Docker_(software)

Docker Ecosystem

Enterprise Docker, Adrien BLIND, Aurelien GABET, Arnaud MAZIN

DockerfileFROM debian:7.7MAINTAINER dev@wso2.org

ENV DEBIAN_FRONTEND noninteractiveENV JAVA_HOME /opt/jdk1.7.0_80

ADD packages/jdk-7u80-linux-x64.tar.gz /optCOPY packages/wso2esb-4.8.1.zip /opt/

WORKDIR /opt/RUN apt-get update && apt-get install -y zipRUN unzip /opt/wso2esb-4.8.1.zip

EXPOSE 9763 9443

ENTRYPOINT /opt/wso2esb-4.8.1/bin/wso2server.sh

Docker Image Structure

https://www.docker.com

An Introduction to Kubernetes

Why Container Clusters?

• Avoid single point of failure• Make horizontally scalable• Have more granular

management for distributed applications (microservices)

• Self healing systems

http://googlecloudplatform.blogspot.com/2015/01/what-makes-a-container-cluster.html[image ref] https://www.docker.com/what-docker

Kubernetes Architecture

Docker Host 1 Docker Host 2 Docker Host n

Physical Network

Controller Host

Kube

rnet

es A

PI

Overlay Network (SDN)Sc

hedu

ler

Kubernetes Component Architecture

Kubernetes Pods

● A group of containers which can share resources and context

● Shared namespaces:○ PID namespace (processes visible

globally)○ network namespace (same IP and port

space)○ IPC namespace (SystemV IPC or POSIX)○ UTS namespace (share a hostname)

Pod

C1 C2 C3

http://kubernetes.io/v1.0/docs/user-guide/pods.html

Kubernetes Labels & Selectors

● Labels are key/value pairs attached to objects

● Selectors use the label key to find a collection of objects matched with the same value○ L1 -> ClusterID = C1○ L2 -> ClusterID = C2

Pod 1

http://kubernetes.io/v1.0/docs/user-guide/labels.html

L1

Pod 2

L1

Pod 3

L1

Pod 3

L2

Pod 4

L2

Pod TemplateReplicas = n

Kubernetes Replication Controllers

● Provides features for replicating pods:○ Auto-healing○ Scaling○ Rolling updates○ Multiple release tracks

Pod 1

http://kubernetes.io/v1.0/docs/user-guide/replication-controller.html

Pod 2 Pod n

Replication Controller

Node IP: 172.17.8.102

Port: 9443

Domain Name: service1IP: 10.2.10.20

Port: 9443NodePort: 32001

Protocol: TCP

Kubernetes Services

● Nginx based load balancing rules● Service types:

○ ClusterIP (default)○ NodePort○ LoadBalancer

● Service discovery:○ DNS○ Environment variables

● Session Affinity

Pod 1

http://kubernetes.io/v1.0/docs/user-guide/services.html

Pod 2 Pod n

Service

L1

L1 L1 L1

Node

● A distributed key value store ● Read and write values with curl

and other HTTP libraries● Store data in directories, similar

to a file system● Watch a key or directory for

changes and react to the new values

https://coreos.com/etcd/

etcd

● A software defined network● Runs an agent, flanneld, on each

host● Uses etcd to store the network

configuration, allocated subnets, and auxiliary data (host ips)

https://github.com/coreos/flannel

flannel

Deploying WSO2 products on Kubernetes

Carbon Cluster Discovery on Kubernetes

Pod 1 Pod 2 Pod n

Service

Server 1 Server 2 Server n

Pod 1

Service

Server 1

Pod 1 Pod 2

Service

Server 1 Server 2

Hazelcast member initialization

Hazelcast member initialization

Hazelcast member initialization

1 2 n

WSO2 Carbon Reference Architecture for Kubernetes

Pod 1 Pod 2

Manager Cluster Services

Pod 3 Pod 4 Pod n

Worker Cluster Services

Manager Replication Controller

Worker Replication Controller

Client

Gov Reg

Conf Reg

User Store

WSO2 Configurator

A python module written using Jinja2 template engine for configuring a product using a set of key/value pairs.

4

1 2

3

Deployment Workflow

Build Template Module

Kubernetes Membership

Scheme

Build Docker Image

Import DI to Kubernetes Nodes

Application Artifacts

Template Module

Product Distribution

Docker Image (DI)

Configurator

JDK

Kernel & Security Patches

Templates

Deploy Kubernetes ArtifactsKubernetes Artifacts

3

1 2

Artifact Distribution/Software Updates

Build Docker Image

Import DI to Kubernetes Nodes

Application Artifacts

Template Module

Product Distribution

Docker Image (DI)

ConfiguratorJDK Execute Rolling Update

Replication Controller

Monitoring Kubernetes Cluster Health

Monitoring Kubernetes Node/Pod Health

cAdvisor

Monitoring Kubernetes Cluster/Node/Pod Health

cAdvisor

Node 1

cAdvisor

Heapster

Node 2 Node n

Grafana

InfluxDB

Demo: Deploying WSO2 ESB on Kubernetes

WSO2 App Factory & App Cloud

WSO2 Platform

Application PaaS from WSO2

WSO2 Platform Deployment Options

Demo Time!

• App Factory and WSO2 App Cloud• Creating applications and uploading existing apps• Hosting web applications and API backends• Using databases in the Cloud• Coding and debugging applications with Cloud IDE or

desktop tools• Application lifecycle management• Custom URLs• Teamwork and social development• Issue and task tracking

Announcing: ESB in App Cloud / App Factory

Demo Time!

• Create ESB project• Create Properties

pointing to endpoints• Move through lifecycle

and observe change in endpoints invoked

Questions

Thank You