triangle k8s meetup 06092016 - nuage network

Copyright 2016 Nokia. All rights reserved.

Copyright 2013 Alcatel-Lucent. All rights reserved. CONFIDENTIAL - SOLELY FOR AUTHORIZED PERSONS HAVING A NEED TO KNOW

PROPRIETARY – USE PURSUANT TO COMPANY INSTRUCTION Nuage Networks

Overlay Networking for Openshift and Kubernetes with Nuage VCS Vishal Patil, Aniket Bhat


Agenda

• Kubernetes (K8s) Overview

• Nuage VCS Integration with OpenShift/K8s

• Demo


Kubernetes Architecture

Master

SCHEDULER

API PROXY

AUTH

REPLICATION CONTROLLER

Node

KUBELET KUBE PROXY

POD (SVC 2)

POD (SVC 2)

Node

KUBELET KUBE PROXY

POD (SVC 1)

POD (SVC 1)

CLIENT

C1 C1

C1 C2 C1 C2

ETCD


▪ etcd ▪ key value store that stores the persistent master state

▪ API Server ▪ Authenticated interface between users and Kubernetes

▪ Scheduler

▪ Schedules pods to nodes/minions

▪ Controller Manager

▪ Monitors replication controllers and adds/removes pods to maintain desired number of pods in a service

Master

Kubernetes Components

Master

SCHEDULER

API PROXY

AUTH

REPLICATION CONTROLLER

CLIENT

ETCD


▪ Docker

▪ Engine for containers

▪ Kubelet

▪ Manages Pods on the nodes

▪ Kube-Proxy

▪ Simple network proxy to implement ‘Services’

▪ Two modes supported - iptables based or user-space proxying

▪ TCP and UDP stream forwarding (round robin) across a set of backends

Node

Kubernetes Components

Node

KUBELET KUBE PROXY

POD (SVC 2)

POD (SVC 2)

C1 C2 C1 C2


Kubernetes Networking

▪ Pod gets an IP from a flat shared networking namespace. ▪ Containers within the pod share the same IP. ▪ Container in a pod use localhost networking. ▪ Pods should typically use service IPs and service ports to communicate with

other pods ▪ Kubernetes is non-prescriptive about the network design per se. ▪ Several vendor choices:

● Calico ● Flannel ● Nuage ● Open-Contrail ● Weave


JBoss xPaaS

VCS Integration with Kubernetes/Openshift

User Experience

Containerized Services

Container Orchestration

Container API

Container Host

Integration layer

Nuage VCS


Cloud Service Management Plane

Data Center Control Plane

Data Center Data Plane

Virtual Routing & Switching

Virtualized Services Directory

Virtualized Services Controller

HYPERVISOR

HYPERVISOR

HYPERVISOR

HYPERVISOR

HYPERVISOR

HYPERVISOR

Virtualized Services Directory (VSD) • Network Policy Engine – abstracts complexity • Service templates and analytics

Virtualized Services Controller (VSC) • SDN Controller, programs the network • Rich routing feature set

Virtual Routing & Switching (VRS) • Distributed switch / router – L2-4 rules • Integration of bare metal assets

Nuage Networks Virtualized Cloud Services Platform (VCS)

IP Fabric

Bare Metal Servers

Nuage Networks Virtualized Cloud Services

MP-BGP

NSG


Enterprise

Domain

Zone

Subnet

VPort


VCS OBJECTS AND OPENSHIFT/KUBERNETES CONCEPTS

Project/Namespace

nuage.io/policy-group Labels

Pods

Zone

Policy Groups

VPorts

Cluster Domain

VCS


Overlay-based Virtual Networks OPENSHIFT DEPLOYMENT WITH VCS

OpenShift Master Openshift Node Openshift Node

VSD

K A

PI

XMPP

VRS VRS nuage-openshift-monitor

vsp-openshift plugin


OpenShift Cluster

Virtualized Services

Controller



• Runs on master node(s)

• Exercises the VSD REST API to ensure that the VSD objects are created

• Create delete vsd zones Projects

• Create/delete network macros Services

• Dynamically scale-up or scale-down subnets

nuage-openshift-monitor

• Runs on each of the nodes

• Implementation of the k8s network exec plugin

• Gets invoked when a node is initialized as well as during pod lifecycle events:

• Create/delete pod

• Status hook that queries pod’s IP information



▪ Pod is created with a veth pair – is eventually mapped to a vPort

▪ Pod gets an IP allocated from the subnet pools for that Openshift Project/K8S Namespace (VCS Zone).

▪ Pods in a given zone belong to one or more subnets irrespective of which node they are spawned on

▪ Labels are optionally used to do the Security and QoS Policy resolution with the VSD

Pods with VCS

OPENSHIFT DEPLOYMENT WITH VCS


Dev Workflow OPENSHIFT DEPLOYMENT WITH VCS

OpenShift Master Node

VSD

K A

PI

XMPP

VRS nuage-openshift-monitor


OpenShift Cluster


Controller


1. Project creation triggers creation of VSD Zone and subnet

2. Node Plugin invoked during Pod creation

4. VSC gets network and security policy from VSD

5. VSC sends network and security policy to the VRS

3. VRS contacts VSC with Project name and metadata information


Operations Workflow OPENSHIFT DEPLOYMENT WITH VCS

OpenShift Master Node

VSD

K A

PI

XMPP

VRS nuage-openshift-monitor


OpenShift Cluster


Controller


1. User creates Domain/Zone and defines Network and Security Policies on VSD

2. Labels in a Pod configuration are used to pass metadata to VSD

3. Node Plugin invoked during Pod creation will fetch Labels from Pod configuration

5. VSC gets network and security policy from VSD

6. VSC sends network and security policy to the VRS

4. VRS contacts VSC with Project name and metadata information


Rich policy model

Use of labels as metadata

Dynamic scaling of subnets in

a given project.

Support for both

operations and

developer workflows.

Support for double

overlays

Support for asymmetric

overlays

HIGHLIGHTS


DEMO

▪ Scenario 1 – Developer Workflow ▪ Communication between an app with Front end (Mail service)

and a pod acting as a mail client

▪ Scenario 2 - Policies ▪ Inter-project communication


THANK YOU

triangle k8s meetup 06092016 - nuage network

Documents