net1523be integrating nsx and cloud foundry · pdf filemicroservices release #1 microservices...

INTEGRATING NSX AND CLOUD FOUNDRY

Usha RamachandranStaff Product Manager, Pivotal

Sai ChaitanyaProduct Line Manager, VMware

NET1523BE

#VMworld #NET1523BE

VMworld 2017 Content: Not fo

r publication or distri

bution

2

AGENDA

IntroductionPivotal Cloud

Foundry

NSX-V integration

with Cloud

Foundry

New Features in

Cloud Foundry

Networking

NSX-T with Cloud

Foundry

Networking

1 2 3 4 5

#NET1523BE



bution

3

Cloud Native Model for Application Delivery

Contin

uous Delivery

An idea in the morning can ship by evening

Microservices

Release #1

Microservices

Release #2

#NET1523BE



bution

4

Customer Personas and Needs

Application Developer

DEVELOPER

Create applications to

meet business goals

Different applications types

• Micro-services

• Clustering Apps

• Latency-sensitive or secure services

Focus on business logic

• Tools and frameworks for easy development

• Write once, run anywhere

Speed and Agility

• Self-Service – no tickets!

• Minimal impact during upgrades

#NET1523BE



bution

5

Customer Personas and Needs

Platform Operator

Security

• Network Security

• Authorization and Authentication

• Platform Security

Platform Stability

• Day-2 operations

• Faster patching and upgrades

Visibility

• Billing and auditing

• Triage and debugging

OPERATOR

Keep the platform

running smoothly

#NET1523BE



bution

6

PIVOTAL CLOUD FOUNDRY

#NET1523BE



bution

7

Operating

System

Cloud API

Container Orchestration

Multiple

Languages

Microservices

Support

Services

Marketplace

DEVELOPMENT

NativeUser

Provided Partner

App Deployment

& Management

Availability

Visibility &

Administration

CI/CD Tools,

ID, Security

Health, Metrics,

Patching

Apps & Platform

Dashboards

OPERATIONS



bution

8

PCF Technical Primer

Cloud

ControllerDiego

cell

cell_1

Go

Router

Simplified view

1 Deploy app

2Uploads app and invokes scheduler

4

App scheduled to a container host

6

CF Services for

persistent storage

3

CF app instance ( Container) – stateless, aka state persisted externally

5 Register route:

app_a.cfapps.cloud.com

cell_1_ip : port_num

Go

RouterApp access

Pivotal Ops Manager and Ops Manager Director are used to install, maintain and upgrade PCF

cf push app_a

N

A

T

Load Balancer

*.cfapps.cloud.com

GoRouters

7



bution

9

Network Security in Cloud Foundry

PCI - Space

PCF Prod

Non PCI - Space

ASGs

Collection of egress allow rules that specify {IP CIDR,

Port, Protocol} that an app can access

Applied to entire foundation or at CF space level

PCF Services -

PCI Net

Challenges

Cannot specify policy at app granularity

PCI and non PCI containers can share some container host

Apps cannot be identified by IP or Subnet to apply ingress security

Source Destination Port and

Proto

Action

Any PCI Services tcp, 3306 Allow

Any Any any Deny

PCF Services –

non PCI Net

Using CF Application Security Groups



bution

10

NSX-V AND Pivotal Cloud Foundry

#NET1523BE



bution

11

PCF Infra Networking and Load Balancing requirements

PCF Deployment Network - 192.168.20.0/22

cellcell

2

Other External Services - 192.168.24.0/224

Ops Man

Director brainbrain

Go

Router

Go

Router

PCF Infra Network - 192.168.10.0/261

Ops

ManagerCC

Four Private Networks

PCF Services Network - 192.168.28.0/223

#NET1523BE



bution

12

PCF and NSX-V Logical Networking & Load Balancing

VPN

NSX LS Infra - 192.168.10.0/26

NSX LS Deployment - 192.168.20.0/22NSX LS Services

- 192.168.28.0/22

NSX LS External Services

- 192.168.24.0/22

Go

Routerbrain Ops

Manager

External Network – 10.114.214.0/24

Service Source Destination

Source NAT 192.168.10.0/16 External IP 1

Dest NAT External IP 2 Ops Man IP

NSX LB can either terminate

SSL or be configured as pass-

through (Go router terminates

SSL)

Service VIP Pool

Load Balancing External IP 3 Go Router IPs

Load Balancing External IP 4 Diego Brain IPsNSX ESG

Basic Routing Design

Design Guide – coming soon !

With Advanced Routing Designs & more #NET1523BE



bution

13

PCF Infrastructure Security Requirements

VPN

NSX LS Infra - 192.168.10.0/26

NSX LS Deployment - 192.168.20.0/22

NSX LS Ext Services

- 192.168.24.0/22

Go

Routerbrain Ops

Manager

External Network – 10.114.214.0/24

NSX ESG

Source Destin Service Actio

n

Any Ops_Manager SSH, HTTP,

HTTPS

Allow

any VIP_Go_Router HTTP,

HTTPS

Allow

… …… …… Allow

… …… …… Allow

Any Any Deny Deny

http://docs.pivotal.io/pivotalcf/1-

11/refarch/vsphere/vsphere_nsx_cookbook.html#l

oad_balancer

ESG Firewall to protect the PCF foundation

NSX LS Services - 192.168.28.0/22 #NET1523BE



bution

14

Cloud Foundry Isolation Segments

bbs

Diego

brain

cell

cell_1

brain

cellcell_1

PCI Isolation

SegmentNon PCI Isolation

Segment

Isolation Segments

Dedicated set of diego cells to enable compute

isolation of apps

Can be assigned to CF org or space

Apps (and instances) in org or space will only be

scheduled to their own dedicated cells

Benefits

Apps of different kinds can be deployed with

compute isolation on shared foundation – e.g.

PCI and non-PCI, Retail Banking and Investment

Banking etc

Save operational and cost overhead of

maintaining multiple foundations



bution

15

PCF Isolation Segments and NSX-V

Ops Manager and NSX integration for CF Isolation Segments

Deploy Isolation Segment

Ops Manager

deploys

dedicated Diego

cells for IS

Ops Manager

adds Diego cells

to NSX-V SG

If SG with same name as

Isolation segment, exists

VMs are added to SG

If SG with name of

Isolation Segment is not

found, create SG and

adds VMs

As Diego Cells are added / deleted, NSX SG membership is maintained#NET1523BE



bution

16

PCF Isolation Segments and NSX-V

Compute Isolation and Network Segmentation

Create NSX SG for PCI & Non-PCI

Create Segmentation Policy

Create Isolation Segments

Assign to Space or Org

Deploy app

Source Destin Service Action

SG_PCI PCI_Services HTTP, HTTPS Allow

SG_non_PCI Non_PCI_Services HTTP, HTTPS Allow

SG_PCI and

SG_non_PCI

Shared Services …… Allow

Any Any Deny Deny

DFW segmentation policy

cell_1 cell_1cell_n cell_n

Isolation Segment : PCIIsolation Segment :

Non-PCI

NSX SG - PCI NSX SG – Non-PCI

Stateful Network Segmentation & Monitoring at the Org / Space granularity



bution

17#NET1523BE



bution

18

NEW FEATURES IN CLOUD FOUNDRY NETWORKING

#NET1523BE



bution

19

LEGACY CLOUD FOUNDRY NETWORKING

#NET1523BE



bution

20

DESIRED STATE

#NET1523BE



bution

21

PCF 1.11 Networking Features

Policies

App to App

Dynamic

CLI or API

Self Service

c2c Connectivity

CNI

Silk CNI plugin

Unique IP on

overlay

3rd party plugins

Existing Features

Application

Security Groups

Egress Cell

IP:SNAT

Ingress Cell

IP:DNAT

#NET1523BE



bution

22

Container Networking Interface (CNI) is an

industry standard API for container runtimes

to call third party networking plugins

#NET1523BE



bution

23

PCF 1.11 Networking

cell

2 PCF Deployment Network - 192.168.20.0/22

cell

2 PCF Container Network – 10.255.0.0/1610.255.10.0/24

Single Overlay network for all containers in a single foundation

Defaults to a /16 range to allow for ~250 cells with ~250 containers per cell

Access to external services and through GoRouter continue to use the PCF Deployment Network

10.255.11.0/24

#NET1523BE



bution

24

PCF 1.11 Policy

APP 1 APP 2

Container Network

Deployment Network

Cell

APP 3

Cell

cf add-network-policy APP1 -> APP 2

Policy

Ingress

traffic

Egress

traffic

Cell

#NET1523BE



bution

25

$ cf add-network-policy SOURCE_APP --destination-app DESTINATION_APP [(--protocol

(tcp | udp) --port RANGE)]

POLICY CONFIGURATION

Allow two apps to talk to each other

List policies

$ cf network-policies [--source SOURCE_APP]

Revoke the policy for two apps to talk to each other

$ cf remove-network-policy SOURCE_APP --destination-app DESTINATION_APP --

protocol (tcp | udp) --port RANGE

#NET1523BE



bution

26

USE CASES

APP 1APP 1frontend

billingbilling

billing

CheckoutCheckout

Auth

inventoryinventory

inventoryinventory

Secure Microservices

Direct east-west communication

Private microservices do not need public routes

Fine-grained application level policies

boot

peer

peerClustering Applications

Same source and destination in policy

Communicate on an TCP or UDP port

#NET1523BE



bution

27

Demo

#NET1523BE



bution

28

NSX-T CONTAINER NETWORKING FOR PCF

#NET1523BE



bution

29

NSX-T & PCF

Network & Security platform for cloud native & traditional apps

CNI Integration with Cloud Foundry

Common operational model for

traditional and cloud native

Integrated with data center network,

tools & processes

Native “Container” Networking & Security

Leverage existing investmentsPhysical Network

& Security

NSX Network & Security



bution

30

NSX-T CONTAINER NETWORKING

Container Network integrated with Data

Center Network with routing (BGP)

Automated creation / deletion of

container network – in response to CF

Org create / delete

Two modes – routed & private container network

PCF Foundation 1

Network Mode : Routed

172.20.1.0/24 172.20.2.0/24

10.4.0.128/27

Org 1

SNAT IP

172.19.0.6

172.20.0.0/27

Org 1

Conserve IP address space in core

DC network

Maintain isolation between core

network & container network

Private Container Network

App identified using SNAT IP address

in core network

PCF Foundation 2

Network Mode : Private



bution

31

NSX-T & PCF SECURITY

Cloud Native App Platform –

Instance 1

Namespace

shopping_cart

Namespace

notifications

Cloud Native App Platform –

Instance n

Namespace

payments

Namespace

auth

Apps & Databases

1Inter Microservice – same cloud

native platform instance

2Inter Microservice – multiple

instances of CNA platform/s

3Microservice to VM or Database

app

1 23

Use Cases

Configuration approaches

1 CF Network Policy

2 NSX APIs – DFW, Section



bution

32

MONITORING FOR CLOUD NATIVE APPS

Send / Receive stats for Unicast, Bcast/Mcast

and Dropped traffic

Traffic Mirroring

Rule statistics – packets, bytes, sessions

Syslog

NSX Traceflow

NSX Search enables co-relating app and

infrastructure instantaneously enabling efficient

incident response

Simulate app traffic between containers and / or

VMs and identify failure points

Container Cluster and App context in NSX



bution

33

NSX-T & CLOUD NATIVE APPS

NSX-T 2.0

Native Container

Networking

Microsegmentation

for Containers

Load Balancing

Monitoring &

Troubleshooting

Containers

Integration with

existing tools &

processes

Reference Designs

Provision & manage network like cloud native apps



bution

34

SUMMARY

Cloud Foundry and NSX together provide the agility and security required for

digital transformation

NSX-V with CF isolation segments provides stateful network segmentation at the

org/space level

Cloud Foundry has a secure and extensible networking stack that enables direct

container communication based on app level policies

NSX-T and Cloud Foundry CNI integration provides native container networking and

security, and a common operational model across cloud native and traditional apps

Cloud Foundry CNI enables third party SDN integrationVMworld 2017 Content: Not fo


bution



bution

36

Questions?



bution



bution

net1523be integrating nsx and cloud foundry · pdf filemicroservices release #1 microservices...

Documents