running cloud foundry at swisscom...cloud infrastructure cloud foundry (paas) services (3rd party)...

Running Cloud Foundry at Swisscom

CF Summit Europe 2015, November 3th

Boban GlisovicCloud Engineer, PaaS Core

Diego ZamboniCloud Architect, Health Management

CF Summit Berlin 2015 (c) Swisscom 2015

About Swisscom

Everythingalways on

Internet-based

Globalcompetition

• Telecommunication, IT Services and more within Switzerland

• Fixed network: By the end of 2015, Swisscom will supply 2.3 million homes and businesses with ultra-fast broadband.

• Mobile network: By 2016, Swisscomwill have extended 4G/LTEcoverage to 99% of theSwiss population.

• > 20’000 Employees

Swisscom’sunified approach to cloud infrastructure & services

IaaS SaaS PaaS

IT Architect EndUser Application Developer

Enterprise Cloud Service Cloud Application Cloud

same

infra-structure

identical

tools

same

services

Building a 360°CloudUsed by everybody: For consuming services, building services, building platforms

Open Standards à no Lock-in

Scalable – based on requirements, time and budget

Simple: a standardized platform



ProductApplication Cloud

Virtual PrivatePublic

Product

Cloud Infrastructure

Cloud Foundry (PaaS)

Services (3rd Party)

Self Service Portal

Support

Business Model

Support

Professional Services Optional

Pay-per-use

Contract base

(Minimum Consumption)

Legacy-Integrations (IdP etc.)

Shared

Shared

Dedicated Network (VXLAN)

Shared

Dedicated

Standard, 24/7

High Availability (96.0-99.9% )

• Base fee• At least 3 months• Setup fee included

Services (Database etc.) Shared and Dedicated Dedicated

Optional

Community & Std

Secure Datacenter Interconnection (VPN/LAN-I) Optional

• No base fee• At least 1 day

Launch: 2nd of Oct 15

Cloud Foundry at Swisscom

IaaS

BOSH

Messaging (NATS)

DEA

Healthmanager

Cloud Controller

Dynamic Router

Service Broker

Logging & Metrics

UAA/Login

User Provided Services

Apps

Buildpacks

Services3rd party integration

Business Integration

Stateful container-based services

SDSOpenstack SDN

External servicesIAM federation ELB/WAF

PortalBilling Extensions

Lifecycle?

Operations?


Lifecycle

IaaS

BOSH

Messaging (NATS)

DEA

Healthmanager

Cloud Controller

Dynamic Router

Service Broker

Logging & Metrics

UAA/Login


Apps

Buildpacks




SDSOpenstack SDN



Lifecycle?

Operations?


Lifecycle

Ia a S

BOS HM e ssa g ing (NAT S )

DE A

He a lthm a na g e rClo ud Co ntro llerDyna m ic R o ute r

S e rvice Bro ke r

Lo g g ing & M e trics

UAA/Lo g in

Use r P ro vid e d S e rvice s

AppsBuild pa cks

S e rvice s3 r d pa rty inte g ra tio n

Busine ss Inte g ra tio n

S ta te ful co nta iner-ba se d se rvice s

S DSOpe nsta ck S DN

E x te rna l se rvice sIAM fe d e ra tio n E LB/WAF

P o rta lBilling E x te nsio ns

Lifecycle?

Operations?

Ia a S


DE A


S e rvice Bro ke r


UAA/Lo g in


AppsBuild pa cks







Ia a S


DE A


S e rvice Bro ke r


UAA/Lo g in


AppsBuild pa cks







Ia a S


DE A


S e rvice Bro ke r


UAA/Lo g in


AppsBuild pa cks







Ia a S


DE A


S e rvice Bro ke r


UAA/Lo g in


AppsBuild pa cks







Ia a S


DE A


S e rvice Bro ke r


UAA/Lo g in


AppsBuild pa cks







Ia a S


DE A


S e rvice Bro ke r


UAA/Lo g in


AppsBuild pa cks







Ia a S


DE A


S e rvice Bro ke r


UAA/Lo g in


AppsBuild pa cks







Ia a S


DE A


S e rvice Bro ke r


UAA/Lo g in


AppsBuild pa cks







Ia a S


DE A


S e rvice Bro ke r


UAA/Lo g in


AppsBuild pa cks








Lifecycle

main challenges

> automated provisioning of PaaS stacks

> keep things “up-to-date”

> integrations


Lifecycle

main challenges



> integrations

> # cat cf-stub.yml | wc –l

> ~ 2200 LOC> # cat cf-bs.yml | wc –l

> ~ 300 LOC

> # cat appcloud.yml | wc –l

> ~ 15 LOC

> spiff / bosh-workspace

> SC appcloud bootstrap


Lifecycle

main challenges



> integrations

Addressed by:

> standardization

> automation

> testing

> layers of abstraction

> strong CI/CD process

> an awesome team JCF Summit Berlin 2015 (c) Swisscom 2015

Operations

> Challenges:

> Tremendous SLA requirements

> Proper instrumentation and constant monitoring

> Fixing problems ASAP

> Reducing the workload of ops teams

> Addressed by:

> Deep instrumentation

> Loosely-coupled systems (add/remove freely)

> Automation

> Aggregation

> Clear separation of automation vs documentation


Any

infrastructure

component

The OODA loop



OODA loops are all over the place(disjoint and incomplete,AKA existing monitoring and management tools)

IaaS

BOSH

Messaging (NATS)

DEA

Healthmanager

Cloud Controller

Dynamic Router

Service Broker

Logging & Metrics

UAA/Login


Apps

Buildpacks




SDSOpenstack SDN




Design principle: don’t reinvent the wheel

Use existing mechanisms as much as possible

Bosh healthmonitor

Consul checks CloudFoundryhealth manager

Puppet policies OpenStack health monitors

PlumGrid health management

Linux/Unix cronjobs

App-specific internal monitoring

InfluxDB data



Build on top to coordinate and add missing information

Design principle: aggregate and unify


Delegate the hard decisions to humans, but use them to improve analysis


Build on top to coordinate and add missing information

Design principle: iterative improvement

Image based on a work at hyperboleandahalf.blogspot.com. CC BY-NC-ND 3.0 US

CF Summit Berlin 2015 (c) Swisscom 2015Inspiration: Netflix’s Chaos Monkey (https://github.com/Netflix/SimianArmy/)

Automated disruption to test automated responses

Self testing: Chaos Heidi

Consul checks

Consul checks

Consul checks

Checkresults

Health manager

Status and other Consul events

Message bus

Responder

Responder

Responder

Responder = Puppet, mco, scripts, etc.

Rules

Events

Resu

lts a

nd

logg

ing

even

tsAnalysis engine

Logical architecture

Heidi

agents


Console


Conclusions

> You cannot run CF on a vacuum

> You can’t always have a green field! Integrations are hard

> Network integration

> Security

> Other legacy systems

> Health management is indispensable

> Automate as much as possible

> Delegate hard decisions to humans

> Don’t try to solve everything at once


developer.swisscom.comCloudFoundry From Switzerland: As Tasty as our Chocolate!

@Swisscom_Dev


Thank you!Boban GlisovicDiego Zamboni

running cloud foundry at swisscom...cloud infrastructure cloud foundry (paas) services (3rd party)...

Documents