STUPS.ioan Open Source Cloud Framework for Scalehenning.jacobs@zalando.de / @try_except_WHD.local Frankfurt, 2015-09-10

background image based onhttps://commons.wikimedia.org/wiki/File:CERN_Server_03.jpg

by Florian Hirzinger - www.fh-ap.com

Henning Jacobs

● STUPS Hacker

● Twitter: @try_except_

● henning.jacobs@zalando.de

15 countries3 fulfillment centers15+ million active customers2.2+ billion € revenue 2014130+ million visits per month8.000+ employees

ONE OF EUROPE’S LARGEST ONLINE FASHION RETAILERS

A BRIEF HISTORY OF ZALANDOTECHNOLOGY

2010

2010

Gütersloh

2012

2012

Gütersloh Berlin

Platform Platform team

request serversdeploy

Platform

THE CHALLENGE

80+ delivery teams

Platform team

deploy

request serversrequest storage

RADICAL AGILITY

AUTONOMY

Compliance Innovation

STUPSSTUPS To Unleash Penguin Swarms

AWS

STUPS

DOCKERDEPLOY

SSH ACCESS

AUDIT REPORTS

FULL AWS ACCESS

STUPS: A PLATFORM ON TOP OF AMAZON WEB SERVICES

ONE DATA CENTER PER TEAM

Internet

*.abc.example.org *.xyz.example.org

Team ABC Team XYZ

ISOLATED AWS ACCOUNTS

EC2EC2

ELBELB

EC2

DEPLOYMENT

IMMUTABLE STACKS

ELB myapp-1

myapp.example.org

EC2+ Docker

EC2+ Docker

EC2+ Docker

IMMUTABLE STACKS

ELB myapp-1

EC2+ Docker

EC2+ Docker

EC2+ Docker

ELB myapp-2

EC2+ Docker

EC2+ Docker

myapp.example.org

ELB myapp-2

EC2+ Docker

EC2+ Docker

myapp.example.org

IMMUTABLE STACKS

AWS

DEPLOYMENT WITH SENZA

Senza CLI

Pier One

docker pull

docker push

Taupage

$ docker build -t ↲ pierone.example.org/myteam/hello-world:0.2 .

DOCKER BUILD & PUSH

$ docker build -t ↲ pierone.example.org/myteam/hello-world:0.2 .

$ pierone loginGetting OAuth2 token "pierone".. OKStoring Docker client configuration in ~/.dockercfg.. OK

$ docker push ↲ pierone.example.org/myteam/hello-world:0.2

DOCKER BUILD & PUSH

SENZA: STACK DEPLOYMENT

$ senza create hello-world.yaml 1 0.2

Generating Cloud Formation template.. OK

Creating Cloud Formation stack hello-world-1.. OK

SENZA: STACK DEPLOYMENT

$ senza create hello-world.yaml 1 0.2

Generating Cloud Formation template.. OK

Creating Cloud Formation stack hello-world-1.. OK

$ senza events hello-world.yaml 1Stack Name│Ver.│Resource Type │Resource ID │Status │Status Reason │Event Time

hello-world 1 CloudFormation::Stack hello-world-1 CREATE_IN_PROGRESS User Initiated 10m ago

...

hello-world 1 CloudFormation::Stack hello-world-1 CREATE_COMPLETE 6m ago

LOGGING

SSH ACCESS

SSH ACCESS: TIME-LIMITED ACCESS TO ANY TEAM SERVER

OAUTH

OAUTH: CREDENTIAL DISTRIBUTION VIA S3 BUCKETS

AWS

WEB UI

get access token

Taupage

OAuthProvider

store passwords

get passwordS3

rotate passwords

NETWORK

● ELB for inbound● NAT for outbound● HTTPS Only● Internal subnets

for app instances● odd SSH bastion

DMZ DMZ DMZ

internalinternal

eu-west-1a eu-west-1b eu-west-1c

ELB

EC2

internal

EC2

NAT

AWS ACCOUNT VPC SETUP

odd

● 800+ in Zalando Tech

● 90+ AWS Accounts

● 160+ Applications

SOME NUMBERS..

5 THINGSWE LEARNED

AUTONOMOUS TEAMS NEED THEIR OWN DATA CENTER

#1

SOME THINGSONLY WORKIN “THE CLOUD”

#2

AWS IS FARFROM PERFECT

#3

TRANSPARENCYIS KEY

#4

OUR NEEDS REQUIREDOUR SOLUTION

#5

Questions?

STUPS Frontpagestups.ioGitHub Repositoriesgithub.com/zalando-stups

tech.zalando.com@try_except_

BACKUP

Docker Registry

build

approved

EC2 Instance

Docker Container

Ticket System

Application Registry

SCM

Docker ImageTicket

Commit

✓

TRACEABILITY

Application Version

STUPS COMPONENTS