Spotify’s journey to GCP

~# whoami

Alexey Lapitsky / @vzctlalexey@spotify.com

Site Reliability Engineer @SpotifyPM for the service track of the GCP migration

Agenda

● History● Why GCP?● The biggest mistake● Learnings● Questions!

Spotify’s cloud history

2010Manual VM provisioning

2012Cloudstack, reached ~700 instances

2013Amazon EC2

2015Google Cloud Platform

Current status

● 4 data centers, ~10k physical servers● Micro-service architecture, ~1k different services● Mostly commodity hardware (Dell)● Fully automated, self-service server provisioning

Moving everything to GCP

Focus

Why GCP?

Why GCP?

● BigData● Culture● Timing● Partnership

Our worst mistake

Spoticloud

CLIREST

DBCloudstack

AWS

Puppet

DNS

Server

● CLI Tool + REST Server● User authentication with ssh agent: github.com/spotify/crtauth

SSH

Jumphost

Looks awesome!

● No need to distribute AWS accounts● No need to manage ACLs● Extremely easy to use

~# spcloud create --name web --project alexey

… or not really :(

● Testing environment, shamelessly misused for production● No accountability or a proper ownership tracking

○ 60% of instances were wasting money● Access only to the basic cloud primitives

○ Which are already available on-premise

Currently being deprecated.

GCP Migration

Cross-project networking (XPN)

Things we learned

● VPN is tricky● GCP network is great● Keep an eye on quotas ● Live VM migration!● Think about ACL mgmt in advance● BigTable, GCS, BigQuery are awesome

Questions?