cloudstack at spotify, nyc

13
2013-04-10 Cloudstack at Spotify

Upload: noa-resare

Post on 24-Jun-2015

1.217 views

Category:

Technology


0 download

DESCRIPTION

Cloudstack at Spotify, a slightly updated version of the presentation I presented at the Cloudstack meetup hosted by WebMD in New York City.

TRANSCRIPT

Page 1: Cloudstack at Spotify, NYC

2013-04-10

/2013

Cloudstackat Spotify

Page 2: Cloudstack at Spotify, NYC

About Spotify

Streaming music serviceBare metal, babyEngineering offices in Stockholm, Gothenburg, NYCHundreds of engineersLots of servicesHighly managed configuration

Page 3: Cloudstack at Spotify, NYC

Challenges

Let developers be developers

Build awesome tools

Geographically distributed teams

Support different use cases

Develop, test, integrate with external parties

Page 4: Cloudstack at Spotify, NYC

spvirtenvKVM hypervisors

DNS as database

Warning: Bad idea!

Everything in the client

Access everywhere

No error handling

Page 5: Cloudstack at Spotify, NYC

SpoticloudBuilt on top of cloudstack

Command line tools

API Server

Async task execution

Integration with puppet, DNS

Not highly available

Something we can grow with

Page 6: Cloudstack at Spotify, NYC

ssh key authenticationssh-agent

Challenge/Response over http

Token gives access

LDAP key & group

github.com/spotify/crtauth

Page 7: Cloudstack at Spotify, NYC

Grow your storageHigh level of integration with guest os

Creates local storage backed block device

Script that checks for new devices

Logical volume grows

Triggers ext4 resize

Page 8: Cloudstack at Spotify, NYC

Seamless configPuppet class set on create

DNS generated on create

User created early

Page 9: Cloudstack at Spotify, NYC

Cloudstack lessonsKiller feature: add hypervisors, provision guests

Moving fast

Lots of features you don’t use

Still an appliance mindset

Feedback could be improved

Page 10: Cloudstack at Spotify, NYC

What we want to focus onImprove the basic use cases

Ever added a hypervisor with wrong pod id?

Setting up a new cluster, what is secondary storage VM doing?

Building, packaging, system administration

Work around maven transitive dependency problems

Make less assumptions about netfilter configuration

Page 11: Cloudstack at Spotify, NYC

Looking aheadLarge scale integration testing

Unified interface for private and public cloud

Spcloud in production

Storage

Page 13: Cloudstack at Spotify, NYC

Apr 2013

spoti.fi/devops

Yes, we’re hiring