mongodb to cassandra
DESCRIPTION
An overview of experiences of moving from MongoDB to Cassandra from the team at metabroadcast.TRANSCRIPT
MongoDB to CassandraThe Atlas Odyssey
Fred van den DriesscheEngineer@fredvdd
Tom McAdamCTO
@tfm
Adam HorwichSystems Engineer@Mmmkayness
http://flickr.com/photos/dhammza/88644497/
tbc
Video and audio metadata from 20+ sources
Profiles and activity from video and audio products, social networks
Our platform - late 2012
tbc
MetaBroadcast platform
Analytic requests and groupings
?
Main clients Main Partners
Data Partners
What is Atlas?
ATLAS
DB
BBC
PA
C4
etc...
/content
/schedules
/topics
sitemaps
radioplayer
interlinking
DEMO
Atlas Data Model
brand item
series version
broadcast location
MongoDB
• flexible
• features
• really simple
• shell
Where MongoDB falls short
• too simple
• lack of control
• sharding
• embedding
Where to?
Where to?
• add a cache?
Atlas API• content
• http://atlas.metabroadcast.com/3.0/content.json?uri=http://www.bbc.co.uk/programmes/b0074g7p&annotations=description,brand_summary,locations&apiKey=6ed2a984627daff816198acde82
• http://atlas.metabroadcast.com/3.0/content.json?apiKey=aaaa&uri=http://www.bbc.co.uk/programmes/b0074g7p&annotations=description,brand_summary,locations
• schedules
• http://atlas.metabroadcast.com/3.0/schedule.json?from=now&to=now.plus.3h&channel=bbcone&publisher=bbc.co.uk
• http://atlas.metabroadcast.com/3.0/schedule.json?from=1948-12-24&to=1948-12-25&channel=radio4&publisher=bbc.co.uk
• api explorer http://atlas.metabroadcast.com/#apiExplorer
Atlas API• content
• http://atlas.metabroadcast.com/3.0/content.json?uri=http://www.bbc.co.uk/programmes/b0074g7p&annotations=description,brand_summary,locations&apiKey=6ed2a984627daff816198acde82
• http://atlas.metabroadcast.com/3.0/content.json?apiKey=aaaa&uri=http://www.bbc.co.uk/programmes/b0074g7p&annotations=description,brand_summary,locations
• schedules
• http://atlas.metabroadcast.com/3.0/schedule.json?from=now&to=now.plus.3h&channel=bbcone&publisher=bbc.co.uk
• http://atlas.metabroadcast.com/3.0/schedule.json?from=1948-12-24&to=1948-12-25&channel=radio4&publisher=bbc.co.uk
• api explorer http://atlas.metabroadcast.com/#apiExplorer
Why Cassandra?
• scalability/performance
• row caches
• consistency control
• column-based model matches our use case
And?
• ElasticSearch
• messaging
• tooling: bootstraps
What is Atlas?
BBC
PA
C4
etc...
Data ingest server DB
Update bus
ES
HTTP server
Data model
• columns to model annotations
• secondary indexes• index.direct(keyspace, SEGMENT_URI_INDEX_CF, ConsistencyLevel.CL_QUORUM).
from(segment.getCanonicalUri()). to(segment.getIdentifier()). index().execute(requestTimeout, TimeUnit.MILLISECONDS);
ID generation
• give external data our own ID on ingest
• needs to be user-friendly:http://www.radiotimes.com/programme/cf2/eastenders
• mongo: findAndModify()
• solution: uses Astyanax client with its distributed locking
• more details: http://metabroadcast.com/blog/let-cassandra-identify-your-data
Where we’re at
• already live with some data
• alpha release of schedule endpoint coming soon
• later: roll out across other endpoints
Ops
Ops in Cassandra
• we love Puppet
• it’s great for automation and deployment
• MongoDB: 1 file
• Cassandra: 2 files!
• oh... tokens
Cassandra Tokens
• define where data is written to in a cluster
• therefore balanced tokens = balanced cluster
• tokens should be rack aware
• tools available to provide appropriate tokens for you
Cassandra plays nicely with AWS
• datacentre / rack aware
• AWS Region = Datacentre
• AWS Availability Zone = Rack
• only recently introduced in MongoDB but simple to implement in Cassandra
• horizontally (and vertically) scalable
Monitoring
• Nagios is a little threadbare for Cassandra
• basic TCP service check
• stats from API not very helpful
• nodetool and CLI tools useful
• manual effort to integrate them
• if only there was some useful service...
OpsCenter
• wonderful for an overview
• not so much for alerting ;)
• ohai API
• can integrate metrics into Nagios
Disaster Recovery
• we operate a 4 node cluster presently
• replication factor of 3 with quorum read/writes
• DR complicated by tokens
• cluster should be balanced
• snapshot + S3 Backups
Cluster Happiness and Headaches
• little maintenance overhead
• cluster rebalancing
• uncommon maintenance procedure
• schema changes are cumbersome
• little scope for rollback, can put cluster in unrecoverable state
Summary
• Mongo is good, Atlas has outgrown it
• Cassandra isn’t a drop-in replacement
• Ops more complex but so far so good
Questions?