how gannett achieved scalability and agility with nosql – couchbase live nyc
TRANSCRIPT
Couchbase @ GANNETTHow Couchbase has become an integral part of GANNETT’s next generation publishing systems
Who we are
Alon Motro - Content Platform Services Manager. Oversees publishing systems and the presentation APIs that power the consuming applications.
Kristopher Alexander - Platform Architecture Lead. Works with various stakeholders within the Technology division on technical vision, process, and culture.
Who we are
● Gannett is one of America’s largest media companies.● We provide full national coverage through 93 local and national sites that reach 96 million total
monthly digital visitors.● Sites include USA TODAY, The Indianapolis Star, The Arizona Republic, etc.● For each property we produce a suite of products including desktop and mobile sites and native
iOS and Android applications. ● Our digital products produce many types of assets including articles, photo galleries, videos and
oembeds● We also provide numerous APIs for developers as well as syndicated content for vendors
Our Current Architecture
Current Challenges
● SQL Server is extremely expensive● Utilize on-prem pet servers that have difficulty scaling● Numerous replication issues with SQL Server● A multitude of business logic coded into Stored Procedures causes
significant delay between publish time and presentation time
How We Used to Retrieve an Asset/Front
What were our transition goals?
● Decrease the latency between publish time and presentation time● Remove business logic from data layer● Write as you want to Read● Allow for growth and scalability in the cloud and move away from pet
servers● Lower cost
New Architecture - Phase 1
How We Retrieve an Asset/Front Now
New Architecture - Phase 2
A Telling Load Test
● Our publishing system required around 500 writes per second and 3000 reads per second
● Successful Load Test of 10,000 writes per second and 30,000 reads per second
● Included spikes of up to 300,000 reads● This was done with 8 r3.xlarge nodes in Amazon (32 vCPU and 172 GB
Memory Allocated)
UGC
http://www.usatoday.com/yourtake/
Memcached Replacement
● 5 Memcached Servers shared by several applications● Each application had to manage its own keyspace correctly● Moved to a single couchbase cluster● Each application gets a bucket
Mongo DB Replacement
● Application Settings● User Preferences For The CMS● Mean JS/ Mean IO● Workflow Planning Module
Cloud Strategy
● “Hybrid Cloud” approach that blends different public offerings with Gannett’s private cloud
● Automation tools simplify configuration management● RabbitMQ, Solr, Couchbase, and Node JS replace .NET, SQL Server, and
MSMQ ● Reduced Cache footprint in the architecture
N1QL
● Reduce reliance on documents that point at other keys (manifests)● Reduce reliance on views● Migrate some simple lookups from Solr● Reduce cognitive load for interacting with documents
Questions?