how build and deploy affects architecture at thetrainline.com - akqa anoraks
Post on 13-May-2015
959 Views
Preview:
DESCRIPTION
TRANSCRIPT
#akqaanoraks @matthewpskelton
How build and deployment shapes software architecture
at thetrainline.com
Matthew Skelton | thetrainline.com
AKQA Anoraks, London | #akqaanoraks
29 November 2012
#akqaanoraks @matthewpskelton
architecture
= f (build & deploy)
(for some systems)
#akqaanoraks @matthewpskelton
Systems engineering (robotics, control theory, sensors, neuroscience)
Software development
(finance, insurance, travel, pharma, media, medical imaging)
now
Build & Deployment at thetrainline.com
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
9m visits per month to thetrainline.com
2nd most visited UK travel booking website
(behind Easyjet)*
Over 100,000 daily visits via our mobile apps
*source: IMRG hotshops list Nov 2011
#akqaanoraks @matthewpskelton
Booking engines for many UK train operators
Ticketing systems for Corporates and TMCs
Several hundred individual websites
> £1bn ($1.5bn USD) per year in train ticket sales
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
• Early “distributed agile” success story
• http://www.thoughtworks.com/trainline
#akqaanoraks @matthewpskelton
10+ dev teams in 2 locations
#akqaanoraks @matthewpskelton
Server Farm (Build, Deployment, Replication)
100+ build agents 20 dev environments x 20 VMs per
environment = 400 dev VMs Git replication (Gitolite), …
MORE SERVERS THAN PRODUCTION
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
architecture
= f (build & deploy)
(for some systems)
#akqaanoraks @matthewpskelton
“HERESY!”
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
Web-based
Frequently- changing
Public-facing
High-volume
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
RELIABLE
REPEATABLE
RAPID
RECURRING #webperfdays
@matthewpskelton #akqaanoraks
@matthewpskelton
#akqaanoraks @matthewpskelton
‘R-R-R-R’ BUILD AND DEPLOYMENT
Helps to avoid the Ball of Mud
Enables high-performance architectures
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
LOAD BALANCING HIGH AVAILABILITY
SCALING ELASTIC
RAPID RECOVERY
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
BUILDABLE
Small pipelined builds on generic build machines
Seconds, not minutes or hours
Short feedback cycles (Dan Worthington-Bodart, @danielbodart - http://bit.ly/M85wsX)
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
Build dependencies may mirror runtime deps
msbuild /m for multi-threaded builds (forces component isolation)
Gallio (MbUnit) – replacement engine for NUnit
NuGet + Squid for super-fast build artefacts
RAM disk for some working copy operations
#akqaanoraks @matthewpskelton
TESTABLE
Test (separation, harnesses, points)
IDENTIFIABLE
Meaningful versions, packages, defined dependencies, artefact
management
(think component boundaries)
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
Enforced “onion” testing layers (test separation, build machine firewalls, build agent types, …)
Semantic Versioning (A.B.C.D + contract to ‘clients’)
30+ external services (test harnesses)
#akqaanoraks @matthewpskelton
DEPLOYABLE Rapid, scriptable, simple failure modes
MONITORABLE Logging, metrics, transaction tracing
CONFIGURABLE
Inject settings – no ‘black boxes’
LIGHTWEIGHT Keep things small and easily comprehendible
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
GO, MSDeploy, hand-rolled, Octopus, PS remoting (“as simple as xcopy”)
Graphite + SqlToGraphite (https://github.com/perryofpeek/SqlToGraphite)
LogStash + log4net (tune logging levels with config, transaction tracing)
SCOM – operations manager (early monitoring in Dev)
#akqaanoraks @matthewpskelton
INSTANTIABLE
No snowflakes or singletons
RECOVERABLE
No nasty zombies after failures
MTTR more important than MTBF* * for most kinds of F
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
Rebuild Build Agents with Chef
Chef for environment rebuilds
Call Start() and Stop() on each component
#akqaanoraks @matthewpskelton
#webperfdays @matthewpskelton
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
Well-known monolithic business rules engine
Costly to build, deploy, configure, test
Being replaced
#akqaanoraks @matthewpskelton
RELIABLE
REPEATABLE
RAPID
RECURRING #webperfdays
@matthewpskelton #akqaanoraks
@matthewpskelton
#akqaanoraks @matthewpskelton
LOAD BALANCING HIGH AVAILABILITY
SCALING ELASTIC
RAPID RECOVERY
#webperfdays @matthewpskelton
LIGHTWEIGHT TESTABLE MONITORABLE CONFIGURABLE RECOVERABLE IDENTIFIABLE
#akqaanoraks @matthewpskelton
#akqaanoraks @matthewpskelton
London Continuous Delivery
Next meetup: 17 January 2013, with
Opscode (Chef)
#londoncd
#akqaanoraks @matthewpskelton
architecture
= f (build & deploy)
(for some systems)
thank you
@AKQAAnoraks
Blog: http://engineering.thetrainline.com/
matthewskelton.net | @matthewpskelton
Thanks to: Attila S, Jack R and Owain P for feedback.
Picture credits:
Label: seanlabel.com; Sheep: gallery.hd.org; Train: worldontrains.blogspot.co.uk; Gurner: mog.com; Petra: Wikimedia/Berthold Werner; army engineers: US DoD; ball of mud: pwern.blogspot.co.uk; sports car: xarj.net; zombie: bjj.org; feather: Wikipedia; punch:
thelegalblitz.com; passport: coverpalace.com; dogs: reluctantmemsahib.wordpress.com; Meccano: dalefield.com
top related