No ops to NoOps
A story of a journey...
github.com/flypunk
● In IT since '99, Unix SysAdmin, OPS@XIV, cloud/devops since 07 (@SeekingAlpha)
● Cloud migration and deployment automation in Amadesa
I am a freelancer specializing in Web Applications Delivery
About me
Not an easy thing to describe - see 'NoOps' Debate Grows Heated@AppFog Carlson - Use 3rd party PaaS
@Netflix Cockroft - Outsource messing with hardware to AWS and build your own PaaS
@Etsy Allspaw - We do it the same way as Netflix, but I HATE the term, especially the word 'NO' in it ;)
WTF is NoOps?!
Culture, software architecture, methodologies and tools that enable a (SaaS) company run their business without a dedicated Operations team.
What do I mean by NoOps
Y U NO OPS!?
Waste - Any activity not contributing to customer value (Agile)
While you have No Nops you have...
No DelaysNo Communications overheadNo Dev vs. Ops fingers pointing - Dev is always to blame ;)
NoOps - Eliminating Waste
Study what is needed
Build what is missing
Get the hell out!
If I did my job well, the company shouldfigure out production/operations on its own.
My goal as a (NoOps) consultant
Today - www.massiveimpact.com
End to end solution for ads on cellular
Conversions, not clicks
The client - MassiveImpact
Hired new people
Written new code
Had to start a new production site
The company pivoted from a different product
.Net on the front end
Big data (hive) on backend
MassiveImpact technology stack:
The head of QA with some Ops experience
I need to ensure that he and his bosses could sleep well at night...
Is it possible for MassiveImpact to continuegrowing without hiring a dedicated Ops employee?
I think - yes!
Who is responsible for ops in the new environment?
Amazon VPC: Harder to set up, more secure, closer to collocated network setup
Let's make everything organized: Chef, Cloud Formation
Btw, we have 2 weeks to launch!
Start manually, add automation later
Requirements for the new environment
ServerDensity for both machine and service monitoring
● Has built in alerts and graphs● Machines have to be able to connect to
the internet to use it● Used the NAT-PAT instance inside VPC +
static IPs to achieve it.
NoOps advantage: A cloud solution.
Monitoring
Written a naming/access policy doc
Created IAM users, groups, policies and resources in accordance with the doc.
Pro tip: don't use MixedCaseBucketNames - hive won't work with them.
Another tip: EMR with IAM keys and policies is not trivial
AWS IAM credentials
Why EMR failed
Set up a server that will work as binaries repository
Put artifacts into sane default places
Will be used later for CI and deployment scripts
Deployment
It (kinda;) works!
You can bootstrap and manage nodes with WinRM!
NoOps advantages:
Configuration as codeNo need to manage AMIs (It's a nightmare)
Chef on Windows
Not everything works
First time in my career able to manage M$ servers and install packages from CLI. == Awesome!!!
Easy way to manage users in centralized manner without AD
Chef on Windows - continue
The system is fully automatic and is run by Cron
Notifications using SNS and email
NoOps advantage: It is fully data driven(Just put the new files to a specified S3 bucket)
Data crunching - Gurobi Cloud
Monitoring - Implemented
Chef - in development
Cloud Formation - planned
CI/CD - planned
Ops/business console (GeckoBoard) - planned
Current status
People, then methodologies, last tools
Architect for no ops!Change is hard, but engineers will accept new tech when shown that it worksMake the changes backwards compatible (Libraries for AWS calls)Use external services as a default choice
Takeaways
BTW, you can get the presentation here:
Q&A