INFRASTRUCTURE DEVELOPMENT ON WINDOWS

@owainperry

London CD meetup – 4th September 2014

Who am I

Owain Perry

Software architect at Justgiving.com

@owainperry

@owainperry

Justgiving.com

Enable people to raise money for charity

£~400 million

20 to 75 transactions / min avg

Steady load

Very spike loads (viral traffic)

@owainperry

What am I going to talk about?An windows infra dev 101 for those starting outThings to think about

@owainperry

Quick poll

Who’s working in a Windows environment?

Puppet / chef / other

@owainperry

Our Architecture (as of today) Lots of change in our software architecture

Existing monolithic application (chopping up)

Micro services

Event Driven Architecture

CQRS

Event sourcing

@owainperry

Just giving server estate

Smaller hosting supplier based on VMware

250 - 300+ servers mostly in AWS, growing rapidly.

Some servers in Azure

@owainperry

Why do automation?

The Friday failure - We go live on Tuesday

Auto upgrade=true

Chocolatey upgraded to a non recoverable broken state

One we realised the issue (Mean Time To Comprehension ~ 1hr)

Changed the setting

We blew away the whole estate ~60 servers while at lunch

Job done.

@owainperry

Tools and tech stack

Windows Linux (a little) Chef Vagrant Minitest (test kitchen) Chocolatey Artifactory Ruby Powershell Berkshelf

@owainperry

Where do you start?

Configuration management

There is a need to manage configuration between environments

Configuration of servers

What software is installed.

We are using Chef

@owainperry

Why Chef?

Ok windows support

DSL and ruby at the same time....

Search (powerful, but you need to use it with thought)

ChefDK

Great community

@owainperry

Doing windows infrastructure development

Vagrant

Work on your servers from the comfort of your own desktop

Images for windows are large 6+ GB

*hint use packer.io to build your base images

@owainperry

Package management

To keep things sane , you need package management

Chocolatey - it works

Oneget – in powershell 5

Download a package over http

Download a specific version

Easy to build

(just, but it's not apt-get or rpm) still a bit beta , but better than nothing

*hint, build your own self contained packages, so you don't depend on the internet

*rant come on MS lets party like it’s 1999

@owainperry

Artifact repository

Store your Software deliverables

Chocolatey packages

Other stuff? (custom gems)

Supports HA , fast , support (becomes very central to infrastrcuture)

examples include Nexus, Artifactory,....

@owainperry

Mini test (test kitchen)

Write tests around your infrastructure - start doing TDD

Just provides some sanity when changing things

Takes discipline

@owainperry

Using CI

to upload your infrastructure code to server

Execute tests on you infra code

Supports TDD

Makes change to the server repeatable.

Version stamping

@owainperry

Writing windows automation

Chef primitive providers (pretty good coverage)

To powershell or not to powershell?

How do you make things idempotent with powershell?

Writing things in ruby? (for chef it's more native)

@owainperry

Who's the boss?

Group policy? Chef?

What is the point of group policy in a chef world?

E.g. Create a local user to run a service using chef, group policy will try and remove it.

@owainperry

Frustrating things about windows SomeUser (domain or local?)

./Someuser

hostname/someuser

etc...

Installing .Net 3.5 can only be done usingfeatures

Writing a shutdown script

Installing .Net 4 via winrm (can’t install update)

@owainperry

Yesterdays issue…

Servers Ran out of diskspace due to local caching

20 mins to delete the data on each server (High CPU load) while serving traffic.

We flicked from the blue to green side

Just deleted the servers and recreated (~10 mins)

Automation is just awesome….

@owainperry

So…

Choose a configuration manager

use vagrant - it's just awesome

sort out package management

Try and do TDD if you can

Get some CI in place

Write code

(get frustrated with Microsoft every couple of hours)

Delete your estate and rebuild it while you have lunch , it makes you feel great

@owainperry

Questions?

@owainperry