DANIEL WESTER • CHIEF ENGINEER • TURNER BROADCASTING SYSTEMS , INC • @DWESTER42a

Adopting Continuous Integration in an Ops Group

All opinions stated are those of the presenter and does not necessarily reflect those of Turner or any of its affiliates or partners.

Disclaimer

• Development occurs in a wide range of languages

• A lot development teams

• A lot of testing already in place

• CI Server service offered by Infrastructure team

CI usage at TBS

• Encourages Source Code Management usage/practices

• Creates a build-focused mentality

• Standardizes the entry point to deployment

Why offer a CI Service?

• Create modular artifacts

• Avoid the kitchen sink type of applications

• Be able to switch application versions

• Reflected in our build plans

Some “tips” for scale, testing and deploys

Please make sure that XXX can scale and handle traffic. Oh we’re launching tomorrow.”“

• Limited resource availability (me)

• Out of band checks usually don’t get fixed

• Not popular with developers

• Last minute requests — not popular with reviewers

The problem

DIB

Daniel Box

Daniel Box

DOB still exists...

• Selenium backed

• Canned version of “DOB”

• Found 80% of DOB tests

• 2 versions — Web-based and Maven plugin

What is DIB?

• Website version could be down for weeks

• Tests in the build — reports in minutes

• Developer relies on in-build tests

• If in-build tests fails... There’s more time to fix...

Website versus in-build tests

• New development teams asked for it to be added

• Features added based on requests

• Other tests were added by dev teams

What did it do?

Build stages

What didn’t work?

What didn’t work - fUnit

Lessons Learned

• In-process automated testing is key

• Run tests as soon as commits are done

• If you’re a downstream team, provide upstream team’s tests

• CULTURE matters

Lessons learned

What about Ops?

• Configuration Management system

• Ruby-based

• Easily expandable

• Many ways of approaching things

What is Chef?

User adoption

We need to have a script that gets triggered whenever there’s a commit and does stuff.”“

Enter CI

Cookbooks

Creating a cookbook

10 minutes later...

• Lint

• Test

• Upload

Plan Stages

• Code style guideline checker & more

• Tabs versus spaces, “” versus ‘’

• Less objections to utilities reviewing code

Lint test

Tests

• CI server generates artifacts and uploads

• Avoid access issues

• CI server becomes “trusted” source of what’s a “good” version

Moving away from tagging

• Lint tests (internal)

• Publicly exposed lint tests

• Uploads

• Tests

Changing the engine while driving

Plan branches

Code Reviews

Before pull requests

• Lightweight review system

• Engineers choose to use pull requests

• Larger changes still go through Peer Review tool

• End result: More core reviewed

Pull requests

Today’s Code Flow

Commit on branch

Plan branchtriggered

Pullrequest

Master branch triggered

Upload and readyfor ‘trigger’

• Faster throughput

• Repeatable process

• Audit trail of when changes were made

• Lightweight process with large impact

End Result

Thank you!

DANIEL WESTER • CHIEF ENGINEER • TURNER BROADCASTING SYSTEMS , INC • @DWESTER42a

Text code below to 22333or visit http://bit.ly/18zxBVY

Adopting Continuous Integration in an Ops Group

To join this session, send text 136888 to

AWESOME = 3E

PRETTY GOOD = 3D

NOT BAD = 3C

MEH = 3B

Rate this Talk