continuous integration with robot sweatshop

ROBOT SWEATSHOPLightweight, non-opinionated Continuous Integration

WHOAMIgithub.com/JScott

WHATAMIDOINGPlease send help

JENKINS

… <builders> <hudson.tasks.Shell> <command>cd telusmkdir -p targetcucumber -p json_report</command> </hudson.tasks.Shell> </builders>…

– a Jenkins user

“I’d rather work around a bad tool than maintain something new.”

BUILDBOTfrom buildbot.process.factory import BuildFactoryfrom buildbot.steps.source import SVNfrom buildbot.steps.shell import ShellCommand

checkout = SVN(baseURL = 'git@myrepos.com:portal-project.git', mode = "update", username = "foo", password = "bar", haltOnFailure = True )

mkdir = ShellCommand(name = "mkdir -p telus/target", command = ["mkdir", "-p", "telus/target"], description = "make the results path")

cucumber = ShellCommand(name = "cucumber -p json_report", command = ["cucumber", "-p", "json_report"], description = "run Cucumber tests")

logresults = ShellCommand(name = "log-cucumber telus/target", command = ["log-cucumber", "telus/target"], description = "upload results")

factory = BuildFactory()factory.addStep(checkout)factory.addStep(mkdir)factory.addStep(cucumber)factory.addStep(logresults)

c['builders'] = [ BuilderConfig(name = "portal-project", slavenames = ['slave1', 'slave2', 'slave3'], factory = factory)]

http://docs.buildbot.net/current/tutorial/fiveminutes.html

“Typically, at first sight it looks like a bunch of complicated concepts that make no sense and whose

relationships with each other are unclear.”

STOP SUFFERING BAD TOOLS

We can do better

ROBOT SWEATSHOP

jobs/portal-project.yaml---branch_whitelist: - develop

commands: - mkdir -p telus/target - cucumber -p json_report - log-cucumber telus/target

SIDE-BY-SIDE CONFIGS<?xml version='1.0' encoding='UTF-8'?><project> <actions/> <description>requires these gems:&#xd;statsd-ruby&#xd;selenium-webdriver&#xd;cucumber&#xd;json&#xd;RMagick</description> <logRotator class="hudson.tasks.LogRotator"> <daysToKeep>10</daysToKeep> <numToKeep>-1</numToKeep> <artifactDaysToKeep>-1</artifactDaysToKeep> <artifactNumToKeep>-1</artifactNumToKeep> </logRotator> <keepDependencies>false</keepDependencies> <properties/> <scm class="hudson.plugins.git.GitSCM" plugin="git@2.2.1"> <configVersion>2</configVersion> <userRemoteConfigs> <hudson.plugins.git.UserRemoteConfig> <url>git@bitbucket.org:webchannel/uss-cucumber.git</url> <credentialsId>ad5d9dcc-cb2f-42a4-8e9a-f47b3208ce0c</credentialsId> </hudson.plugins.git.UserRemoteConfig> </userRemoteConfigs> <branches> <hudson.plugins.git.BranchSpec> <name>*/develop</name> </hudson.plugins.git.BranchSpec> </branches> <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations> <submoduleCfg class="list"/> <extensions> <hudson.plugins.git.extensions.impl.CloneOption> <shallow>true</shallow> <reference></reference> </hudson.plugins.git.extensions.impl.CloneOption> </extensions> </scm> <canRoam>true</canRoam> <disabled>false</disabled> <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding> <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding> <triggers> <hudson.triggers.TimerTrigger> <spec>H 11 * * *</spec> </hudson.triggers.TimerTrigger> </triggers> <concurrentBuild>false</concurrentBuild> <builders> <hudson.tasks.Shell> <command>cd telusmkdir -p targetcucumber -p json_report -p remote_ff</command> </hudson.tasks.Shell> </builders> <publishers> <org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver plugin="cucumber-testresult-plugin@0.4"> <testResults>telus/target/features_report.json</testResults> </org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver> <hudson.tasks.Mailer plugin="mailer@1.8"> <recipients>krystal.jackson2@telus.com leon.yu@telus.com jonathan.shin@telus.com Brendan.Brow@telus.com</recipients> <dontNotifyEveryUnstableBuild>false</dontNotifyEveryUnstableBuild> <sendToIndividuals>false</sendToIndividuals> </hudson.tasks.Mailer> </publishers> <buildWrapp

from buildbot.process.factory import BuildFactoryfrom buildbot.steps.source import SVNfrom buildbot.steps.shell import ShellCommand

checkout = SVN(baseURL = 'git@myrepos.com:portal-project.git', mode = "update", username = "foo", password = "bar", haltOnFailure = True )

mkdir = ShellCommand(name = "mkdir -p telus/target", command = ["mkdir", "-p", "telus/target"], description = "make the results path")

cucumber = ShellCommand(name = "cucumber -p json_report", command = ["cucumber", "-p", "json_report"], description = "run Cucumber tests")

logresults = ShellCommand(name = "log-cucumber telus/target", command = ["log-cucumber", "telus/target"], description = "upload results")

factory = BuildFactory()factory.addStep(checkout)factory.addStep(mkdir)factory.addStep(cucumber)factory.addStep(logresults)

c['builders'] = [ BuilderConfig(name = "portal-project", slavenames = ['slave1', 'slave2', 'slave3'], factory = factory)]

---branch_whitelist: - develop

commands: - mkdir -p telus/target - cucumber -p json_report - log-cucumber telus/target

67

33+

8

USAGE DEMOhttps://asciinema.org/a/22878

http://localhost:10554

THE FUTURERobot Sweatshop

• Output node

• Multiple workers easily

• Implicit job (done!)

• Non-TCP connections

• Worker Docker support

Sweatshop Gears

• Scaffolding scripts

• Custom repos

• Service hub

• Update/uninstall

• List/show

• Docker helper scripts

QUESTIONS?robotsweat.com

But what about all the crunchy parts…?

UNDER THE HOODovercoming some interesting challenges

DISTRIBUTED ARCHITECTURE

github.com/kostya/eye

JOB REQUEST RESILIENCE

GEARS DEMOpackage management tool for non-core bits

https://asciinema.org/a/22878

QUESTIONS?robotsweat.com