juc europe 2015: multi-node environment as a jenkins slave (compound-slave)
TRANSCRIPT
Multi-node environment as a Jenkins slave aka “compound-slave”
Denis Chernilevskiy
Head of Media Advertising Department
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
Common use-case the “prehistory”
Multi-node environment as a Jenkins slave
Common use-case
4
He’s famous The “slave” The “job”
│ Test this please!
The quest
6
Nodes Cloud Deploy Get results Test
The solution
▌ Cloud – Amazon/OpenStack/Docker/…
▌ Deploy – JClouds/DockerCloud + Chef/Salt/Ansible/…
▌ Test – PyTest/TestNG/…
▌ Results – Allure Framework/xUnit/…
7
How it works
▌ The common job concept
› Take 1 slave
› Deploy the system under test to the slave
› Deploy and run tests on the same slave
› Clean the slave
› Return the slave to the pool
8
│ Perfectly fits with 1 slave
│ But what if…
11
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
The “real life” of QA/DevOps the problem
Multi-node environment as a Jenkins slave
│ Yandex MediaAd platform
│ Test this please!
Process requirements
▌ Auto build, deploy, test
▌ Continuous Integration
▌ “Green trunk”
15
System components
▌ Front-end UI and API
▌ Ad.Management
▌ Ad.Selection ▌ Ad.Storage
▌ SocDem prediction & machine learning
16
│ None fits into 1 slave…
The obvious “not a solution” J
▌ Use Jenkins
▌ Get a slave for the job
▌ Get some more cloud nodes (somehow)
▌ Use the slave as a launchpad (run deploy, tests remotely)
18
│ Not for us J
Why not?
▌ Laziness. The less new code, the better! ▌ Separate nodes management
▌ JClouds not reused ▌ Lack of visibility
20
│ Several slaves per job?
│ Need to refactor core…
│ Several nodes as a slave!
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
“Compound-slaves” plugin the solution
Multi-node environment as a Jenkins slave
Features
▌ Group several nodes as a compound-slave
› Existing nodes grouping
› Get & group from the cloud
25
Features
▌ Cloud provisioning
› Plugins with a “Cloud” interface
› Using labels from the cloud plugin
› Number of each label to provision
26
Features
▌ Roles and labelling
› Distinct nodes
› Default labels
27
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
Here comes the DEMO! J
Features
▌ General
› Built-in slave management
› Built-in job-slave provisioning
▌ Visibility
› Built-in dashboards
› Built-in job steps configuration
29
│ Profit! J
Links
▌ wiki.jenkins-ci.org/display/JENKINS/Compound+Slaves
▌ github.com/jenkinsci/compound-slaves-plugin
31
Contacts
+79250351311 @dchernilevskiy
denis.chernilevskiy
Denis Chernilevskiy
Head of Yandex MediaAd Department