javaone 2014: next step in automation: elastic build environment
DESCRIPTION
This is the JavaOne 2014 session by Jesse Glick & myself that makes a case for elastic build environment (such as EC2, OpenStack, etc) for JenkinsTRANSCRIPT
![Page 1: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/1.jpg)
©2013 CloudBees, Inc. All Rights Reserved 1©2013 CloudBees, Inc. All Rights Reserved
Next Step in Automation:
Elastic Build Environment
Kohsuke Kawaguchi / CloudBees, Inc.
[email protected] / @kohsukekawa
Jesse Glick / CloudBees, Inc.
[email protected] / @tyvole
![Page 2: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/2.jpg)
©2013 CloudBees, Inc. All Rights Reserved 2
Have You Met Jenkins? http://jenkins-ci.org/
![Page 3: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/3.jpg)
©2013 CloudBees, Inc. All Rights Reserved 3
![Page 4: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/4.jpg)
©2013 CloudBees, Inc. All Rights Reserved 4
![Page 5: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/5.jpg)
©2013 CloudBees, Inc. All Rights Reserved 5
![Page 6: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/6.jpg)
©2013 CloudBees, Inc. All Rights Reserved 6
My Jenkins around 2006
![Page 7: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/7.jpg)
©2013 CloudBees, Inc. All Rights Reserved 7
![Page 8: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/8.jpg)
©2013 CloudBees, Inc. All Rights Reserved 8
![Page 9: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/9.jpg)
©2013 CloudBees, Inc. All Rights Reserved 9http://www.flickr.com/photos/gbyrnes/912576883/
![Page 10: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/10.jpg)
©2013 CloudBees, Inc. All Rights Reserved 10
If only we had more computers…
• Just building & testing them all…
• Running tests more frequently
• Testing individual commits
![Page 11: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/11.jpg)
©2013 CloudBees, Inc. All Rights Reserved 11http://www.flickr.com/photos/drocpsu/8546730021/
![Page 12: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/12.jpg)
©2013 CloudBees, Inc. All Rights Reserved 12
Just enough computers
just in time
Elasticity
![Page 13: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/13.jpg)
©2013 CloudBees, Inc. All Rights Reserved 13
My Jenkins around 2007
![Page 14: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/14.jpg)
©2013 CloudBees, Inc. All Rights Reserved 14
![Page 15: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/15.jpg)
©2013 CloudBees, Inc. All Rights Reserved 15
![Page 16: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/16.jpg)
©2013 CloudBees, Inc. All Rights Reserved 16
![Page 17: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/17.jpg)
©2013 CloudBees, Inc. All Rights Reserved 17http://www.flickr.com/photos/drocpsu/8546730021/
![Page 18: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/18.jpg)
©2013 CloudBees, Inc. All Rights Reserved 18
18
![Page 19: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/19.jpg)
©2013 CloudBees, Inc. All Rights Reserved 19
Just enough computers
of the right kind
just in time
Elasticity!
![Page 20: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/20.jpg)
©2013 CloudBees, Inc. All Rights Reserved 20http://www.flickr.com/photos/82219206@N00/7003641975/
![Page 21: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/21.jpg)
©2013 CloudBees, Inc. All Rights Reserved 21
Correct answer
• Test assumes a fixture running on port 8080
– Doesn’t check if it’s already being used
• If another test runs at the same time…?
![Page 22: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/22.jpg)
©2013 CloudBees, Inc. All Rights Reserved 22http://www.flickr.com/photos/82219206@N00/7003641975/
![Page 23: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/23.jpg)
©2013 CloudBees, Inc. All Rights Reserved 23
Correct answer
• Because of “pkill -f -9 tomcat” cleanup
![Page 24: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/24.jpg)
©2013 CloudBees, Inc. All Rights Reserved 24http://www.flickr.com/photos/jumilla/8667648797/
![Page 25: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/25.jpg)
©2013 CloudBees, Inc. All Rights Reserved 25
Isolation
• At odds with large multi-core systems
• x86 virtual machines
• User isolation
• Kernel containers
![Page 26: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/26.jpg)
©2013 CloudBees, Inc. All Rights Reserved 26
![Page 27: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/27.jpg)
©2013 CloudBees, Inc. All Rights Reserved 27http://www.flickr.com/photos/82219206@N00/7003641975/
![Page 28: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/28.jpg)
©2013 CloudBees, Inc. All Rights Reserved 28
Correct answer
• Same Maven ID, two different jars
• Different projects designate different ones
• Local cache gets cleaned up periodically
• Whichever first runs after cache cleanup
“wins”
![Page 29: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/29.jpg)
©2013 CloudBees, Inc. All Rights Reserved 29http://www.flickr.com/photos/82219206@N00/7003641975/
![Page 30: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/30.jpg)
©2013 CloudBees, Inc. All Rights Reserved 30
Correct answer
• Test script leaves background daemon
process behind
• Over time it’ll slowly choke slaves
![Page 31: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/31.jpg)
©2013 CloudBees, Inc. All Rights Reserved 31
![Page 32: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/32.jpg)
©2013 CloudBees, Inc. All Rights Reserved 32
Throw away & create new
Elasticity!
![Page 33: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/33.jpg)
©2013 CloudBees, Inc. All Rights Reserved 33
![Page 34: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/34.jpg)
©2013 CloudBees, Inc. All Rights Reserved 34
Ladder to Cloud
Single
Multiple
Elastic
![Page 35: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/35.jpg)
©2013 CloudBees, Inc. All Rights Reserved 35
Solid OSS Elasticity Plugins
• EC2 plugin
• Jclouds plugin
– OpenStack, CloudStack
• Launch and tear down slaves on demand
![Page 36: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/36.jpg)
©2013 CloudBees, Inc. All Rights Reserved 36
VMWare auto-scaling plugin
• Snapshot
• Power on-off management
• Hypervisor-aware scheduling
• Folder based pooling
• VMWare tools integration
• One-time use support
![Page 37: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/37.jpg)
©2013 CloudBees, Inc. All Rights Reserved 37
Host that runs Docker
Docker plugin
![Page 38: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/38.jpg)
©2013 CloudBees, Inc. All Rights Reserved 38
Mansion
Slave Slave
Slave Slave
CloudBees DEV@cloud
![Page 39: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/39.jpg)
©2013 CloudBees, Inc. All Rights Reserved 39
Mave
n
Git
An
t
Me
rcurial
Grad
le
Sub
versio
n
Linux Kernel
Hardware
Linux Container = zero cost virtualization
![Page 40: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/40.jpg)
©2013 CloudBees, Inc. All Rights Reserved 40
For OS X
Mave
n
Git
XC
od
e
Git
XC
od
e
Sub
versio
n
OS X OS X OS X
QEMU QEMU QEMU
Linux Kernel
Apple Hardware
![Page 41: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/41.jpg)
©2013 CloudBees, Inc. All Rights Reserved 41
Kernel Same-page Merging
OS X OS X OS X
![Page 42: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/42.jpg)
©2013 CloudBees, Inc. All Rights Reserved 42
![Page 43: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/43.jpg)
©2013 CloudBees, Inc. All Rights Reserved 43
Mansion
Slave
Slave
Slave
Workspace 1
Workspace 2
Workspace 3
Workspace 4
![Page 44: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/44.jpg)
©2013 CloudBees, Inc. All Rights Reserved 44
Slave
Slave
Workspace
Workspace’’
Workspace’
![Page 45: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/45.jpg)
©2013 CloudBees, Inc. All Rights Reserved 45
Slave
Slave
~/.m2/repository
~/.m2/repository
~/.m2/repository
![Page 46: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/46.jpg)
©2013 CloudBees, Inc. All Rights Reserved 46
Parallel Testing
Test Group #1 Test Group #2 Test Group #3
![Page 47: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/47.jpg)
©2013 CloudBees, Inc. All Rights Reserved 47
Parallel Testing
foo #10 Test Group #1
foo #11 Test Group #2
foo #12 Test Goup #3
![Page 48: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/48.jpg)
©2013 CloudBees, Inc. All Rights Reserved 48
Validated Merge
upstreamrepo
gate repo
![Page 49: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/49.jpg)
©2013 CloudBees, Inc. All Rights Reserved 49
![Page 50: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/50.jpg)
©2013 CloudBees, Inc. All Rights Reserved 50
Workflow System
• Alternative to “freestyle” projects
• Scripted control flow
• Resumable execution across restarts
• All-in-one build/test/deploy pipelines
• Under active development
![Page 51: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/51.jpg)
©2013 CloudBees, Inc. All Rights Reserved 51
Workflow with Elastic Slaves
• One-line provisioning from cloud
• Language-level parallelism
• Run commands, archive files, test results
• Now integrates with parallel test plugin
![Page 52: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/52.jpg)
©2013 CloudBees, Inc. All Rights Reserved 52
![Page 53: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/53.jpg)
©2013 CloudBees, Inc. All Rights Reserved 53
Conclusion: Elasticity Benefits
• Just-in-time capacity
• Diversity without overhead
• Isolation
• Productivity gain
– parallel testing
– validated merge
– workflow
![Page 54: JavaOne 2014: Next Step in Automation: Elastic Build Environment](https://reader034.vdocuments.site/reader034/viewer/2022042816/559434111a28ab9b1a8b4657/html5/thumbnails/54.jpg)
©2013 CloudBees, Inc. All Rights Reserved 54