continuous integration in the cloud with hudson
DESCRIPTION
Continuous Integration in the Cloud with Hudson. Kohsuke Kawaguchi Jesse Glick Sun Microsystems, Inc. Hudson committers. Rise of Continuous Integration. Offload from people, push to computers. $. computers. us. time. Spend more CPU power to help you. … even if it only helps a little - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/1.jpg)
Continuous Integrationin the Cloudwith HudsonKohsuke KawaguchiJesse GlickSun Microsystems, Inc.Hudson committers
![Page 2: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/2.jpg)
Rise of Continuous Integration
> Offload from people, push to computers
2
$
time
computers
us
![Page 3: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/3.jpg)
Spend more CPU power to help you
> … even if it only helps a little
> First on your laptops and workstations IDEs are at the forefront
> And then to the servers a.k.a. “Continuous Integration” More frequent build/test executions Static code analysis tools And more to come
3
![Page 4: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/4.jpg)
Hudson
> Open-source CI server at java.net
> Emphasis on ease of installation and use “java -jar hudson.war” execution Configure everything from browsers
> Extensibility 140+ community-developed public plugins By 150+ contributors
> Estimated 13,000 installations4
https://hudson.dev.java.net/
![Page 5: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/5.jpg)
It basically does builds and tests
> Check out the source code Subversion, Perforce, Git, Mercurial, CVS, …
> Do builds and/or tests Ant, Maven, MSBuild, shell script, …
> Record results Binary, test results, code coverage, static analysis
> Notify people E-mail, IM, RSS, tray apps, IDEs
5
![Page 6: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/6.jpg)
Localized to 8 languages
6
![Page 7: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/7.jpg)
And hopefully more to come…
7
![Page 8: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/8.jpg)
8
Adoption in all kinds of businesses
![Page 9: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/9.jpg)
9
![Page 10: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/10.jpg)
Why Distributed Builds?
> You need to use multiple computers because… You need different environments You need isolation
> There’s only so much you can do with 1 computer
10
![Page 11: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/11.jpg)
Before we talk about clouds…
> Going virtual doesn’t solve… Software installation problem Node failure problem Remote maintenance problem …
> What does Hudson do to help you with these?
11
![Page 12: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/12.jpg)
Installing new slaves
> For first 20 or so slaves, we did it manually Insert CD, click, type, click, type, click, … But that doesn’t scale
> Then we automated Available as “Hudson PXE Plugin”
12
![Page 13: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/13.jpg)
Automated System Installations
13
> Slaves Power on, hit F12 PC boots from network (PXE)
> Hudson + PXE plugin ISO images of OS
![Page 14: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/14.jpg)
Automated System Installations
14
> Slaves Power on, hit F12 PC boots from network (PXE) Choose OS from menu Installs non-interactively
> Hudson + PXE plugin ISO images of OS
Your corporate IT guy & his DHCP server
![Page 15: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/15.jpg)
Automated System Installations
> Supports OpenSolaris, Ubuntu, CentOS, Fedora Trivial with most Linux
> Cooperate with Windows, too
> Quite useful outside Hudson, too No more broken CD drives No more CD-Rs
15
![Page 16: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/16.jpg)
Distributed builds with Hudson
> Master Serves HTTP requests Stores all important info
> Slaves 170KB single JAR Assumed to be unreliable Scale to at least 100
> Link Single bi-di byte stream No other requirements
16
![Page 17: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/17.jpg)
How master and slaves start talking
> For Unix slaves, via SSH Only need SSH and JRE on slaves We just need a host name
17
![Page 18: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/18.jpg)
How master and slaves start talking
> For Windows, DCOM We just need admin user name and password No manual intervention Works even from Unix masters
18
![Page 19: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/19.jpg)
How master and slaves start talking
> Via Java Web Start When master cannot see slaves A separate socket connection is made
19
![Page 20: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/20.jpg)
Automating JNLP launch
> Once started, can be installed as Windows service
20
![Page 21: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/21.jpg)
Automating JNLP launch
> Emulate the JNLP client headless
21
$ java -jar slave.jar -jnlpUrl URL
![Page 22: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/22.jpg)
Automated Tool Installation - JDK
> JDK from http://java.sun.com/ Hudson automatically chooses the right bundle Always up to date with new releases
22
![Page 23: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/23.jpg)
Automated Tool Installation - Apache
> Ant and Maven from Apache
23
![Page 24: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/24.jpg)
Automated Tool Installation - Custom
> Download arbitrary archive and unpack
24
![Page 25: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/25.jpg)
Automated Tool Installation - Custom
> Run arbitrary shell commands Can have variants by OS
25
![Page 26: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/26.jpg)
Automated Tool Installation - Extensible
> Write your own Simple Hudson extension Just write Java code to create tool on slave
> In progress: SCMs – Mercurial, … Install from Subversion
26
![Page 27: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/27.jpg)
Heterogeneous Cluster Challenge
> Your builds/tests need to run in specific environment
> Dependency on individual nodes hurts utilization
27
WombatWindows test
WombatWindows test
Hudson Windows test
Hudson Windows test
Windows #1
Windows #1
jobs slaves
GlassFishWindows test
GlassFishWindows test
Windows #2
Windows #2
Solaris#1
Solaris#1
Hudson Solaris test
Hudson Solaris test
![Page 28: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/28.jpg)
Labels to rescue
> Label is a group of slaves> Tie jobs to labels
28
WombatWindows test
WombatWindows test
Hudson Windows test
Hudson Windows test
Windows #1
Windows #1
jobs slaves
GlassFishWindows test
GlassFishWindows test
Windows #2
Windows #2
Solaris#1
Solaris#1
Hudson Solaris test
Hudson Solaris test
WindowsWindows
SolarisSolaris
Windows #3
Windows #3
![Page 29: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/29.jpg)
Forecasting failures
> Hudson monitors key health metrics of slaves Low disk space, insufficient swap Clock out of synch Extensible
> Slaves go offline automatically> Catch problems before they break builds
30
![Page 30: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/30.jpg)
Clean up mess after builds
> Kill runaway processes Daemons, background processes left by your build Works on Windows, Linux, Mac, and Solaris
31
![Page 31: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/31.jpg)
Load Statistics Monitoring
32
![Page 32: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/32.jpg)
When it’s time to add more slaves
33
![Page 33: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/33.jpg)
Hudson made this extensible
> Hudson detects excessive workload> Hudson notifies plugins> Plugins can provision more slaves
… assuming that you have that infrastructure
34
![Page 34: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/34.jpg)
35
![Page 35: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/35.jpg)
Amazon EC2: The Good
> Pay as you go (10¢/h or so) Loads on Hudson tend to be spiky
> Programmable API> Instances launch at machine-speed> EC2 instances are forgetful
36
![Page 36: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/36.jpg)
Amazon EC2: The Bad
> Your data is still inside your firewall Takes time to check out code … or to archive build artifacts Some data just can’t be moved
> EC2 instances are forgetful> Can your tests run in parallel?
37
![Page 37: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/37.jpg)
Hudson EC2 plugin
> Built on top of typica*> What does it do?
Automatically provisions slaves on EC2 on demand Picks the right AMI depending on demand Starts slave agent Shuts down unused instances
38* http://code.google.com/p/typica/
![Page 38: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/38.jpg)
Putting it all together
39
time
# of
exe
cuto
rs
capacity
usagequeue length
![Page 39: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/39.jpg)
Hudson “Appliance” on EC2
> Run the master in the cloud too, if you like Hudson on stock OpenSolaris AMI Data stored persistently in Elastic Block Storage
Dynamically expandable thanks to ZFS Online, too
> Packaged as a wizard
40
![Page 40: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/40.jpg)
41
![Page 41: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/41.jpg)
42
![Page 42: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/42.jpg)
> Hudson Hadoop plugin Just a few mouse clicks to install Turn every Hudson slave into a Hadoop node
> Distributed file system Automatic data replication (fault tolerant) Nice for storing old artifacts, logs, test records, …
> Map/reduce framework Large scale test results analysis / datamining More interesting work to be done in the future
43
![Page 43: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/43.jpg)
Hudson Selenium Plugin
> Selenium Tests webapps by scripting browsers
> Selenium Grid Runs Selenium over a grid of computers
> Allow Hudson labels to specify where to start browsers
> Hudson & Selenium both need heterogeneous cluster
44
![Page 44: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/44.jpg)
Selenium Grid
45
Hudson master(selenium hub)
Hudson slaves
![Page 45: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/45.jpg)
Conclusion
> CI is here to stay We’ll continue to push more workload to servers
> Hudson makes this easy for you> Reap the benefit of a cluster in multiple ways
46
![Page 46: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/46.jpg)
Resources
> http://hudson.dev.java.net/
> BOF-5105 “Hudson Community Meet up” Today 7:45pm same room
> Hudson booth inside Sun Pavilion
> Support Subscription [email protected]
47
![Page 48: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/48.jpg)
Agenda
> Quick Hudson introduction> Doing distributed builds> How Hudson make distributed builds easier> Taking infrastructure to cloud
49
![Page 49: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/49.jpg)
Setting up slaves
> Keeping slaves consistent is a good thing Particularly hard on heterogeneous environment
> General system administration tasks Network configuration Package installations for native tools Tools like Puppet or cfEngine are supposed to help
> Install build tools in the cluster Prepare tools on one file system rsync to everywhere
> This part of Hudson needs improvements50
![Page 50: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/50.jpg)
Hudson EC2 plugin usage
> Tell Hudson your AWS account information
51
![Page 51: Continuous Integration in the Cloud with Hudson](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f54550346895daa1596/html5/thumbnails/51.jpg)
Hudson EC2 plugin usage
> Tell Hudson what AMIs you want to start
52