©2013 cloudbees, inc. all rights reserved 1 jenkins scalability summit oct 2013
TRANSCRIPT
©2013 CloudBees, Inc. All Rights Reserved 1©2013 CloudBees, Inc. All Rights Reserved
Jenkins Scalability SummitOct 2013
©2013 CloudBees, Inc. All Rights Reserved 2
Logistics
• Where is …?• Network
©2013 CloudBees, Inc. All Rights Reserved 3
Agenda Part 1: Story time
• Learn from our collective experience• Identify high-priority problems• What is hurting serious users?• Concrete details are good
©2013 CloudBees, Inc. All Rights Reserved 4
Agenda Part 2: Discuss & Design
• Collectively pick a few topics• Split into 2 tracks
• See if we can start shaping up solutions
• Please keep shared notes– Details!
©2013 CloudBees, Inc. All Rights Reserved 5
Possible topics includes …
• CPU/memory/disk/network/… consumption• Stability/diagnosability of slaves• Access control of builds/UI• Organizing jobs and build records• Master to master communication• Workflow / choreography• User interface• Stability of Jenkins releases• Plugin compatibilities
©2013 CloudBees, Inc. All Rights Reserved 6
Goals
• No one goes listen-only mode• Produce notes to show to the broader
community• Figure out how to do this better next
year
©2013 CloudBees, Inc. All Rights Reserved 7©2013 CloudBees, Inc. All Rights Reserved
What We’ve Done&
What We Can
©2013 CloudBees, Inc. All Rights Reserved 8
Thread Reduction
• Per-slave consumption– SSH (down to 1)– Channel (down to 1)
• Executor thread on demand
©2013 CloudBees, Inc. All Rights Reserved 9
Thread Reduction: TODO
• NIO• Asynchronous job execution–More about this in workflow
©2013 CloudBees, Inc. All Rights Reserved 10
Memory reduction
• Lazy loading of build records• Database plugin
©2013 CloudBees, Inc. All Rights Reserved 11
Design Choices That Constrain Us
• Compatibility– Adding is doable, changing is often hard
• Heterogeneousness in data model• Thread-driven execution model
©2013 CloudBees, Inc. All Rights Reserved 12©2013 CloudBees, Inc. All Rights Reserved
CloudBees Scalability Efforts
©2013 CloudBees, Inc. All Rights Reserved 13
Multi-master / Meta Jenkins
• Master-to-master channel• Extensions on top of it– “Cloud” impl to lease slaves– Push security realm– Push update center–…
©2013 CloudBees, Inc. All Rights Reserved 14
Multi-master
• Where we think we can go with this–Move jobs around– Sharding with UI mashup
• Where this wouldn’t take us– Loss of inflight builds– True horizontal scaling
©2013 CloudBees, Inc. All Rights Reserved 15
Workflow
• Brand new job type• Kill multiple birds in one stone• Inspired by buildflow & jenkow
©2013 CloudBees, Inc. All Rights Reserved 16
Ingredients
• No single thread to execute the whole thing– Analogous to NIO– Check pointing
• BPMN-like workflow execution model– Surface syntax independent
• Groovy DSL–With continuation-passing style execution
model
• Post-execution visualization– But no pre-execution visualization
©2013 CloudBees, Inc. All Rights Reserved 17
Ingredients
• Unified properties, build variables, and environments
• Open up direct access to file store• Hands-free process forking• Interop with existing job types