expert talks cardiff 2017 - keeping your ci-cd system as fast as it needs to be
TRANSCRIPT
![Page 1: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/1.jpg)
Abraham Marin-Perez@AbrahamMarin
fromfragiletoagile.com
Keeping Your CI / CD Pipeline as Fast as It Needs to Be
#ExpertTalks @AbrahamMarin @EqualExperts
![Page 2: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/2.jpg)
![Page 3: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/3.jpg)
![Page 4: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/4.jpg)
Abraham Marin-Perez@AbrahamMarin
fromfragiletoagile.com
Keeping Your CI / CD Pipeline as Fast as It Needs to Be
#ExpertTalks @AbrahamMarin @EqualExperts
![Page 5: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/5.jpg)
About Me
![Page 6: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/6.jpg)
About Me
![Page 7: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/7.jpg)
About Me
![Page 8: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/8.jpg)
About Me
![Page 9: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/9.jpg)
About Me
![Page 10: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/10.jpg)
About Me
![Page 11: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/11.jpg)
About This Talk
![Page 12: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/12.jpg)
Continuous Integration: check everything is still working after every commit
Continuous Deployment: every successful commit turns into a release
About This Talk
![Page 13: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/13.jpg)
About This Talk
![Page 14: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/14.jpg)
About This Talk
![Page 15: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/15.jpg)
About This Talk
![Page 16: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/16.jpg)
About This Talk
![Page 17: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/17.jpg)
![Page 18: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/18.jpg)
SUPER APP
# Files: 75# Tests: 800Build Time: 4 minOutput: superapp.war
![Page 19: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/19.jpg)
SUPER APP
# Files: 113# Tests: 1200Build Time: 6 minOutput: superapp.war
![Page 20: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/20.jpg)
SUPER APP
# Files: 169# Tests: 1800Build Time: 9 minOutput: superapp.war
![Page 21: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/21.jpg)
Slow feedback
Broken builds mask issues
Development paralysis
Impact on ability to meet our SLAs
Pay per use
Missed business opportunities
The Problems Of Size
![Page 22: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/22.jpg)
Live with it
Partial CD: only quick tests
Phased CD: split into components
Test Deprecation Policy
Microservices
How Organisations Manage Size
![Page 23: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/23.jpg)
Microservices
![Page 24: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/24.jpg)
Microservices
![Page 25: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/25.jpg)
Microservices
![Page 26: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/26.jpg)
![Page 27: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/27.jpg)
![Page 28: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/28.jpg)
SUPER APP
# Files: 169# Tests: 1800Build Time: 9 minOutput: superapp.war
APP BACKEN
DSUPER
APP
# Files: 115# Tests: 1200Build Time: 6 minOutput: superapp.war
# Files: 72# Tests: 800Build Time: 4 minOutput: appbackend.jar
![Page 29: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/29.jpg)
Microservices?
![Page 30: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/30.jpg)
Microservices
![Page 31: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/31.jpg)
Microservices
![Page 32: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/32.jpg)
Microservices
![Page 33: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/33.jpg)
Microservices
![Page 34: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/34.jpg)
Microservices
![Page 35: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/35.jpg)
Microservices
![Page 36: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/36.jpg)
Build Pipeline Becomes a Network
![Page 37: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/37.jpg)
https://goo.gl/LvkkRq
Scalable Continuous DeploymentWith Maven
![Page 38: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/38.jpg)
![Page 39: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/39.jpg)
A real case scenario
![Page 40: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/40.jpg)
Service
Service
Service
Parent POM
Logging
![Page 41: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/41.jpg)
Service
Service
Service
Parent POM
Logging
28%
![Page 42: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/42.jpg)
Service
Service
Service
Parent POM
Logging
28%28%
28%
![Page 43: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/43.jpg)
Service
Service
Service
Parent POM
Logging
28%28%
28%
20%
![Page 44: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/44.jpg)
Service
Service
Service
Parent POM
Logging
48%
28%
28%
20%
![Page 45: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/45.jpg)
Service
Service
Service
Parent POM
Logging
Highest runfrequency
Lowest runfrequency
![Page 46: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/46.jpg)
Service
Service
Service
Highest runfrequency
Lowest runfrequency
![Page 47: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/47.jpg)
Build Time (BT): time an individual build takes to run
Change Rate (CR): percentage of commits upon an individual build with respect to the whole system
Useful Metrics
![Page 48: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/48.jpg)
Service
Service
Service
Highest runfrequency
Lowest runfrequency
![Page 49: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/49.jpg)
Service
Service
Service
Parent POM
Logging
28%
![Page 50: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/50.jpg)
Impact Time (IT): total time to run a build and all the builds that will be triggered as a result
Useful Metrics
![Page 51: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/51.jpg)
No dependants IT(A) = BT(A)
A
Useful Metrics
![Page 52: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/52.jpg)
Serial execution IT(A) = BT(A) + IT(B) + IT(C)
B
A
C
Useful Metrics
![Page 53: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/53.jpg)
Parallel execution IT(A) = BT(A) + max(IT(B), IT(C))
B
A
C
Useful Metrics
![Page 54: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/54.jpg)
Service
Service
Service
Highest runfrequency
Lowest runfrequency
![Page 55: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/55.jpg)
Weighted Impact Time (WIT): impact time of a build weighted according to its change rage
WIT(A) = IT(A) * CR(A)
Useful Metrics
![Page 56: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/56.jpg)
Average Impact Time (AIT): total time needed, on average, to execute all necessary builds after any given commit anywhere in the system
AIT = WIT(A) + WIT(B) + ... + WIT(Z)
Useful Metrics
![Page 57: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/57.jpg)
Sample Thresholds
![Page 58: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/58.jpg)
Average Impact Time
Average Impact Time is what indicates how well you have scaled your system
Sample Thresholds
![Page 59: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/59.jpg)
Maximum Impact Time
In a worst-case scenario, a build won’t take longer than this.
Sample Thresholds
![Page 60: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/60.jpg)
Maximum Impact Time for Critical Components
The same, but only for your most sensitive modules (log-in, payment gateway, etc.)
Beware of dependencies!
Sample Thresholds
![Page 61: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/61.jpg)
Service
Service
Service
Highest runfrequency
Lowest runfrequency
![Page 62: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/62.jpg)
Manual processingtakes time...
![Page 63: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/63.jpg)
Most CI systems provide an API Calculations aren’t complex Multiple graphical tools available
Automating Build Analysis
![Page 64: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/64.jpg)
github.com/quiram/build-hotspotsBuild Hotspots
![Page 65: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/65.jpg)
https://commons.wikimedia.org/wiki/File:2012_Italian_GP_-_Lotus_wheel.jpg
![Page 66: Expert Talks Cardiff 2017 - Keeping your ci-cd system as fast as it needs to be](https://reader035.vdocuments.site/reader035/viewer/2022070602/587d866a1a28abcd648b52e3/html5/thumbnails/66.jpg)
Thank You
@EqualExperts
equal-experts
equalexperts.com
Thank You
fromfragiletoagile.com@AbrahamMarin
#FastCI