continuous integration: a practical approach
DESCRIPTION
ESUG 2012, GhentTRANSCRIPT
![Page 1: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/1.jpg)
CONTENTS
MediaGeniX Con,nuous integra,on § Context § Current system § Tools
CONTINUOUS INTEGRATION
Elke Ma@hijs and Maïkel Vandorpe elke.ma@[email protected]
![Page 2: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/2.jpg)
MEDIAGENIX
Our Product Our Customers Technical Product Informa,on
![Page 3: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/3.jpg)
PRODUCT COMPANY 20 YEARS OF WHATS’ON
![Page 4: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/4.jpg)
![Page 5: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/5.jpg)
Halloween Next Year
![Page 6: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/6.jpg)
![Page 7: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/7.jpg)
![Page 8: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/8.jpg)
![Page 9: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/9.jpg)
![Page 10: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/10.jpg)
![Page 11: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/11.jpg)
![Page 12: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/12.jpg)
![Page 13: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/13.jpg)
![Page 14: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/14.jpg)
Halloween next year
![Page 15: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/15.jpg)
CUSTOMERS
![Page 16: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/16.jpg)
Product (15.000 classes)
MTV
VRT RTBF
VT4
VTM
YLE
TV2 NRK
Pro7
MANY CUSTOMIZATIONS
![Page 17: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/17.jpg)
90 Employees § 30 SoZware Developers
§ 6 Project Managers
§ 13 Func,onal Analysts
§ 17 Customer Support Analysts
§ Other …
![Page 18: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/18.jpg)
Applica,on Overview § 1.9 Million lines of code
§ 15.000 own classes
§ 200.000 methods
§ 1000 database tables
![Page 19: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/19.jpg)
§ 35 customers wan,ng different things
§ They want it tomorrow
§ 30 developers ‘spawning’ code
§ Short ,me to market
=> Con,nuous Integra,on
![Page 20: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/20.jpg)
CONTINUOUS INTEGRATION
Context Current tes,ng system Tools
![Page 21: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/21.jpg)
Con$nuous integra$on § Many isolated changes § Many integra,ons § Each is immediately tested § Detect and fix problems soon
![Page 22: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/22.jpg)
Con$nuous integra$on
Development teams integrate frequently
§ Avg: 70/day
Tes,ng system
§ Each integra,on is automa,cally built and tested
Efficiently detect and fix problems
Significant reduc,on of integra,on problems
Develop cohesive soZware even in a limited ,me frame
![Page 23: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/23.jpg)
What do we test? As much as possible
§ Unit tests
§ Consistency checks § UI tests and Acceptance tests in general § Refactoring tests
Sequen,al running of tests take over 12 hours
![Page 24: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/24.jpg)
Code consistency checks
Check if code guidelines have been followed
Why ?
§ Consistent code makes life easy
§ Reduce number of bugs
![Page 25: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/25.jpg)
Code consistency checks
TopApplica?on
Applica,on A
Applica,on B
Applica,on C
buildMenuBar
| menu | (menu := Menu new)
addItem: self fileMenuItem; addItem: self editMenuItem; addItem: self toolsMenuItem.
self addMenuBarItemsTo: menu. menu
addItem: self viewMenuItem; addItem: self windowsMenuItem.
^menu
![Page 26: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/26.jpg)
Code consistency checks
test_topApplica,on_buildMenuBar "Don't implement #buildMenuBar. use #addMenuBarItemsTo:"
self assert: (self implementorsOf: #buildMenuBar
inSubclassesOf: TopApplica,on) isEmpty.
TopApplica?on
Applica,on A
Applica,on B
Applica,on C
![Page 27: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/27.jpg)
UI tests and Acceptance tests
Why ?
§ Unit test
§ Test a method
§ Change the method, change the test
§ Acceptance test § Emulate a user process
§ More stable tests
=> Never change func,onality, unless user asks for it
![Page 28: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/28.jpg)
UI Tes,ng – A Person Editor
![Page 29: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/29.jpg)
test_editPerson | person| person := Person withFirstName: 'Angelina' andLastName: 'Jolie'. self deny: (Person existsWhere: [:p| (p firstName = 'Bruce') & (p lastName = 'Willis')]). self openPersonEditorOn: person. self editField: #lastName value: 'Willis'. self editField: #firstName value: 'Bruce'. self toolBarPress: #saveCommand. self assert: (Person existsWhere: [:p | (p firstName = 'Bruce') & (p lastName = 'Willis')])
![Page 30: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/30.jpg)
test_editPerson | person| person := Person withFirstName: 'Angelina' andLastName: 'Jolie'. self deny: (Person existsWhere: [:p| (p firstName = 'Bruce') & (p lastName = 'Willis')]). self openPersonEditorOn: person. self editField: #lastName value: 'Willis'. self editField: #firstName value: 'Bruce'.
self toolBarPress: #saveCommand. self assert: (Person existsWhere: [:p | (p firstName = 'Bruce') & (p lastName = 'Willis')])
![Page 31: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/31.jpg)
test_editPerson | person| person := Person withFirstName: 'Angelina' andLastName: 'Jolie'. self deny: (Person existsWhere: [:p| (p firstName = 'Bruce') & (p lastName = 'Willis')]). self openPersonEditorOn: person. self editField: #lastName value: 'Willis'. self editField: #firstName value: 'Bruce'.
self toolBarPress: #saveCommand. self assert: (Person existsWhere: [:p | (p firstName = 'Bruce') & (p lastName = 'Willis')])
![Page 32: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/32.jpg)
UI Tes,ng – A Person Editor
![Page 33: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/33.jpg)
test_editPerson | person| person := Person withFirstName: 'Angelina' andLastName: 'Jolie'. self deny: (Person existsWhere: [:p| (p firstName = 'Bruce') & (p lastName = 'Willis')]). self openPersonEditorOn: person. self editField: #lastName value: 'Willis'. self editField: #firstName value: 'Bruce'.
self toolBarPress: #saveCommand. self assert: (Person existsWhere: [:p | (p firstName = 'Bruce') & (p lastName = 'Willis')])
![Page 34: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/34.jpg)
UI Tes,ng – A Person Editor
![Page 35: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/35.jpg)
test_editPerson | person| person := Person withFirstName: 'Angelina' andLastName: 'Jolie'. self deny: (Person existsWhere: [:p| (p firstName = 'Bruce') & (p lastName = 'Willis')]). self openPersonEditorOn: person. self editField: #lastName value: 'Willis'. self editField: #firstName value: 'Bruce'.
self toolBarPress: #saveCommand. self assert: (Person existsWhere: [:p | (p firstName = 'Bruce') & (p lastName = 'Willis')])
![Page 36: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/36.jpg)
Refactoring tests
Why ?
Basic Whats’On
Whats’On for a customer
=> Important during upgrades
Method X Ac?on A
Method X Ac?on B
Call X Press Bu@on
Method Y Call Y Ac?on C
![Page 37: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/37.jpg)
Refactoring tests
test_methodX_deprecated
"use methodY" self assertNoImplementorsOf: ‘methodX‘. self assertNoReferencesTo: ‘methodX’.
![Page 38: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/38.jpg)
Goals § Get test results as fast as possible
§ Alert ASAP when build is broken
§ Emulate actual users
§ SLA: Priority 1 bug
=> deliver bugfix within 1 day
![Page 39: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/39.jpg)
Challenges § Tests take over 12 hours to run
§ No out-‐of-‐the-‐box solu,on ⇒ We Built a distributed system ourselves
§ Mul,ple tests on the same DB
![Page 40: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/40.jpg)
Mul$ple tests on same DB § Problem: You assume a ‘clean DB’
when you start running a test
§ Solu$on: Take snapshot before test, restore snapshot aZer test
![Page 41: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/41.jpg)
Current Tes,ng System
Image Creator
4 Test Coordinators 4 x16 Test Slaves
Test Master
![Page 42: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/42.jpg)
Test Master
Store .
Tes,ng a version § Step by step:
§ The version has to be loaded
§ Create a database § The version has to be tested § The developer gets the results
§ The test master orchestrates this whole process
![Page 43: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/43.jpg)
Image Creator
Test Master
Image Creator
§ TM instructs IC to load your version
§ Get customers code from Store
§ AZer loading § Puts the new image on a fileserver
§ IC no,fies TM he’s done
To Load
…
…
Your version
To Load
Your version
…
…
To Test
…
…
Your version
Store .
To Load
…
Your version
…
To Load
…
…
To Test
…
Your version
…
To Test
…
…
![Page 44: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/44.jpg)
To Load
…
…
Test Coordinator
§ TM instructs a TC to start tes,ng a version
§ A TM is responsible for 1 whole version
4 Test Coordinators
Test Master
To Test
Your version
…
![Page 45: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/45.jpg)
1 of the 4 Test Coordinators
16 Test Slaves
Test Slave
§ TC orders TSs to copy image and create a local database
§ When database ready, start tes,ng!
§ Test Class per Test Class
§ Report back to TC when tes,ng of 1 Class is done
and receive a new class un,l finished
Test Master
![Page 46: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/46.jpg)
![Page 47: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/47.jpg)
Image Creator
16 Test Slaves
1 of the 4 Test Coordinators
Op,miza,ons
§ Parallel vs sequen,al § IC can load 4 versions simultaneously § 4 TCs 4 Versions are tested simultaneously § 16 TSs Tes,ng 1 version is much faster
Test Master
![Page 48: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/48.jpg)
Op,miza,ons
§ Dialog ‘Would you like to schedule the tests’
§ Only test latest available version
§ TM queues can be manipulated
§ Priori,ze § Unschedule
Test Master
To Load
Version z for customer C
…
To Test
Version x for customer A
Version y for customer B
…
To Test
Version y for customer B
Version (x+1) for customer A
…
To Test
Version (x+1) for customer A
Version y for customer B
…
To Test
Version x for customer A
Version y for customer B
Version (x+1) for customer A
…
![Page 49: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/49.jpg)
Image Creator
Op,miza,ons
§ Time-‐outs
§ IC aZer 30minutes
§ TC aZer 75minutes
§ TS aZer +/-‐ 10minutes
![Page 50: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/50.jpg)
16 Test Slaves
Op,miza,ons
§ Assign test classes in a smart order
§ ‘Nuke’ tests
![Page 51: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/51.jpg)
Test Results
§ What? Timing?
§ Build failed < 45minutes
§ AZer tests have successfully run § X total tests, Y failures, Z errors, T not run
§ With wai,ng in a queue +/-‐ 4hours
§ Without wai,ng in a queue < 1hour
§ Saved on Store!
![Page 52: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/52.jpg)
Con,nuous integra,on – VERA/Seaside interface
![Page 53: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/53.jpg)
![Page 54: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/54.jpg)
Con,nuous integra,on – VERA/Seaside interface
![Page 55: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/55.jpg)
VERA/Seaside interface
![Page 56: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/56.jpg)
VERA/Seaside interface
![Page 57: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/57.jpg)
Integrated Tools § Open SUnit on failed tests
§ Find origin failing test
![Page 58: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/58.jpg)
Demo
![Page 59: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/59.jpg)
Goals § Get test results as fast as possible
§ Alert ASAP when build is broken
§ Emulate actual users
§ SLA: Priority 1 bug
=> deliver bugfix within 1 day
![Page 60: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/60.jpg)
Conclusion Goals have been met!
§ Build with report on broken builds within 45 minutes
§ Test results can come in within 1 hour § Prio 1 bugs can be fixed within a day
§ ‘Reasonable’ ,me for results § 4 hours on average
§ Bonus: integrated tools on SUnit
![Page 61: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/61.jpg)
Conclusion Future areas of improvement …
§ Test every version § Be@er pin-‐point where an error started
§ Speed up through-‐put of unpriori,sed tests
![Page 62: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/62.jpg)
Ques,ons?
![Page 63: Continuous Integration: a practical approach](https://reader034.vdocuments.site/reader034/viewer/2022052321/549c6606b47959cf318b4722/html5/thumbnails/63.jpg)
Q: Which customisa,ons do customers want? A: See next presenta,on Q: Why does the product need to change that oZen? A: Technical evolu,on, target different markets, new regula,ons, …