take the red pill: how criteo revamped its software development process
DESCRIPTION
These are the slides of the talk I gave at the Global Scrum Gathering Berlin in 2014. It describes how Criteo managed to change the software engineering culture of its R&D department from 'Testing is cheating' to 22K+ tests being run every half hour.TRANSCRIPT
![Page 1: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/1.jpg)
TA K E T H E R E D P I L LD R . A D R I A N P E R R E A U D E P I N N I N C K
![Page 2: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/2.jpg)
T R E M E N D O U S G R O W T H . C R I T E O : G O G O G O ! ! !
• From 20 to 160 engineers in less than 5 years
• 160 code repositories
• 7 million lines of code
• Took months to release
![Page 3: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/3.jpg)
W E M U S T S E E R E A L I T Y T O P U T T H E M A C H I N E S U N D E R C O N T R O L
![Page 4: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/4.jpg)
W H AT W A S R E A L I T Y ?
• Average lead time for new features = 60 days
• Average time to merge commits = 20 days
• The release pipeline was blocked for months
• It was getting worse each year
![Page 5: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/5.jpg)
S O M E T H I N G H A D T O C H A N G EI T W A S G O I N G T O TA K E T O P M A N A G E M E N T T O A G R E E O N S O M E T H I N G L I K E T H I S
![Page 6: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/6.jpg)
H O W W O U L D Y O U S E L L T H I S I N I T I AT I V E ?
PA I R D I S C U S S I O N :
![Page 7: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/7.jpg)
O B J E C T I V E S
• Reduce commit merge time to hours instead of days
• Reduce time from commit to Prod to < 4 days
• Reduce number of cancelled release candidates
![Page 8: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/8.jpg)
T E C H N I C A L S O L U T I O N
• Tailor-made Build System
• Mandatory Code Reviews
• Continuous Integration Pipeline
• Sandbox to execute tests
![Page 9: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/9.jpg)
P R O C E S S F O L L O W E D
![Page 10: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/10.jpg)
E S TA B L I S H S E N S E O F U R G E N C Y
• Code freeze in 2012 crippled the team during months
• The release pipeline had been blocked for months
• Things weren’t getting better for 2013’s code freeze
![Page 11: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/11.jpg)
E N E R G Y W I L L D I S S I PAT EW I T H O U T A S T R O N G G U I D I N G C O A L I T I O N
![Page 12: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/12.jpg)
S T R O N G G U I D I N G C O A L I T I O N
• Executive Vice President of Engineering
• Vice President of QA
• 10 Software Engineers, high level of seniority
• 2 Senior Engineering Program Managers
![Page 13: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/13.jpg)
D E V E L O P I N G A C H A N G E V I S I O N
• Brainstormed for the key elements
• Prepared a document explaining the vision
• Shared the document with Key players
• Modified the document to take into account feedback
• Shared the document with Dev Leads
• Modified the document to take into account feedback
• Gave presentation explaining the project
• Sent document to all
![Page 14: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/14.jpg)
C H A N G E V I S I O N
• 26 page document
• Took over a month to get alignment in version 0.1
• Sent to 20 senior engineers for feedback
• Last version took into account all feedback
![Page 15: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/15.jpg)
C O M M U N I C AT E T H E V I S I O N
• Monthly updates at all hands meetings
• Weekly emails with progress reports
• Events with food
![Page 16: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/16.jpg)
E M P O W E R B R O A D - B A S E D A C T I O N
![Page 17: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/17.jpg)
E M P O W E R A C T I O N
• The improvement backlog was shared with everyone
• Some items prioritised by voting
• Voluntary-based work encouraged
• Test-a-thons and other wide range initiatives
![Page 18: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/18.jpg)
S H O R T- T E R M W I N S
• Used Scrum to guide the change process
• Developed usable tools from the beginning
• Developed one tool at a time
• Pushed adoption slowly until tipping point
• Forced rest to adopt once tipping point passed
![Page 19: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/19.jpg)
I N T E G R AT I O N T I M E S
![Page 20: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/20.jpg)
0.00%$
10.00%$
20.00%$
30.00%$
40.00%$
50.00%$
60.00%$
70.00%$
80.00%$
90.00%$
100.00%$
12/11/13$
12/18/13$
12/25/13$
1/1/14$
1/8/14$
1/15/14$
1/22/14$
1/29/14$
2/5/14$
2/12/14$
2/19/14$
2/26/14$
3/5/14$
3/12/14$
3/19/14$
T E S T P L A N
![Page 21: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/21.jpg)
P R O J E C T B U R N D O W N
![Page 22: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/22.jpg)
0"
10"
20"
30"
40"
50"
60"
70"
80"
90"
100"
5/13/13" 6/13/13" 7/13/13" 8/13/13" 9/13/13" 10/13/13" 11/13/13" 12/13/13" 1/13/14" 2/13/14" 3/13/14" 4/13/14" 5/13/14"
Commit&to&Prod&Lag&C O M M I T A G EF R O M 3 5 D A Y S T O 1 5 D A Y S
![Page 23: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/23.jpg)
D O N ’ T L E T G OI T ’ S A LW A Y S H A R D E R T H A N Y O U T H O U G H T
![Page 24: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/24.jpg)
L A R G E S T I M P E D I M E N T S
• Technical
• Eat your own dog food
• Replicating the Sandbox
• Human
• People added to the team not sharing vision
• Getting teams to use the tools
• Managing expectations and communication
![Page 25: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/25.jpg)
D O N ’ T L E T G O
• Fully dedicated team for a whole year.
• After the project ended the team remained as a new unit.
• Even through the hardest times management gave full support.
![Page 26: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/26.jpg)
I N C O R P O R AT E C H A N G E I N T O T H E C U LT U R E
![Page 27: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/27.jpg)
I N C O R P O R AT E C H A N G E I N T O T H E C U LT U R E
• Training program for new recruits
• Engineering partners are expected to conform to new way of working
• Training program for engineering partners
• Tools in place become the cultural gates
![Page 28: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/28.jpg)
T H E R E I S M O R E T O C H A N G E T H A N K O T T E R
O T H E R T E C H N I Q U E S W E U S E D
![Page 29: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/29.jpg)
B R I D G E B U I L D E RW E A S K E D T H O S E A L R E A D Y O N B O A R D T O H E L P O T H E R S D O T H E S A M E
![Page 30: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/30.jpg)
C H A M P I O N S K E P T I C
Find someone senior in the organisation:
• That wants the goal to be reached
• That doesn’t fully support your technical solution
• Talk with him regularly
![Page 31: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/31.jpg)
D E D I C AT E D C H A M P I O N
Y O U W O N ’ T G O FA R W I T H O U T O N E
![Page 32: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/32.jpg)
D O F O O DM A K E S U R E T O B R I N G S N A C K S T O G E T T O G E T H E R S
![Page 33: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/33.jpg)
E A R LY A D O P T E R SW I L L I N G T O T R Y T H I N G S B E F O R E T H E Y A R E R E A D Y
![Page 34: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/34.jpg)
E X T E R N A L VA L I D AT I O N
L I N K E D I N W E N T T H R O U G H T H E S A M E P R O C E S S
![Page 35: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/35.jpg)
G R O U P I D E N T I T Y
M A K E P E O P L E P R O U D O F B E I N G PA R T O F T H E C H A N G E
![Page 36: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/36.jpg)
I N Y O U R S PA C EW E U S E D L A R G E S C R E E N S T O S H O W P R O G R E S S
![Page 37: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/37.jpg)
I N V O LV E E V E R Y O N EI F Y O U W A N T L A S T I N G C H A N G E …
![Page 38: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/38.jpg)
U S I N G S C R U MA N D W E D I D I T A L L
![Page 39: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/39.jpg)
- A G E N T S M I T H
“Never send a human to do a machine’s job”
![Page 40: Take the Red Pill: How Criteo revamped its software development process](https://reader033.vdocuments.site/reader033/viewer/2022052617/547e5472b37959582b8b549d/html5/thumbnails/40.jpg)
R E F E R E N C E S
• Leading Change - John P. Kotter
• Fearless Change - Linda Rising, PhD
• Continous Delivery - Jez Humble, David Farley