![Page 1: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/1.jpg)
Marc Tollens Air France - KLM Process Mining Camp 2018
![Page 2: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/2.jpg)
![Page 3: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/3.jpg)
Marc Tollens
Digital Product Owner Air France - KLM Committee member Platform Digital Analytics
@marctollens /marctollens
![Page 4: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/4.jpg)
![Page 5: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/5.jpg)
![Page 6: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/6.jpg)
What is new today…
![Page 7: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/7.jpg)
is the new normal tomorrow.
![Page 8: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/8.jpg)
Customer expectations are getting higher every day.
And we have to keep up.
![Page 9: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/9.jpg)
Our way of working didn’t match the pace of the new reality.
Get requirements
Maintain
Create design
Implement
Verify
6 months
![Page 10: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/10.jpg)
We adopted a different approach.
Product owner Prioritises the work.
Developer Builds the product.
Scrum master Guards the process.
QA Tests the product.
![Page 11: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/11.jpg)
Velocity.
The measure of the amount of work a team can tackle during a single sprint.
![Page 12: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/12.jpg)
Build wheels
Build board
Attach the two
5
2
13
Velocity.
Feature: build a skateboard
![Page 13: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/13.jpg)
Yellow team
Internet check-in
1 Product owner 1 Scrum master
1 Designer 1 Tester
4 Developers
Meet the teams.
Green team
Trip management
1 Product owner 1 Scrum master
1 Designer 1 Tester
3 Developers
Orange team
Profile
1 Product owner 1 Scrum master
1 Designer 1 Tester
6 Developers
![Page 14: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/14.jpg)
So…
We have very talented scrum teams, however our velocity feels quite low.
Any idea why?
![Page 15: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/15.jpg)
And this is the only clue you get.
![Page 16: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/16.jpg)
Enter process mining.
Once something becomes digital, it becomes measurable. Once something is measurable, you can improve it.
![Page 17: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/17.jpg)
The approach
Define scope.
Mine the activity data from JIRA.
Cleaning.
Visualise process.
Start the conversation.
A predefined set of sprints.
Only include items that were started and ended during those sprints.
Only include activities that happen after a story went into progress.
![Page 18: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/18.jpg)
Team yellow process map
![Page 19: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/19.jpg)
Team yellow process map
![Page 20: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/20.jpg)
Open In progress In review In QA Done
77% of the stories are completed in a timeframe that fits within a sprint
62% of the stories are not completed in a sprint
40 working hours it takes to QA stories.
55% to next step 18 working hours till next step
60% to next step 40 working hours till next step
100% to next step 19 working hours till next step
Throughput rates and times looking only at the direct steps in the ideal development flow
Team yellow outcome.
![Page 21: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/21.jpg)
Team green process map
![Page 22: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/22.jpg)
Team green process map
![Page 23: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/23.jpg)
Open In progress In review In QA Done
11 working days the average time it takes to complete a user story.
0% of stories followed the ideal development path without any deviation.
66% of stories followed the ideal development path without any deviation when ignoring those who were only moved a sprint.
17% to next step 52 working hours till next step
50% to next step 61 working hours till next step
80% to next step 56 working hours till next step
Throughput rates and times looking only at the direct steps in the ideal development flow
Team green outcome.
![Page 24: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/24.jpg)
Team orange process map
![Page 25: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/25.jpg)
Team orange process map
![Page 26: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/26.jpg)
Open In progress In review In QA Done
17% of the stories went from progress to review within a sprint without deviation.
53% of the stories were adjusted (estimations, content & design) after they went in
progress .
6 working days the average time it takes to complete a user story.
17% to next step 2 working hours till next step
67% to next step 2 working hours till next step
100% to next step 13 working hours till next step
Throughput rates and times looking only at the direct steps in the ideal development flow
Team orange outcome.
![Page 27: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/27.jpg)
In progress In review In QA Done55% to next step 18 hours till next step
60% to next step 40 hours till next step
100% to next step 19 hours till next step
In progress In review In QA Done
17% to next step 52 hours till next step
50% to next step 61 hours till next step
80% to next step 56 hours till next step
Orange team
Yellow team
Green team
In review In QA Done
17% to next step 2 hours till next step
67% to next step 2 hours till next step
100% to next step 13 hours till next step
In progress
Comparing the teams.
![Page 28: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/28.jpg)
Maturity, stability and preparation matters.
![Page 29: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/29.jpg)
QA overload > Slicing up user stories in subtasks.
QA overload > Review scope acceptance criteria.
Deviating development paths > Not ready, not in sprint.
Suggested improvements.
![Page 30: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/30.jpg)
Team yellow process map period 1
![Page 31: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/31.jpg)
Team yellow process map period 2
![Page 32: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/32.jpg)
Team orange process map period 1
![Page 33: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/33.jpg)
Team orange process map period 2
![Page 34: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/34.jpg)
The good -Getting data out of JIRA was easy, either manually or via the API. -Lots of variables available. -Visualisations help non-data savvy people understand what is happening. -JIRA projects for the different teams followed a uniform process.
The bad -Deciding on what keep (e.g. comments like “Is available on test environment”). -Lots of activities during the planning session.
The ugly -Dealing with flexible working hours and working remotely (timezones). -Lack of timely JIRA administration by scrum team.
![Page 35: Marc Tollens Air France - KLM Process Mining Camp 2018 · 2020. 1. 6. · QA overload > Slicing up user stories in subtasks. QA overload > Review scope acceptance criteria. Deviating](https://reader034.vdocuments.site/reader034/viewer/2022051408/600791146dca614f7a43230a/html5/thumbnails/35.jpg)
Take-aways.
Pictures say more than words.
Data is black or white, reality is not.
Don’t offer solutions, pave the way instead.
Once a measure becomes a target, it ceases to be a good measure.