Chair of Software Engineering for Business Information Systems (sebis) Faculty of InformaticsTechnische Universität Münchenwwwmatthes.in.tum.de

Observing the Learning Process of a Large-Scale Agile Development Program - A Case Study from the Technology SectorNiels Holz, 02.03.2020, Master’s Thesis Final Presentation

Motivation

Research Questions & Research Approach

FindingsCase Description

Adopted Agile Program

Identified Recurring Concerns

Identified Best Practices

Lessons Learned of Implementing Pattern Catalogue Best Practice

Limitations

Conclusion & Future Work

Outline

200302 Holz MT Final

Motivation

200302 Holz MT Final

Agile… deals with changing requirements and promises higher customer satisfaction through continuous delivery and high customer involvement [3,6]

Large-Scale… companies are inspired by the success of agile practices and apply agile practices to large-scale projects[9].

Challenges… arise through the large-scale adaption. Geographically-distributed teams, the scaling of agile practices and adoption to the project team are just a few[2,3,7,9].

Research… already identified 14 Stakeholder groups and 79 challenges[8]… offers Patterns to be applied for those stakeholder groups [5]

Patterns… as a structured solution to recurring problems [1,7]

Evaluation… of observed Patterns in research according to concerns and stakeholder groups. [7]

Motivation

Research Questions & Research Approach

FindingsCase Description

Adopted Agile Program

Identified Recurring Concerns

Identified Best Practices

Lessons Learned of Implementing Pattern Catalogue Best Practice

Limitations

Conclusion & Future Work

Outline

200302 Holz MT Final

Research Questions

200302 Holz MT Final

RQ1

RQ2

RQ4

RQ3

RQ5

How has LeSS been adopted and applied at Robert Bosch?

What are recurring concerns of stakeholders at the product organization of Robert Bosch?

Which bad practices should be avoided in the product organization of Robert Bosch?

What are good practices for addressing recurring concerns of stakeholders of the productorganization of Robert Bosch?

What are the lessons learned of implementing already observed best practices in the productorganization of Robert Bosch?

Research Approach

200302 Holz MT Final

1. Interviews for identifying adaptions of LeSS at agile program

2. Interviews identifying concerns and patterns with stakeholder groups (SM,Dev, PO) following Pattern-Based Research Design[1]

3. Select & instantiate solution according to Pattern-Based Research Design[1]

RQ1

RQ2

RQ4

RQ3

RQ5

[1]

Research Approach

200302 Holz MT Final

1. Interviews for identifying adaptions of LeSS at agile program

2. Interviews identifying concerns and patterns with stakeholder groups (SM,Dev, PO) following Pattern-Based Research Design[1]

3. Select & instantiate solution according to Pattern-Based Research Design[1]

3 interviews with agile-adoption questionnaire

11 interviews with concerns and patterns questionnaire

3 categories identified;2 patterns per category1 pattern implemented

[6]

[5]

Motivation

Research Questions & Research Approach

FindingsCase Description

Adopted Agile Program

Identified Recurring Concerns

Identified Best Practices

Lessons Learned of Implementing Pattern Catalogue Best Practice

Limitations

Conclusion & Future Work

Outline

200302 Holz MT Final

Adopted Agile Framework

200302 Holz MT Final

Case Description

GS/POI21Purchasing Integration, Lean and Unified Management

• 5 Purpose Teams• total of 52 project members

Findings

No. Role Duration Questionnaire1 Product Owner 1:06:21 h Concerns & Patterns2 Scrum Master 1:07:24 h Concerns & Patterns3 Product Owner 1:05:41 h Concerns & Patterns4 Scrum Master 1:23:33 h Agile-Adoption5 Product Owner 1:23:54 h Concerns & Patterns6 Product Owner 1:17:55 h Agile-Adoption7 Development Team 1:12:48 h Concerns & Patterns8 Product Owner 1:13:40 h Concerns & Patterns9 Development Team 1:02:52 h Concerns & Patterns10 Development Team 0:50:42 h Concerns & Patterns11 Product Owner 1:13:52 h Concerns & Patterns12 Development Team 1:18:29 h Concerns & Patterns13 Development Team 0:54:52 h Concerns & Patterns14 Scrum Master 1:12:02 h Agile-Adoption

200302 Holz MT Final

Interviews general Information

- 3 structured interviews using agile-adoption questionnaire

- 11 semi-structured interviews using concerns and patterns questionnaire

- Average time 1:10:09 h

Findings

200302 Holz MT Final

Adopted Agile Program Case Study Partner

Findings

200302 Holz MT Final

Purpose Team = Feature Team with Process Consultant and Product Owner Team

Scrum Master

Product Owner Team

Process Consultant (50/50)

Development & Test Team

Findings

200302 Holz MT Final

Newly identified Recurring Concerns

Recurring Concern Stakeholder CategoryC-79 How to balance amount and quality of delivered requirements Product Owner Quality Assurance

C-80 How to manage overarching backlog item prioritization with multiple product owners Product Owner Communication & Coordination

C-81 How to understand all interfaces and dependencies of the system Program Specific Knowledge Management

C-82 How to support an onboarding approach for different stakeholder groups Program Specific Knowledge Management

C-83 How to manage requirement development for multiple teams Product Owner Project Management

C-84 How to involve all team members in solution generation Scrum Master Culture & Mindset

C-85 How to share domain knowledge across agile teams Development Team Knowledge Management

C-86 How to involve remotely working/external colleagues Development Team Tooling

C-87 How to clarify details outside of meetings in cross-shore agile teams Development Team Communication & Coordination

Findings

200302 Holz MT Final

Identified Best Practices

Anti-PatternPrincipleMethodology Pattern

Viewpoint Pattern

Coordination Pattern

PatternID

NameOccurence

V-01Dependency

Matrix*

V-02Burn-Down Chart

*A-11

Don’t Capsulate Teams too much

*

A-1Don’t Use

Frameworks as Recipes

****

A-05 Don’t Forward Requirements

*

A-06Don’t Overshoot

Coordination Meetings

**

A-07Don’t Have New Year Resolution

Dilemma*

A-08Don’t Limit Knowledge Transfer to Knowledge Transfer-

Workshops*

A-09Don’t Misuse Estimation Creation

*

A-10Don’t Limit

External Colleagues

*

A-01Don’t Have

Blurred Boundaries

*

A-02Don’t Force Team

Coherence*

A-03 Don’t Assume Autonomous Onboarding

*

A-04 Don’t Assume

Mutual Terminology

*

V-03JIRA-Board

* V-05Power BI

*

V-04Interface

Architecture*

V-06Epic-Plan Game

Board*

M-01Reserved Capacity

*

M-02Scope Change

*

M-03Bug Prioritization

*

M-04Acceptance

Criteria

M-05Definition of Ready and

Definition of Done*

M-06Functional Splitting

+

M-07Process

Consultant*

M-08Purpose Teams

*

M-09Story Points

*

M-10Subtask-Testing

*

M-11Product Owner

Team*

M-12Shifting

Responsibility*

M-13Automation Lead

*

M-14Docupedia for Architecture

Documentation*

M-15Impact Analysis

*

M-16Incremental Onboarding

*

M-17Planning Poker

light*

M-18Proof of Concept

+

CO-1Community of

Practice****

CO-01Pre-Planning Coordination

*

CO-02Face-to-Face Knowledge

Transfer*

CO-03Exemplary Knowledge

Transfer*

CO-04Kickoff

*

CO-05Product Backlog

Refinement*

CO-06Direct Customer Communication

*

CO-07Regelrunden

*

CO-08Newsflash

*

CO-09Process

Consultant Meeting

*

CO-10Go-Live

Celebration*

CO-11Third Party Interface -

Planning Meeting*

P-01Geographically

Distributed Meeting Hours

*

P-02Velocity Sheet

*

P-03Avoid Extra

Meetings+

P-04Semi Co-Location

*

V-01

M-05

CO-01

P-03

C-19

C-26

C-60

C-31

C-85

C-69

C-37

C-68

C-36

C-53

C-76

C-63

C-86

C-87

Development Team

A-06

A-11

A-07

A-09

A-10

M-09

M-08

M-14

M-13

M-15

M-16

M-10

M-17

M-18

CO-03

CO-04

CO-05

V-03

V-04

V-06

M-12

A-08

CO-02

CO-09

M-07

P-04

FindingsFindingsMapping Concerns to Patterns for Development Team

Focus Concern Categories

200302 Holz MT Final

Culture & Mindset

13 Concerns (1 new)

1 Anti-Patterns1 Anti Pattern Candidate

Knowledge Management

8 Concerns (3 new)

5 Anti-Pattern Candidates

Quality Assurance

5 Concerns (1 new)

3 Anti-Pattern Candidates

Findings

200302 Holz MT Final

Lessons Learned Community of Practice for PO

Community of Practice for POs

Why?

Each agile team has several Product Owners

8 Concerns (3 newly identified)

5 Anti-Patterns

How?

Following CoP pattern identified by sebis-chair[7]

Addressing characteristics of a successful CoP byPaasivaara [4]

Comparing to interview information from CoPs at case study partner

Findings

200302 Holz MT Final

Lessons Learned Community of Practice for PO

Findings

200302 Holz MT Final

Lessons Learned Community of Practice for PO contd.

Findings

200302 Holz MT Final

Lessons Learned contd.

Motivation

Research Questions & Research Approach

FindingsCase Description

Adopted Agile Program

Identified Recurring Concerns

Identified Best Practices

Lessons Learned of Implementing Pattern Catalogue Best Practice

Limitations

Conclusion & Future Work

Outline

200302 Holz MT Final

Limitations

200302 Holz MT Final

Several new best practices identified

Only two patterns identified

Longer time horizon needed for impactful observation and adaption of patterns of thepattern catalogue

Only action plan observed No certainty whether the implemented pattern and the planned to be

implementend patterns will help solve the concerns or are continually applied

Motivation

Research Questions & Research Approach

FindingsCase Description

Adopted Agile Program

Identified Recurring Concerns

Identified Best Practices

Lessons Learned of Implementing Pattern Catalogue Best Practice

Limitations

Conclusion & Future Work

Outline

200302 Holz MT Final

Conclusion & Future Work

Key Findings• Introduction of new role Process Consultant creates an

openly admitted internal silo, however, the role helps addressing concerns in Knowledge Management.• Domain knowledge with single point of contact• Mini-Architects as part of team

• Kickoffs and similar events, bringing together geographically distributed teams, are vital

• Patterns of the pattern catalogue address concerns associated with them

• Willingness for adopting patterns from other organizations is high

• Out of the six presented patterns, one was implemented and three more are planned to be implemented in the future

Future Work• Actively implement several patterns at case study partner

and observe for a longer time horizon• Observing the lessons learned for multiple patterns

• Application of more patterns addressing concerns of other stakeholder groups (Agile Coach, …)

• Further validation of patterns identified in practice at case study partner

200302 Holz MT Final

Key ArtifactsLSAD framework for case study partner, 54 Concerns identified, +9 new Concerns identified, 1 new role (PC), 1 Anti-Pattern,1 CO-Pattern, 50 Pattern-Candidates identified( 4 Principle, 11 Anti-Patterns, 11 CO- , 18 M- and 6 V-Pattern-Candidates) ,1 Pattern successfully validated

[1] Buckl, S., Matthes, F., Schneider, A. W., & Schweda, C. M. (2013). Pattern-based design research–an iterative research method balancing rigor and relevance. In International Conference on Design Science Research in Information Systems (pp. 73-87). Springer, Berlin, Heidelberg.[2] Dingsøyr, T., & Moe, N. B. (2014). Towards principles of large-scale agile development. In International Conference on Agile Software Development (pp. 1-8). Springer, Cham.[3] Larman, C., & Vodde, B. (2009). Scaling lean & agile development. Organization, 230(11).[4] Paasivaara, M., & Lassenius, C. (2014). Communities of practice in a large distributed agile software development organization–Case Ericsson. Information and Software Technology, 56(12), 1556-1577.[5] Scaling-Agile Hub. Patterns.https://scaling-agile-hub.sebis.in.tum.de/#/patterns.[6] The LeSS Company B.V. (2014). Overview – Large Scale Scrum(LeSS). https://less.works/.[7] Uludag, Ö., Harders, N. M., & Matthes, F. (2019, July). Documenting recurring concerns and patterns in large-scale agile development. In Proceedings of the 24th European Conference on Pattern Languages of Programs (pp. 1-17).[8] Uludag, Ö., Kleehaus, M., Caprano, C., & Matthes, F. (2018). Identifying and structuring challenges in large-scale agile development based on a structured literature review. In 2018 IEEE 22nd International Enterprise Distributed Object Computing Conference (EDOC) (pp. 191-197). IEEE.[9] VersionOne, C. 13th Annual State of Agile Report. (2019).

References

200302 Holz MT Final

Technische Universität MünchenFaculty of InformaticsChair of Software Engineering for Business Information Systems

Boltzmannstraße 385748 Garching bei München

Tel +49.89.289.Fax +49.89.289.17136

wwwmatthes.in.tum.de

Niels HolzB.Sc.

17132

matthes@in.tum.de

BACKUP SLIDES

Responsibilities and Challenges Scrum Master, Development Team and Product Owner (1/2)

200302 Holz MT Final

Scrum Master Development Team

Responsibilities • Part of agile team• Enable development process• Remove impediments• Promote agile program• Manage meetings

• Part of agile team• Clarification non-functional requirements*• Estimating user stories• Architecture and design • Implementation • Sprint Review

Challenges 1. How to deal with incorrect agile practices2. How to provide sufficient tools and infrastructure for remote

communications3. How to deal with increasing workload of key stakeholders4. How to synchronize working hours of cross-shore meetings5. How to create a culture of continuous improvement6. How to rearrange physical spaces7. How to deal with closed mindedness8. How to deal with increased efforts by establishing inter-team

communication9. How to deal with lacking sense of ownership responsibilities for developed

services10. How to define clear roles and responsibilities11. How to establish a common understanding of agile software development12. How to deal with cultural differences between cross-shore agile teams13. How to encourage development teams to talk about tasks and

impediments14. How to empower agile teams to make decisions15. How to form and manage autonomous teams16. How to involve all team members in solution generation**

1. How to deal with increasing workload of key stakeholders 2. How to deal with internal silos3. How to align and communicate architectural decisions4. How to deal with geographical distance between agile teams5. How to establish automated testing6. How to create lightweight documentation7. How to apply agile practices for developing or maintaining

legacy systems8. How to ensure traceability of tests and requirements 9. How to create and estimate user stories10. How to explain requirements to stakeholders11. How to write understandable automated tests12. How to establish test verification13. How to coordinate test and deployment with external parties14. How to share domain knowledge across agile teams**15. How to involve remotely working/external colleagues**16. How to clarify details outside of meetings in cross-shore agile

teams**

* Process Consultant as part of Development Team **Counting from 78 identified by [4]

Responsibilities and Challenges Scrum Master, Development Team and Product Owner (2/2)

200302 Holz MT Final

Product Owner

Responsibilities • Part of agile team• Clarification of functional requirements• User Story description• Customer communication• Manage epic plan

Challenges 1. How to create precise requirement specifications for the development team2. How to elicit and refine requirements of end users3. How to split large and complex requirements into smaller requirements 4. How to deal with internal silos5. How to facilitate communication between agile teams and other teams

using traditional practices6. How to balance short-term and long-term goals7. How to communicate business requirements to development teams8. How to define clear and visible priorities9. How to deal with unplanned requirements and risks10. How to enforce customer involvement11. How to ensure traceability of tests and requirements12. How to make a cost and schedule estimation13. How to create and estimate user stories14. How to deal with fixed price contracts in agile software development 15. How to establish requirements verification16. How to define high-level requirements a.k.a. epics

17. How to measure the success of the large-scale agile development program

18. How to balance amount and quality of delivered requirements**19. How to manage overarching backlog item prioritization with multiple

product owners**20. How to understand all interfaces and dependencies of the system**21. How to support an onboarding approach for different stakeholder groups**22. How to manage requirement development for multiple teams**

**Counting from 78 identified by [4]

Occurences Concerns

200302 Holz MT Final

Occurences Concerns contd.

200302 Holz MT Final

Exemplary Pattern Candidate

200302 Holz MT Final

Process Consultant

ID M-07

Name Process Consultant

Alias Lead Developer

Summary

Process Consultants combine responsibilities of the roles of a developer, architect and consultant. The role findsapplication in LSAD with domain-specific knowledge. It can be compared to the concept of an Lead Developer.The Process Consultant aids in architecture decision making, knowledge sharing with less experienced developersand Product Owners and more. The Process Consultant is the most experienced developer of a domainresponsible for the support in development of requirements as well as development itself.

Exemplary Pattern Candidate contd.

200302 Holz MT Final

Process Consultant

ExampleTechnical LLC identified during the adoption process of a LSAD framework, the need for a Process Consultant. Process Consultants (Lead Developer) are domain experts supporting the development process of an system consisting of migrated legacy systems with inexperienced developers working on it.

Context Complex and historically grown system. Migrating several legacy systems and providing complex functions. Project growth leads to inexperienced developers working on the system.

Problem C-19 How to deal with internal silosC-85 How to share domain knowledge across teams

Forces High pace feature-driven development with newly created teams. Several bottlenecks in respect to knowledge as the teams work on legacy systems with complex functions and dependencies.

Solution

Process Consultant is like a technical lead for a specific domain and product. Creating an access point to knowledge by assigning a experienced developer, who can help less experienced developers and Product Owners understand system specifications and support creation process of new features and feature change. The Process Consultant is a 50/50 role, half of his time is allocated in supporting other developers and the PO work with the process.The other half is allocated for implementations and supporting architectural conception, alignment and implementation.

Exemplary Pattern Candidate contd.

200302 Holz MT Final

Process Consultant

Variants The Program uses experienced developers for multiple topics( architecture and team support, coaching for specificarchitecture topics, like UI/UX and Testing). Knowledge Transfer Workshops offer a variation.

Consequences

Advantages

• Offers solution for knowledge holes by forcefully creating internal silos

• Clear Contact Point for developers and PO alike through technical responsible person for domain

• Accountability for product and domain from technical aspect

• Efficiency Improvement with inexperienced developers

• No stoppage during development process• introduction reduces bottlenecks by freeing up

the day of these bottleneck to play a supportive knowledge transfer role

Disadvantages

• Limits the functional knowledge and process of gaining functional knowledge of developers

• Intentional Internal silos

See also M-08 Purpose Teams, C-09 Process Consultant Meeting

Overall Concern Categories

Communication & Coordination; 8

Culture & Mindset; 13

Geographical Distribution; 6

Knowledge Management; 8Methodology; 3

Project Management; 10

Quality Assurance; 5

Requirements Egineering; 9

Software Architecture; 1Tooling; 3

200302 Holz MT Final

Introduced Patterns

• Knowledge Management• Supervision• CoP for POs

• Culture & Mindset• Event-Storming Workshops (DDD)• Celebrate and Publish Success

• Quality Assurance• Quality Gates• Communicate Architecture

200302 Holz MT Final

Instantiation Process Proposed

200302 Holz MT Final