software architects’ experiences of quality requirements: what we know and what we do not know?
DESCRIPTION
Abstract. [Context/motivation] Quality requirements (QRs) are a concern of both requirement engineering (RE) specialists and software architects (SAs). However, the majority of empirical studies on QRs take the RE analysts’/clients’ perspectives, and only recently very few included the SAs’ perspective. As a result, (i) relatively little is known about SAs’ involvement in QRs engineering and their coping strategies, and (ii) whatever is known mostly comes from small and midsized projects. [Question/problem] The question in this exploratory study is how SAs cope with QRs in the context of large and contract-based software system delivery projects. [Principal ideas/results] We executed an exploratory case study with 20 SAs in the context of interest. The key results indicate the role SAs play in QRs engineering, the type of requirements communication processes SAs are involved in, the ways QRs are discovered, documented, quantified, validated and negotiated. Our most important findings are that in contract-based contexts: (1) the QRs are approached with the same due diligence as the functional requirements and the architecture design demand, (2) the SAs act proactively and embrace responsibilities over the QRs, (3) willingness to pay and affordability seem as important QRs prioritization criteria as cost and benefits do, and (4) QRs engineering is perceived as a social activity and not as much as a tool and method centric activity.[Contribution] The main contributions of the paper are (i) the explication of the QRs process from SAs’ perspective, and (ii) the comparison of our findings with previously published resultsTRANSCRIPT
1
Software Architects’ Experiences of Quality Requirements: What we Know and What we do not Know?
Maya Daneva, Luigi Buglione, Andrea Herrmann
2
Table of Contents
1. Introduction2. Background and Motivation 3. The Research Design: Qualitative Case
Study4. The Application of the Method
Results Limitations
1. Comparison to Literature 2. Implications 3. Wrap-Up
Maya Daneva, Wed, April 16, 2009
3
Background and Motivation
Most empirical research on QRs takes the perspective of clients, RE researchers, practitioners.
Quality requirements (QRs) are a concern of multiple stakeholders; in particular: software architects (SAs)
Relatively little is know about SA’s involvement
Evidence comes from small and mid-sized projects; very few studies in large projects
4
The Research Questions
Research Goal: to understand how SAs cope with QRs in large and contract-based software system development projects.
1. How do the SAs understand their role? 2. Do SAs and RE staff use different QRs terminology? 3. How do QRs get elicited? 4. How do QRs get documented? 5. How do QRs get prioritized? 6. How do QRs get quantified, if at all? 7. How do QRs get validated? 8. How do QRs get negotiated? 9. What role does the contract play in the way SAs cope
with QRs?
5
The Case Study Research Design
Maya Daneva, Wed, April 22, 2009
Key Steps (R. Yin, 2008):
1. Define interview guide
2. Pilot the interview
3. Collect data by interviewing participants
4. Analyze the data
5. Report on the results
6
Who Did We Interview?
20 Architects from 14 companies in North Europe
All have 10+ years of experience in large systems
All work on large contract-based projects (3 dev locations and 2 client locations)
Various pricing agreements
Sectors: large IT vendors, Oil &Gas, Insurance, Real estate, Video streaming, online systems (travel, book store, games)
7
Who Did We Analyze the Data?
Coding practices based on the grounded theory book of K. Charmaz (2006)
Iterative procedure
8
Results (1): How do the SAs understand their role?
Formal job descriptions and competence models
Self-described roles as:
‘a bridge’ b/n QRs and underlying technology
‘a translator’ from the user language into the feature specification language
‘review gate keepers’ regarding e.g. contract compliance
9
Results (2): Do SAs and RE staff use different terminology for QRs?
Gaining communication clarity was a non-issue
What helped? Domain knowledge Experience If ISO-certified, then training, quality
manuals, product quality handbooks made the difference
Issue: interchangeable use of terms from two streams of standards (management and technical how-to)
10
Results (3): How do QRs get elicited?
14 SAs use checklists based on: ISO standards Architecture frameworks Internal standards Stakeholder engagement
standards, AA1000SES
4 SAs uses game-based processes
2 SAs used story-telling techniques
11
Results (4): How do QRs get documented?
15 SAs: By using predefined templates based on : ISO standards the Quality Function Deployment
framework Planguage the INVEST grid approach Vendor-specific notations (e.g. SAP)
5 SAs: By using natural language
12
Results (5): How do QRs get prioritized?
The business case is the driver behind trade-offs, e.g. KPIs
No particular prioritization method
Prioritization criteria: cost, benefits, client’s willingness to pay affordability
Who decides? Steering committees SAs
2 SAs used story-telling techniques
13
Results (6): How do QRs get quantified?
Quantification is useful, but should not happen too early
How you get them? pre-specified in the contract Engage a specialist expert (e.g. in
performance) Decompose, operationalize and use
estimation technique (IFPUG NFR)
Issue: product and project measures are used
incorrectly QRs are confused with design-level req’ts
14
Results (7): How do QRs get validated?
By using requirements walkthroughs
The QFD framework
Internal architecture standards
15
Results (8): How do QRs get negotiated?
The business case is the commonly used vehicle
The QFD framework
EasyWinWin
The six-thinking-hat method
16
Results (9): What role does the contract play in the way
SAs cope with QRs?
3 ways for influencing QRs processes: Cost-conciousness Service level agreements Pre-defined priorities for QRs
Contracts are conductive as ways to maintain control
3SAs: “contract is was not that important”
EasyWinWin
The Six-Thinking-Hats method
17
Limitations of the Study
The possible validity concerns:
1. External validity Similarity with contract-based system delivery
settings Application domain, organizational maturity
1. Data collection threats: did SAs answer the questions truthfully?
2. Data analysis threats: researcher’s bias
18
Comparison to LiteratureInput: 5 empirical studiesvan Heesch et al, Ameller et al (2) Poort et al (2) 1.QRs are:
elicited by using checklists, documented by means of templates prioritized based on willingness to pay and affordability quantified by using size estimation standards (IFPUG) negotiated by using the business case
1.SAs: Serve as ‘a bridge’ and have formal job descriptions, Have terminology (established by standards)
1.Contract plays an important role for SAs to act the ways we observed.
19
Implications
1. To SAs: this study suggests QRs conversations start with contracts, SLA, and business cases
2. To RE practitioners: your SA could be quite a valuable resource! She/he can save you time.
3. To RE tool vendors: it seems more important to figure out how to embed tools into social processes and broader social interaction
4. To RE researchers: extend the focus on methods, models and automation by including analysis of QRs processes as socially constructed ones.
20
Wrap-Up
1. We carried out an interview-based study
2. It revealed what SAs were thinking on QR
3. We compared and contrasted the findings with published literature
4. Implications fro practice and research are made.
21