stakeholders’ information needs for artifacts and their dependencies in a real world context
DESCRIPTION
n the evolution of software, stakeholders continuously seek and consult various information artifacts and their interdependencies to successfully complete their daily activities. While a lot of research has focused on supporting stakeholders in satisfying various information needs, there is little empirical evidence on how these information needs manifest themselves in the context of professional software development teams of real world companies. To investigate the information needs of the different stakeholder roles involved in software evolution activities, we conducted an empirical study with 23 participants from two professional development teams of one company. The analysis of the gathered data shows that information needs exhibit a crosscutting nature with respect to stakeholder role, activity, artifacts and even fragments of artifacts. We also found that the dependencies between information artifacts are important for the successful performance of software evolution activities, but often not captured explicitly. The lack of an explicit representation of these interdependencies often result in difficulties identifying dependent artifacts and additional communication effort. Based on our findings, we suggest ways to better support stakeholders with their information needs.TRANSCRIPT
Sebastian Müller and Thomas Fritz
Stakeholders’ Information Needs for Artifacts and their Dependencies in a Real World Context
ICSM 2013, Eindhoven
1
Stakeholder Activities
22
Stakeholder Activities
22
Stakeholder Activities
22
Stakeholder Activities
22
Stakeholder Activities
2(c) blog.cloudbooking.com and Sebastian Bergmann
2
Stakeholder Activities
2
Evolution of a project requires stakeholders to perform a lot of different activities.
(c) blog.cloudbooking.com and Sebastian Bergmann
2
Stakeholders Information Needs
3
Developer
Bug fixing
Bug
3
Stakeholders Information Needs
3
Developer
Bug fixing
Requirements
Bug
3
Stakeholders Information Needs
3
Developer
Bug fixing
Requirements
Code
Bug
3
Stakeholders Information Needs
3
Developer
Bug fixing
Requirements
Code
Bug
3
Stakeholders Information Needs
3
Developer
Bug fixing Software testing
Softwaretester
Requirements
Code
Bug
3
Stakeholders Information Needs
3
Developer
Requirementsengineer
Bug fixing Software testing
Softwaretester
Requirements
Code
Bug
3
Stakeholders Information Needs
3
Developer
Requirementsengineer
Bug fixing Software testing
Softwaretester
Requirements
Code Test
Bug
3
Stakeholders Information Needs
3
Developer
Requirementsengineer
Bug fixing Software testing
Softwaretester
Requirements
Code Test
Bug
3
Stakeholders Information Needs
3
Developer
Requirementsengineer
Bug fixing Software testing
Softwaretester
Requirements
Code Test
Bug
Stakeholders need to acquire and manage a lot of different information artifacts to perform activities.
3
Research Questions
4
RQ 1:
RQ 2:
What are the characteristics of information artifacts needed by different kinds of stakeholder roles in the daily software evolution activities?
How are these artifacts interdependent?
4
Related Work
Focused on• single stakeholder role
(e.g. Schröter et al CSWC’12, Seaman ICSM’02)
• single information artifact(e.g. Breu et al CSCW’10, Aranda et al ICSE’09)
• single activity(e.g. Sillito et al TSE’08, Roehm et al ICSE’12)
55
Study Methods
6
403 instancesof activity
description & interview
transcriptions
23 participants from 2 software
development teams
diary study
interviews
6
Study Participants
7
Role # SubjectsTeam 1Team 1
Software developer 5 (4 interviewed)Software tester 1Requirements engineer 1Project manager 1 (0 interviewed)Line manager 1
Team 2Team 2Software developer 6 (5 interviewed)Software tester 4Requirements engineer 2Project manager 1Line manager 1All stakeholders 23 (20 interviewed)
7
Study Methods
8
403 instancesof activity
description & interview
transcriptions
23 participants from 2 software
development teams
diary study
interviews
8
Data Analysis
99
Data Analysis
9
Collected Data
Interview transcripts from 20 follow-up
interviews
Activity descriptions from a six day diary study
9
Data Analysis
9
Data Coding
Open coding todevelop concepts
and categories
Axial coding torelated these to
each other
Selective codingto related data tothese categories
Collected Data
Interview transcripts from 20 follow-up
interviews
Activity descriptions from a six day diary study
9
Data Analysis
9
Data Coding
Open coding todevelop concepts
and categories
Axial coding torelated these to
each other
Selective codingto related data tothese categories
Collected Data
Interview transcripts from 20 follow-up
interviews
Activity descriptions from a six day diary study
Findings
Information needs exhibit a
crosscutting nature
Information needs are fragmented
Dependencies between information
artifacts are very important
9
Stakeholders’ Activities
10
Bug Fixing
Software developer
Software tester
Requirements engineer
Project manager
Line manager
10
Software developer
Software tester
Requirements engineer
Project manager
Line manager
Bug Analysis
10
Software developer
Software tester
Requirements engineer
Project manager
Line manager
Software Testing
10
Software developer
Software tester
Requirements engineer
Project manager
Line manager
Requirements Creation
10
Research Question 1
What are the characteristics of information artifacts needed by different kinds of stakeholder roles in the daily software evolution activities?
1111
! Code!
Requirements!
Release!notes!
Change!set!
Planning!docum
ents!
Change!request!
Code!docum
entation!
Test!report!
Personal!rotes!
Logs!
HR9related!
documents!
Test!case!
Code!review!
Code!model!
Configu
ration!file!
Website!
#!of!!artifacts!
Bug!fixing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Implementing!features! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Attending!meetings! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 6!
Bug!analysis! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Project!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Software!testing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Communication!/!Interaction!
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Requirements!creation! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 7!
Refactoring! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 4!
Department!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 2!
!
Crosscutting Information Needs
12
Activities
Informationartifact
12
! Code!
Requirements!
Release!notes!
Change!set!
Planning!docum
ents!
Change!request!
Code!docum
entation!
Test!report!
Personal!rotes!
Logs!
HR9related!
documents!
Test!case!
Code!review!
Code!model!
Configu
ration!file!
Website!
#!of!!artifacts!
Bug!fixing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Implementing!features! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Attending!meetings! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 6!
Bug!analysis! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Project!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Software!testing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Communication!/!Interaction!
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Requirements!creation! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 7!
Refactoring! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 4!
Department!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 2!
!
Crosscutting Information Needs
12
Activities
Informationartifact
12
! Code!
Requirements!
Release!notes!
Change!set!
Planning!docum
ents!
Change!request!
Code!docum
entation!
Test!report!
Personal!rotes!
Logs!
HR9related!
documents!
Test!case!
Code!review!
Code!model!
Configu
ration!file!
Website!
#!of!!artifacts!
Bug!fixing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Implementing!features! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Attending!meetings! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 6!
Bug!analysis! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Project!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Software!testing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Communication!/!Interaction!
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Requirements!creation! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 7!
Refactoring! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 4!
Department!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 2!
!
Crosscutting Information Needs
12
Activities
Informationartifact
12
! Code!
Requirements!
Release!notes!
Change!set!
Planning!docum
ents!
Change!request!
Code!docum
entation!
Test!report!
Personal!rotes!
Logs!
HR9related!
documents!
Test!case!
Code!review!
Code!model!
Configu
ration!file!
Website!
#!of!!artifacts!
Bug!fixing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Implementing!features! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Attending!meetings! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 6!
Bug!analysis! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Project!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Software!testing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Communication!/!Interaction!
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Requirements!creation! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 7!
Refactoring! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 4!
Department!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 2!
!
Crosscutting Information Needs
12
Activities
Informationartifact
12
! Code!
Requirements!
Release!notes!
Change!set!
Planning!docum
ents!
Change!request!
Code!docum
entation!
Test!report!
Personal!rotes!
Logs!
HR9related!
documents!
Test!case!
Code!review!
Code!model!
Configu
ration!file!
Website!
#!of!!artifacts!
Bug!fixing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Implementing!features! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Attending!meetings! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 6!
Bug!analysis! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Project!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Software!testing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Communication!/!Interaction!
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Requirements!creation! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 7!
Refactoring! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 4!
Department!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 2!
!
Crosscutting Information Needs
12
Information needs exhibit a crosscuttingnature over information artifacts.
Activities
Informationartifact
12
Crosscutting Information Needs for Bug Analysis
Information Artifacts needed for Bug Analysis:
13
13
Crosscutting Information Needs for Bug Analysis
Information Artifacts needed for Bug Analysis:
13
Software developers:6 artifacts: change request, code, requirements, test cases, logs, and code models
13
Crosscutting Information Needs for Bug Analysis
Information Artifacts needed for Bug Analysis:
13
Software developers:6 artifacts: change request, code, requirements, test cases, logs, and code models
Line managers:2 artifacts: change request, requirements
13
Crosscutting Information Needs
Information Artifacts needed for Bug Analysis:
14
Software developers:6 artifacts: change request, code, requirements, test cases, logs, and code models
Line managers:2 artifacts: change request, requirements
14
Crosscutting Information Needs
Information Artifacts needed for Bug Analysis:
14
Information needs vary by stakeholderrole for each activity.
Software developers:6 artifacts: change request, code, requirements, test cases, logs, and code models
Line managers:2 artifacts: change request, requirements
14
Information Needs are Fragmented
Information needed in a change request while bug triaging:
15
15
Information Needs are Fragmented
Information needed in a change request while bug triaging:
15
“Especially important is the person who has
reported the change request.”
Line manager15
Information Needs are Fragmented
Information needed in a change request while bug triaging:
15
“Especially important is the person who has
reported the change request.”
Line manager
“The type of the change request is important. If it is a
defect, the severity is important.”
Project manager15
Information Needs are Fragmented
Information needed in a change request while bug triaging:
15
“Especially important is the person who has
reported the change request.”
Line manager
“The type of the change request is important. If it is a
defect, the severity is important.”
Project manager
Information needs per artifact are also fragmented.
15
Crosscutting Information Needs
16
16
Crosscutting Information Needs
16
For different activities, different stakeholder roles need different information artifacts.
16
Crosscutting Information Needs
16
For different activities, different stakeholder roles need different information artifacts.
For the same activity, different stakeholder roles might used different information artifacts.
16
Crosscutting Information Needs
16
For different activities, different stakeholder roles need different information artifacts.
For the same activity, different stakeholder roles might used different information artifacts.
Even if the same information artifact is used, different fragments in this information artifact are important for different stakeholder roles.
16
Research Question 2
How are these artifacts interdependent?
1717
Levels of Artifact Dependencies
18
18
Levels of Artifact Dependencies
18
Test case
Requirements
18
Levels of Artifact Dependencies
18
Test case
Requirements
Severity of achange request
Test case
18
Levels of Artifact Dependencies
18
Location of incident in a bug reported
from field
Specific part of arequirements document
Test case
Requirements
Severity of achange request
Test case
18
Levels of Artifact Dependencies
18
Location of incident in a bug reported
from field
Specific part of arequirements document
Test case
Requirements
Severity of achange request
Test case
Multiple artifact dependencies are needed for an activity
18
Levels of Artifact Dependencies
19
Multiple artifact dependencies are needed for an activity
Links between multiple information artifacts have to be established and maintained.
19
Artifact Dependencies in Reality
Links between information artifacts are often missing for many reasonsThese missing links lead to additional and repeated communication effort by all stakeholder roles
20
20
Artifact Dependencies in Reality
21
21
Artifact Dependencies in Reality
21
“I guess that in only 6% of all cases there is really a link between the test case and the requirements.” (R2)
21
Artifact Dependencies in Reality
21
“I guess that in only 6% of all cases there is really a link between the test case and the requirements.” (R2)
“There is a lot of documentation available, but it is not widely known where to find it.” (D9)
21
Artifact Dependencies in Reality
21
“I guess that in only 6% of all cases there is really a link between the test case and the requirements.” (R2)
“There is a lot of documentation available, but it is not widely known where to find it.” (D9)
“4-5 iterations [with a requirements engineer] are necessary until every issue is clarified and until I can start to implement anything.” (D2)
21
Current Practices to Link Artifacts
22
22
Current Practices to Link Artifacts
Stakeholders already used a lot of different tools to recover from this problem, e.g.:
22
“In the wiki links to other important internal documents, e.g. requirements specifications, are stored.” (D3)
22
Current Practices to Link Artifacts
Stakeholders already used a lot of different tools to recover from this problem, e.g.:
22
“In the wiki links to other important internal documents, e.g. requirements specifications, are stored.” (D3)
“We in the requirements engineering team do know that developers use a wiki, but we do not think that this is useful.” (R2)
No success so far:
22
Better Tool Support for Linking Information Artifacts
Make links first class entities
Focus on many-to-many links instead of 1-to-1
Aggregate and tailor information based on activity and role context
23
23
Conclusion
24
24
Conclusion
24
24
Conclusion
24
24
Conclusion
24
! Code!
Requirements!
Release!notes!
Change!set!
Planning!docum
ents!
Change!request!
Code!docum
entation!
Test!report!
Personal!rotes!
Logs!
HR9related!
documents!
Test!case!
Code!review!
Code!model!
Configu
ration!file!
Website!
#!of!!artifacts!
Bug!fixing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Implementing!features! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Attending!meetings! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 6!
Bug!analysis! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Project!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Software!testing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Communication!/!Interaction!
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Requirements!creation! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 7!
Refactoring! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 4!
Department!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 2!
!
24
Conclusion
24
! Code!
Requirements!
Release!notes!
Change!set!
Planning!docum
ents!
Change!request!
Code!docum
entation!
Test!report!
Personal!rotes!
Logs!
HR9related!
documents!
Test!case!
Code!review!
Code!model!
Configu
ration!file!
Website!
#!of!!artifacts!
Bug!fixing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Implementing!features! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Attending!meetings! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 6!
Bug!analysis! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10!
Project!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Software!testing! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 8!
Communication!/!Interaction!
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 11!
Requirements!creation! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 7!
Refactoring! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 4!
Department!management! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 2!
!
24
Survey / Interview Guide
Diary study survey and interview guide are available online:http://www.ifi.uzh.ch/seal/people/mueller/info-needs
25
25