predicting build failures using social network analysis developer ... · social network analysis...

Predicting Build Failuresusing Social Network Analysis

on Developer Communication

Timo Wolf Adrian Schröter Daniela Damian Thanh NguyenSIEMENS AGCorporate TechnologySoftware & Engineering

Software Engineering Global interAction Lab – SEGALComputer Science DepartmentUniversity of Victoria, BC Canada

SIEMENS AG timowolf@siemens.com

Introduction

Build Failure- Compile Error- Test Failure

IntroductionThe damn build broke again! WHY ?

DB changed!But that’s transparent.Analyses takes time!

We should have talked earlier!

Introduction

• Collaboration essential in software development

• Communication problems lead to coordination and integration failures

The damn build broke again! WHY ?

Introduction

• Collaboration essential in software development

• Communication problems lead to coordination and integration failures

Do specific communication behaviors enable effective and successful coordination?

The damn build broke again! WHY ?

Communication for collaboration

• Communication:

• Task related content

Task relatedContent &

Information

• Communication:

Task relatedContent &

Information

• Structure

• Communication:

Our focus:Communication Structure and relationship to coordination outcome

• Structure

SIEMENS AG timowolf@siemens.com 4

Study Settings: IBM Jazz

IBM Research&

151 developers

47 functional teams

7 development sites in USA, Canada, Europe

Coordination & Integration in Jazz

Team A

Stream

Build 1

Team A Contributor 1 Contributor 2 Contributor 3

Team A

Stream

Build 1

!! !! !!

Team A

Stream

Build 1

Team A

Stream

Build 1

Team B

Stream

Build 1

commits changes

Team B Contributor 4 Contributor 5 Contributor 6

ChangeSets

Team A

Stream

Build 1

Team B

Stream

Build 1

commits changes

ChangeSets

ProjectBuild 1Project

IntegrationStream

publish changesget changes

Team A

Stream

Build 1

Team B

Stream

Build 1

commits changes

ChangeSets

ProjectBuild 1Project

IntegrationStream

publish changesget changes

Coordination Outcome: Build Result8

Research Questions

! OK ERROR

Research Questions

! OK ERROR

Research Questions

SNA Measurese.g. Density = 0.8

SNA Measurese.g. Density = 0.4predictmeasure predictmeasure

! OK ERROR

Research Questions

• RQ1: Can individual measures of communication structure predict integration failure?

! OK ERROR

Research Questions

• RQ1: Can individual measures of communication structure predict integration failure?

• RQ2: Can the combination of communication structure measures predict integration failure?

Constructing Social Networks in Jazz

Stream

Build N

Failed

Stream

Build N

Failed

Stream

Build N

Failed

Build N Network

Stream

Build N

Failed

Build N Network

Stream

Build N

Failed

Build N Network

quantitativemeasures

of communication structure

Stream

Build N

Failed

c1c2 c3 c4

SNA Measures

Data Collection & Selection

• Time Range: Nov. 5, 2007 - Feb. 26, 2008

• 1288 Build Results

• 25713 Work Items

• 71019 Comments

• Selected Build Results of 5 Teams

• Selected 3 Integration Builds (nightly, weekly, beta)

RQ1: individual measure prediction?

SN Build 1

SN Build 3

SN Build 4

SN Build 2

SN Build 5

SN Build 6

SN Build 1

SN Build 3

SN Build 4

SN Build 2

SN Build 5

SN Build 6

Social Network Measures

• Densityfor Completeness

• Degree Centralityfor Activity

• Betweenness Centralityfor Control

• Structural Holesfor Redundancy

SN Build 1

SN Build 3

SN Build 4

SN Build 2

SN Build 5

SN Build 6

Social Network Measures

• Densityfor Completeness

• Degree Centralityfor Activity

• Betweenness Centralityfor Control

• Structural Holesfor Redundancy

No statistically significant difference!

RQ2: multi measures prediction?

Bayesian Classifier

1. Train withall measures

2. Input

Bayesian Classifier

2. Input3. Predict

Bayesian Classifier

2. Input3. Predict

Bayesian Classifier

Prediction Results

Prediction ResultsERROR Team Builds Project Builds

B C F P W nightly weekly beta

Recall 55% 75% 62% 66% 74% 89% 100% 92%

Precision 52% 50% 75% 76% 66% 73% 92% 92%

Prediction ResultsERROR Team Builds Project Builds

Recall 55% 75% 62% 66% 74% 89% 100% 92%

Precision 52% 50% 75% 76% 66% 73% 92% 92%

Prediction Results

• Recall:

• 62 of 100 failed builds are classified correctly

ERROR Team Builds Project Builds

Recall 55% 75% 62% 66% 74% 89% 100% 92%

Precision 52% 50% 75% 76% 66% 73% 92% 92%

Prediction Results

• Recall:

• 62 of 100 failed builds are classified correctly

• Precision:

• 75 of 100 failure predictions are actual failures

• 75% confidence that a prediction is correct

ERROR Team Builds Project Builds

Recall 55% 75% 62% 66% 74% 89% 100% 92%

Precision 52% 50% 75% 76% 66% 73% 92% 92%

Summary

• Quantitative empirical evidence:Communication structures make a difference!

Summary

• Communication structures:

• impact integration build results

• are key for successful collaboration

Summary

• Single measures do not predict build results

Summary

• Single measures do not predict build results

• Combination measures do predict build results

Practical Implications

• Communication assessment for future builds

• Notification system for build failures

• Communication structure improvement

Future Work

• Factor analysis of combined measures

• Include technical information

• Prediction effectiveness case study

Questions?

Timo Wolftimowolf@siemens.com

Adrian Schröterschadr@uvic.ca

Daniela Damiandanielad@cs.uvic.ca

Thanh Nguyenduythanh@uvic.ca

http://segal.uvic.ca

SIEMENS AGCorporate TechnologySoftware & Engineering

Backup Slides

Descriptive Statistics I

Descriptive Statistics II

predicting build failures using social network analysis developer ... · social network analysis...

Documents

social network analysis (sna) made easy

trilcke - social network analysis (sna) als methode einer...

siemens profinet co-operative network training

systems network architecture (sna) onno w. purbo...

dynamic structure mining. limitations to traditional sna...

nokia siemens network sra 4

sna 8: network resilience -...

network planning siemens

social network analysis “sna” for complete...

network graph databases, rdf, sparql, and sna

social network analysis with sna package

sna-systems network architecture

social network analysis (sna) · social network analysis...

siemens profibus network manual

sna wired & wireless network solutions doc

alessandro sciullo, dip. cps, università di torino...

sna기반의 social networks - duredure.net/ebiz0910.pdf ·...

package ‘network’ · pdf filepackage ‘network ’...

siemens icn r & d network pmo

networking on z/os - ibmand functions of network protocols,...