analyzing the eclipse api usage: putting the developer in the loop
DESCRIPTION
John's talk at CSMR 2013TRANSCRIPT
Analyzing the Eclipse API Usage: Putting the Developer
in the Loop
John Businge, Alexander Serebrenik, Mark van den Brand
Software Engineering and Technology (SET) 12-04-2023
Introduction
PAGE 2
Software engineering researchers obtaining convincing evidence
Software Engineering and Technology (SET) 12-04-2023
Introduction
PAGE 3
Software engineering researchers obtaining convincing evidence
Few examples of convincing evidence
Software Engineering and Technology (SET) 12-04-2023
Introduction
PAGE 4
Software engineering researchers obtaining convincing evidence
Few examples of convincing evidence
Researchers generate evidence about A, C, D and E yet developers
care about B
Software Engineering and Technology (SET) 12-04-2023
Introduction
PAGE 5
Software engineering researchers obtaining convincing evidence
Few examples of convincing evidence
Researchers generate evidence about A, C, D and E yet developers
care about B
Include developer in the loop
Software Engineering and Technology (SET) 12-04-2023
Terms used
PAGE 6
P1 P4P3P2
Eclipse Framework
Eclipse Third-party Plug-ins (ETPs)
Software Engineering and Technology (SET) 12-04-2023
Terms used
PAGE 7
P1 P4P3P2
Eclipse Framework
Eclipse Third-party Plug-ins (ETPs)
Eclipse APIs(“good”)• no “internal”• stable, • supported
Eclipse non-APIs (“bad”)• “internal”• unstable,• discouraged,• unsupported
P3 – good ETPs P1, P2 and P4 – bad ETPs
Software Engineering and Technology (SET) 12-04-2023
Terms used
• Eclipse provisional API guidelines – document provided by Eclipse describing the different interfaces
PAGE 8
Embryonic interfaces
Software Engineering and Technology (SET) 12-04-2023
Terms used
• Eclipse provisional API guidelines – document provided by Eclipse describing the different interfaces
PAGE 9
Embryonic interfaces Battle-headed interfaces
Software Engineering and Technology (SET) 12-04-2023
Motivation of our study
• Our previous studies about Eclipse API Usage were based on source code analysis.
• Our observations:1. 44% of ETPs on SourceForge use bad interfaces.
2. bad ETPs are larger than good ETPs.
3. Good interfaces are indeed very stable in new framework releases.
4. Bad interfaces are indeed unstable in new framework releases.
PAGE 10
Software Engineering and Technology (SET) 12-04-2023
Motivation of our study
• Omitting developer in the loop in previous studies
• Why bad ETPs are larger than good ETPs????
• Why developers use bad interfaces????
PAGE 11
Include developer in the loop
Software Engineering and Technology (SET) 12-04-2023
Survey to include developer in loop
• Goal – obtain the state-of-practice Eclipse interface usage
1. Reasons for use of bad interfaces
2. Difference in characteristics of bad ETPs and good ETPs
• Research questionsRQ1: Can we observe some relationships in factors related to Eclipse interface usage in the survey?
RQ2: What are the differences in characteristics between bad and good ETPs?
RQ3: What are the differences in characteristics between proprietary and open-source ETPs?
PAGE 12
Software Engineering and Technology (SET) 12-04-2023
Survey to include developer in loop
• Goal – obtain the state-of-practice Eclipse interface usage
1. Reasons for use of bad interfaces
2. Difference in characteristics of bad ETPs and good ETPs
• Research questionsRQ1: Can we observe some relationships in factors related to Eclipse interface usage in the survey?
RQ2: What are the differences in characteristics between bad and good ETPs?
RQ3: What are the differences in characteristics between proprietary and open-source ETPs?
PAGE 13
Software Engineering and Technology (SET) 12-04-2023
Survey factors of investigation
PAGE 14
No Factors1 Years of Education
2 Experience as a software engineer
3 Experience as a Java developer
4 Experience as an ETP developer
5 ETP development team size
6 Importance of updating ETP with new Eclipse releases
7 Number of versions of the ETP
8 Number of files of the ETP
9 Awareness of Eclipse guidelines
10 Following Eclipse guidelines
Software Engineering and Technology (SET) 12-04-2023
Sample Questions in the survey
i. How many years of experience as an Eclipse product/solution developer do you have?
ii. Are you aware of the “Eclipse Provisional API Guidelines”?
iii. Do you use non-APIs?
iv. If answer was “NO” in (iii). Do you deliberately avoid using Eclipse non-APIs?
• Survey Response− 30 ETP developers fully answered the questionnaire.
PAGE 15
Software Engineering and Technology (SET) 12-04-2023
RQ2 and RQ3 - groups
• RQ2 – Difference between good and bad ETPs
PAGE 16
Use of bad interfacesGroup Respondents
Developers who use bad interfaces 21
Developers who do not use bad interfaces 9
Open-source vs proprietary ETPsGroup Respondents
Developers who develop open-source ETPs 16
Developers who develop proprietary ETPs 14
• RQ3 – Difference between proprietary and open-source ETPs
Software Engineering and Technology (SET) 12-04-2023
RQ2 and RQ3 - groups
• The two groups of RQ2 and RQ3 can be compared using the 10 identified factors.
PAGE 17
Software Engineering and Technology (SET) 12-04-2023
Differences between bad and good ETPs – RQ2
• Hypotheses:• H0: Developers who use bad interfaces and
those who do not use bad interfaces have the same average values on each of the identified factors.
• Ha: There is a difference on the average values of the identified factors between the developers that use and do not use bad interfaces.
PAGE 18
Software Engineering and Technology (SET) 12-04-2023
Differences between developers that use and those that do not use bad interfaces
Factors DifferenceYears of Education NO
Experience as a software engineer YES
Experience as a Java developer YES
Experience as an Eclipse plug-in developer YES
ETP development team size YES
Importance of updating the plug-in with Eclipse NO
Number of versions NO
Number of files YES
Awareness of Eclipse guidelines NO
Following Eclipse guidelines NO
PAGE 19
Software Engineering and Technology (SET) 12-04-2023
Differences between developers that use and those that do not use bad interfaces
Factors DifferenceYears of Education NO
Experience as a software engineer YES
Experience as a Java developer YES
Experience as an Eclipse plug-in developer YES
ETP development team size YES
Importance of updating the plug-in with Eclipse NO
Number of versions NO
Number of files YES
Awareness of Eclipse guidelines NO
Following Eclipse guidelines NO
PAGE 20
We reject H0 that Developers who use bad interfaces and those who do not use bad interfaces have the same average values on each of the identified factors
Software Engineering and Technology (SET) 12-04-2023
How different are the two groups of developers?
• We discovered that developers that use bad interfaces have higher values on these factors compared to those that do not use bad interfaces
PAGE 21
FactorsExperience as a software engineer
Experience as a Java developer
Experience as an Eclipse plug-in developer
ETP development team size
Number of files
Software Engineering and Technology (SET) 12-04-2023
How different are the two groups of developers?
• We discovered that developers that use bad interfaces have higher values on these factors compared to those that do not use bad interfaces
PAGE 22
FactorsExperience as a software engineer
Experience as a Java developer
Experience as an Eclipse plug-in developer
ETP development team size
Number of files
1. More experiences developers
2. Larger ETP development teams
Previously unanswered question “Why bad ETPs are larger than good ETPs?”
Human factors
Developers of bad ETPs
Software Engineering and Technology (SET) 12-04-2023
Differences between open-source and proprietary ETPs• Motivation:• Previous studies we only considered open-source
software potentially putting generalizability to proprietary software in jeopardy.
• Hypotheses:• H0: Developers of open-source and proprietary ETPs
have the same average values on each of the identified factors.
• Ha: There is a difference on the average values of the identified factors between the developers of open-source and proprietary ETPs.
PAGE 23
Software Engineering and Technology (SET) 12-04-2023
How different are the two groups of open-source and proprietary developers?
PAGE 24
Factors DifferenceYears of Education NO
Experience as a software engineer NO
Experience as a Java developer NO
Experience as an Eclipse plug-in developer NO
ETP development team size NO
Importance of updating the plug-in with Eclipse NO
Number of versions NO
Number of files NO
Awareness of Eclipse guidelines NO
Following Eclipse guidelines NO
Software Engineering and Technology (SET) 12-04-2023
How different are the two groups of open-source and proprietary developers?
• We accept H0 that there is no difference between open-
source and proprietary ETPs in terms of the factors investigated
PAGE 25
Factors DifferenceYears of Education NO
Experience as a software engineer NO
Experience as a Java developer NO
Experience as an Eclipse plug-in developer NO
ETP development team size NO
Importance of updating the plug-in with Eclipse NO
Number of versions NO
Number of files NO
Awareness of Eclipse guidelines NO
Following Eclipse guidelines NO
Software Engineering and Technology (SET) 12-04-2023
How different are the two groups of open-source and proprietary developers?
• Current findings clear our doubts on generalizability of our previous studies.
PAGE 26
Software Engineering and Technology (SET) 12-04-2023
Use of bad interfaces
• Asked why developers deliberately avoid using bad interfaces.
− Answerers were related to, knowing/assuming that the bad interfaces are unstable.
• Asked why developers deliberately use bad interfaces − No good interface with necessary functionality.− Avoid reinventing the wheel.− To expose these bad interfaces.− Old bad interfaces are unlikely to disappear
PAGE 27
Software Engineering and Technology (SET) 12-04-2023
Use of bad interfaces
• Asked why developers deliberately avoid using bad interfaces.
− Answerers were related to, knowing/assuming that the bad interfaces are unstable.
Less experienced developers
• Asked why developers deliberately use bad interfaces − No good interface with necessary functionality.− Avoid reinventing the wheel.− To expose these bad interfaces.− Old bad interfaces are unlikely to disappear
More experienced developers
PAGE 28
Software Engineering and Technology (SET) 12-04-2023
Conclusion
PAGE 29
Survey was conducted to understand why developers use bad interfaces.
Software Engineering and Technology (SET) 12-04-2023
Conclusion
PAGE 30
Less experienced developers
Survey was conducted to understand why developers use bad interfaces.
Instability overshadows benefits
Software Engineering and Technology (SET) 12-04-2023
Conclusion
PAGE 31
Less experienced developersMore experienced developers
Survey was conducted to understand why developers use bad interfaces.
Instability overshadows benefits
Enjoy benefits despite the instability
Software Engineering and Technology (SET) 12-04-2023
Conclusion
PAGE 32
Less experienced developersMore experienced developers
Survey was conducted to understand why developers use bad interfaces.
Instability overshadows benefits
Enjoy benefits despite the instability
Discovered that there is no difference between proprietary and open-source ETPs based on the factors we studied.
Software Engineering and Technology (SET) 12-04-2023
Thank you for listening
PAGE 33
• http://www.win.tue.nl/~jbusinge/CSMR13/
Researchers should generate evidence about topics developers
care about
Include developer in the loop
by