extracting examples for api usage patterns

Extracting Examples for API Usage Patterns

Hudson Silva Borges(Msc Student)

Marco Túlio Valente(Advisor)

March 20th, 2014

Agenda

(1) Introduction

(2) Proposed Solution

(3) Evaluation

(4) Conclusions

2March 20th, 2014 Extracting Examplesfor API Usage Patterns

Introduction


Introduction

● Systems often reuse functionality provided by APIs

● Reusing APIs generally requires a non-trivial effort

● Learning resources are a major obstacle● Documentation is the most severe obstacle



Motivation

● Rely on client usage data to improve APIs learning resources


Why Examples?

Robillard e DeLine, Empirical Software Engineering – 2011

“code examples are an essential element of API learning”


APIMiner 1.0

● Examples for single API methods


Problem

Robillard e DeLine, Empirical Software Engineering – 2011

“Small examples that nevertheless demonstrate API usage patterns involving

more than one method call will be more useful than single-call examples”

Proposed Solution


APIMiner 2.0 – Overview


Extends APIMiner 1.0 to consider usage patterns and readability improvements to provide

better source code examples

Usage Patterns

● Set of API methods frequently called together

● Example:

open() ⇒ close()


APIMiner 2.0 – Architecture


Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc

RankingEngine JavaDoc Weaver

Selected API Clients

Architecture: Source Code Analyzer


Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



Architecture: Source Code Analyzer


Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



Client System Method API Calls

A ma1

m1, m

10

A ma2

...

B mb1

m22

B mb2

m2, m

3, m

11

...

Architecture: Patterns Analyzer


Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



● API Usage Pattern

● Implications of the form X Y⇒

open() ⇒ close()

Patterns Analyzer – Association Rules


Client System Method API Calls

A ma1

m1, m

10

A ma2

...

B mb1

m22

B mb2

m2, m

3, m

11

...

● Usage Data



Transaction Items

t1

m1, m

10

t2

...

tn

m22

tn+1

m2, m

3, m

11

...

● Input to the algorithm



Transaction Items

t1

m1, m

3, m

10

t2

m1, m

10

t3

m2, m

3, m

10

t4

m1

t5

m1, m

5, m

8, m

10

t6

m10

● m1, m10

● m1 m⇒ 10

● Algorithm



Transaction Items

t1

m1, m

3, m

10

t2

m1, m

10

t3

m1, m

3, m

9

t4

m2

t5

m1, m

5, m

8, m

10

t6

m10

● m1, m10

● 3 transactions

● m1 m⇒ 10

● Support = 3● Confidence = ( 3 / 4 ) = 75%

● Algorithm

Architecture: Example Extractor


Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



● For each client's method:

1. Identifies calls to API methods

2. Extracts examples for single API calls

3. Extracts examples for usage patterns

4. Removes similar examples








Slicing algorithm

Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



Extracting Examples – Readability improvements


(1) Variables not identified




Original Extraction

Improved



(2) Empty blocks



(2) Empty blocks

?



(2) Empty blocks



JavaDoc






Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



Architecture: Ranking Engine


Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



● Ranking examples● Completeness● Lines of Code● Users Feedback

● Ranking usage patterns● Lift● Number of methods in the association rule

Architecture: JavaDoc Weaver


Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



Evaluation


Android APIMiner 2.0


www.apiminer.org

Android APIMiner 2.0

● Version 4.1.2-r1

● 26,540 methods● 2,968 classes

● 17,303 (65.2%) methods with JavaDoc annotation● 2,575 (86.76%) classes with JavaDoc annotation


Mining Dataset

● Script to automatically search and download in Github

● Criteria:● Java Programming language ● Minimum of 50 commits● Not a fork

● Some projects inserted manually


● Numbers:● 396 projects● 57,658 classes ● 450,762 methods● 362,900 calls to API methods.

● Highlight:● CyanogenMod● Astrid● Wordpress


Mining Dataset

● Rigid strategy

● Clients selected based on their popularity among the Android developers community

● Example Dataset Mining Dataset⊆

● Results:● 151 projects● 287,263 examples


Example Dataset

Transactions

● Number of method calls in the useful transactions


010

2030

API methods calls

% o

f use

ful t

rans

actio

ns

2 4 6 8 10 14 18 22 26 30 34 38

Median of 3

Mean of 4.6

Transactions

● Intra-class and Inter-class calls in useful transactions


Association Rules

● Problem: What values to use for support and confidence?

● Our strategy:● A not very large number of rules● Maximize the API methods coverage

● Tests varying the values of support and confidence


Association Rules

● Number of rules by varying the minimum support value


Association Rules

● API methods coverage by varying the minimum support value


Association Rules

● Used values:● Support = 21● Confidence = 70%

● Results:● 38,661 rules● 1,952 rules with antecedent size equal one● Coverage of 624 API methods; 191 API classes


Examples


● 151 Android client systems

● 287,263 examples● 102,442 examples for single API methods ● 184,821 examples for the usage patterns

● Coverage:● 3,915 Android API methods (14.75%) ● 549 Android API classes (18.5%)

Examples


● Size of the examples

● For single API methods● 50% have at most three lines of code

● For usage patterns● 50% have at most 14 lines of code

Ranking


● Activity.findViewById(int)

● API method with more examples

● Top 20 examples analyzed

● Examples size equal 1

● No compilation problems

Activity.findViewById(int)


● To return a value

● In an assignment

● As a target

● In a casting

Usage Patterns


● MediaRecorder.prepare()

● First usage patterns recommended:

MediaRecorder.prepare()

MediaRecorder.start()

MediaRecorder.setAudioSource(int)

MediaRecorder.setOutputFile(String)



● http://developer.android.com/reference/android/media/MediaRecorder.html

http://developer.android.com/reference/android/media/MediaRecorder.html

Field Study


Field Study


● Period of 13th May 2013 to 11th October 2013 (five months)

● No promotions

● Total of 32,335 visits

Field Study


● Top ten countries in visits

Field Study


● 5,545 requests for single API examples

● 2,601 (46.9%) clicks on “Example” button

● 176 requests for usage patterns

● 135 (out of 536) API methods with usage pattern were requested

User Study


User Study


● 29 subjects

● Two corrective maintenance tasks in a small Android app

Do the recommendations provided by APIMiner 2.0 help developers to implement

maintenance tasks with less effort?

Do the recommendations provided by APIMiner 2.0 help developers to implement

maintenance tasks with less effort?

More Aqui


Maintenance Tasks


● Designed to represent real maintenance activities

● Tasks with requirements well defined

● Tasks:● Obtaining location (Tutorial)● Inserting new properties● System messages

● Automated tests to check

● Crossover methodology


● Number of subjects who completed the tasks

User Study – Results


● Time spent in the tasks



● Examples Requests

● 686 examples requests for single API methods● 182 examples requests for usage patterns

● 23 examples for single API methods by subject● 6 examples for usage patterns examples by subject


Conclusions


Contributions


● Usage patterns examples

● Readability improvements

● Android APIMiner 2.0● 1,952 usage patterns for 624 API methods● 287,263 examples● www.apiminer.org

● Evaluation● Data produced when creating the instance● Usage data obtained from real Android developers● Controlled experiment with 29 subjects

http://www.apiminer.org/

Future Work


● Detect different usage types among the examples

● Ranking criteria based on readability

● Other static slicing implementations

Publications


● Montandon J. E. et al. Documenting APIs with Examples: Lessons Learned with the APIMiner Platform. In 20th Working Conference on Reverse Engineering (WCRE), 2013.

● Borges H. S. et al. APIMiner 2.0: Recomendaçao de exemplos de uso de APIs usando regras de associaçao. IV Congresso Brasileiro de Software: Teoria e Prática (Sessão de Ferramentas), 2013.

● Borges H. S.. APIMiner 2.0: Recomendação de Exemplos de Uso de APIs usando Regras de Associação. Poster, ELA-ES, 2013.


Thanks!

[email protected]

Architecture: API and Client systems


Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



● API Source Code

API’s source code

● API Clients to Mining and Selected API Clients

Repository with API client systems

Architecture: Mining Database


Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



● Database that stores the data generated by the Source Code Analyzer

Architecture: Examples Database


Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



● Store examples and usage patterns produced

Architecture: JavaDoc


Mining Database

Patterns Analyzer

APISourceCode

API Clients to

Mining

Souce Code Analyzer

Examples Extractor

Examples Database

JavaDoc



● Output of the proposed approach

● Ratio of methods with at least an API call among the systems

● Considering all client methods● In median, for each 3 methods are called 4 API methods● 75% of the systems had a ratio between 0.78 and 4.40 API

calls/method

● Considering only client methods with API method calls● In median, for each 3 methods are called 9 API methods● 50% of the projects have between 2.72 and 3.89 API

calls/method


Mining Dataset

Transactions

● Transaction = Client method

● Input to the association rules algorithm

● Not all clients' methods call an API method

● Transactions with a single call are not relevant


Transactions

● Characterization of the all 512,993 transactions


77.42%

9.19%

13.38%

Without calls

With a single calls

With more than one call

Transactions

● Percentage of useful transactions per project


Average of 30.49%

Median of 26.53%

Transactions

● Top five most common transaction items


Association Rules


● Top five rules with higher support

Examples


● Number of API method calls in the examples

Examples


● API methods with more examples

Examples


● Classes that most appeared in all examples

Usage Patterns


● Usage patterns without examples

● 280 association rules (14% of 1,952)

● 88 API methods out of the 624 distinct API methods

Field Study


● 31 evaluations of examples by end-users

● Examples of 26 API methods

● Methods with more evaluations:● Color.rgb(int, int, int)

● ViewPager.OnPageChangeListener.onPageSelected(int)

● Intent.getStringArrayExtra(String)

● NotificationCompat.Builder.setVibrate(long[])

NotificationCompat.Builder.setVibrate(long[])


User Study – Setup


● Subject characterization

● Crossover methodology● Group A and Group B

● Restrictions:● Subjects should have skills on the Java● Subjects should not be experts in the Android platform

User Study – Execution


● Brief introduction

● More Aqui App

● Task Obtaining location as tutorial

● Subjects:

1. Read the description

2. Register the starting time

3. Run the automated test

4. Perform the task

5. Register the ending time



● Knowledge of the subjects in the Java


● Knowledge of the subjects in the Android



● Professional experience of the subjects (in years)


Related Work


● IDE-based Recommendation Systems

● Vertical Code Completion (VCC)● Mining API Code Snippets (MACs)

● API Documentation

● APIMiner 1.0● ExoaDocs

● APIMiner 2.0

extracting examples for api usage patterns

Technology