Towards Supporting On-Demand Virtual

Remodularization Using Program Graphs

David Shepherd, Lori Pollock, and K. Vijay-Shanker

University of Delaware

Overview and Rationale

on-demandremodularization

//Activates the tool for the given view. //Turn on dragtool by adding....

activate tool

save drawing

update drawing

undo action

object oriented systemaction

oriented system

public void Tool.activate()

Overview and Rationale

public interface Storable{...

activate tool

save drawing

update drawing

undo action

public void Circle.save()

//Store the fields in a file....

action oriented system

object oriented system

Key Insight: Programmers leave natural language clues that

reference concerns

State of the Art in Feature Location/Aspect Mining

• Dynamic Information – [Wilde ICSM 00], [Michail ICSE 02], [Zhao ICSE

04], [Breu ASE 04], [Ceccato ICSM 05]...

• Program Structure– FEAT, [Sextant ICSM 05], [Robillard FSE 05]...

• Information Retrieval– [Marcus PhDThesis 03], [ICSE 03]

• Lexical Information– [Aspect Mining Tool], [Aspect Browser]

Foundations

Low-level Clues

Complementary

A Motivating Example

Recipe for Vegetable Soup

1. Place asparagus and onion in a saucepan with 1/2 cup vegetable broth. Bring the broth to a boil, reduce heat and let simmer until the vegetables are tender.

2. Reserve a few asparagus tips for garnish. Place remaining vegetable mixture in an electric ...

Recipe Writing Factory

Procedural instructions are easy to read

Recipe Writing System 1.0Choosing a Dominant Decomposition

asparagus • Place asparagus in a saucepan.• Reserve a few asparagus tips for

garnish• Garnish with reserved asparagus

tips.

flour

• Sprinkle flour into the butter. Do not let the flour brown.

butter

• Melt butter in the pan that was used for simmering the asparagus and onions…stir.

salt

• Sprinkle salt into the butter.

soy milk

• Stir the milk into the saucepan.

yogurt

• Whisk yogurt into the mixture

OO decomposition provides many benefits

Recipe Writing System 1.0Choosing a Dominant Decomposition

Action-Oriented CCCs Appear

asparagus • Place asparagus in a saucepan.• Reserve a few asparagus tips for

garnish• Garnish with reserved asparagus

tips.

flour

• Sprinkle flour into the butter. Do not let the flour brown.

butter

• Melt butter in the pan that was used for simmering the asparagus and onions…stir.

salt

• Sprinkle salt into the butter.

onion

• Place onion in the saucepan

yogurt

• Whisk yogurt into the mixture

OOP causes scattered actions

Evolution Causes Scattered Actions

Beef

place(p)checkIfFitsInPan(p)actuallyPlaceInPan(

p)

//Place the Beef into the panplace(Pan p){ checkIfFitsInPan(p); actuallyPlaceInPan(p);}

//Make sure the beef fits in the pan,//before Placing the Beef into the pancheckIfItFitsInPan(Pan p){ ...}

//Place the beef into the panactuallyPlaceInPan(Pan p){ ...}

Evolution causes more scattered actions

Maintenance Tasks Involve Scattered Actions

Something goes wrong when I place ingredients in the

saucepan.

Maintenance tasks require action location

How can we find actions in a noun's world?

Our Strategy1. Extract natural language clues2. Query clues and remodularize

Extract Clues with NLP

//Place the beef into the panBeef.place(Pan p){ checkIfFitsInPan(p); actuallyPlaceInPan(p);}

//Make sure the beef fits in the pan,//before placing the beef into the panBeef.checkIfItFitsInPan(Pan p){ ...}

//Place the beef into the panBeef.actuallyPlaceInPan(Pan p){ ...}

//Place the onion in the panOnion.place(Pan p){ ...}

place onion

place beef

place beef, check beef

place beef

//Make sure the beef fits in the pan,//before placing the beef into the panBeef.checkIfItFitsInPan(Pan p){ ...}

//Place the beef into the panBeef.place(Pan p){ checkIfFitsInPan(p); actuallyPlaceInPan(p);}

//Place the beef into the panBeef.actuallyPlaceInPan(Pan p){ ...}

Key Insight: Use language clues to extract actions

Query Clues and Remodularize

//Place the beef into the panBeef.place(Pan p){ checkIfFitsInPan(p); actuallyPlaceInPan(p);}

//Make sure the beef fits in the pan,//before placing the beef into the panBeef.checkIfItFitsInPan(Pan p){ ...}

//Place the beef into the panBeef.actuallyPlaceInPan(Pan p){ ...}

//Place the onion in the panOnion.place(Pan p){ ...}

simple query: place

Virtual Source File

From OO code to Action-Oriented code!

A Closer Look at the Extraction Process...

What to Extract

• information about actions– verbs– direct-objects

ex: Place the item in the panex2: Place the beef in the pan

Something goes wrong when I

place ingredients into the saucepan.

Something goes wrong when I

place beef into the saucepan.

Our Strategy: extract verb and direct object pairs

How to extract clues

Two types of extraction

class Beef{

//Chop the beef into cubes. chop(){ unwrap(); slice(CUBES); }

}

Extraction from comments

Extraction from method signatures

Extracting Clues from Comments

1. Part-of-speech tagging2. Chunking3. Role assignment

Remove an item from a cartRemove<v> an<dt> item<n> from<p> a<dt>

cart<n>Remove<v> [an item]<noun phrase>

[from [a cart] <noun phrase>] <prepositional phrase>

rule: when a verb is followed by a noun phrase, extract

Remove item

Extracting Clues from Signatures

1. Process method name2. Use hierarchy of rules to classify3. Use classification to extract

class Beef{ //Chop the beef into cubes. chop(){ unwrap(); slice(CUBES); }}

Extracting Clues from Signatures

1. Process method name2. Use hierarchy of rules to classify3. Use classification to extract

class Beef{ //Chop the beef into cubes. chop(){ unwrap(); slice(CUBES); }}

chop<v> [the beef]<noun

phrase>

chop<

v>

1. Process method name2. Use hierarchy of rules to classify3. Use classification to extract

class Beef{ //Chop the beef into cubes. chop(){ unwrap(); slice(CUBES); }}

Example Classification: method has no parameters and the method name is a verb

chop<

v>

Extracting Clues from SignaturesHand Tuned

Rule Tree

1. Process method name2. Use hierarchy of rules to classify3. Use classification to extract

class Beef{ //Chop the beef into cubes. chop(){ unwrap(); slice(CUBES); }}

Example Extraction Rule:

•get verb from method name

•get DO from class namechop beef

Extracting Clues from Signatures

Storing the Clues

Action-Oriented Identifier Graph (AOIG)

verb1 verb2 verb3 DO1 DO2 DO3

verb1, DO1 verb1, DO2 verb3, DO2 verb2, DO3

source code files

use

use

use

use

use

use

useuse

Action-Oriented Identifier Graph (AOIG)

chop grill puree onion beets steak

chop, onion chop, beets puree, beets grill, steak

source code files

use

use

use

use

use

use

useuse

Storing the Clues

Survey of AOIG-Builder's Accuracy How Accurate is Extraction?

Category No. of Verb-DO Pairs

No. of Verb-DO Pairs

Declarations

Comments

correct 50 43

incorrect 4 1

omitted 2 11

total 57 55

Good Precision and Recall

Avg. Precision

95%

Avg. Recall

83%

Costs of Building the AOIGper source file

• Space– O(number of methods)– minimal

• Time– O(number of methods)– 10 seconds for a large source file with

lots of multi-line comments

Reasonable cost - can be done incrementally

Contributions...

Action-Oriented View

• High-level CCCs instead of low-level– clear hypothesis of why actions are

CCCs

• Useful type of CCC– maintainers are looking for actions

Contributions (continued)

• Action-Oriented View• Combined NLP+Program Analysis

(NLPA)• Developed the AOIG for virtual

remodularization• Applied AOIG to

– Feature Location*– Aspect Mining– Working Set Recovery

Questions?

activ

ate

tool

save d

raw

ing

up

date

dra

win

g

un

do a

ction

object oriented systemaction

oriented system

Conclusions

• NLP– is sufficiently robust to use in SE tools

• NLP + Program Analysis– can extract more exact information from

code– can drive SE tools

We have shown how to virtually remodularize code into actions using the AOIG

Demo of Current/Future Work

• Complete Feature Location Methodology– sophisticated query expansion– action-oriented code view for program

understanding

Current/Future Work

generate

PowerGrid.timePassing()

PowerGrid.powerSurplusPowerGeneration.constructor

PowerGeneration.getGeneratedPower()

PowerSource.getCurrentPower()calls

SolarPowerSource.getCurrentPower()

PowerSource.getCurrentPower()

StandardPowerSource.getCurrentPower()

Experiments

• Subjects: QLI projects• Procedure:

1. Identify bugs or feature requests which require action location

2. Use tool to locate and squash bugs

For submission to ICSM Industry Track

User FeedbackDIRECTIONS:1. Place asparagus and onion in a saucepan with 1/2 cup vegetable broth. Bring

the broth to a boil, reduce heat and let simmer until the vegetables are tender. 2. Reserve a few asparagus tips for garnish. Place remaining vegetable mixture in

an electric blender and puree until smooth. 3. Melt butter in the pan that was used for simmering the asparagus and onions.

Stir while sprinkling flour, salt, and pepper into the butter. Do not let the flour brown. Allow the mixture to cook only 2 minutes. Stir in remaining 1 1/4 cups vegetable broth and increase the heat. Continue stirring until the mixture comes to a boil.

4. Stir the vegetable puree and milk into the saucepan. Whisk yogurt into the mixture, followed by lemon juice. Stir until heated through, then ladle into bowls. Garnish with reserved asparagus tips. Sprinkle with Parmesan cheese if desired.

Something goes wrong when I am

sprinkling things into the butter.

Recipe Developer, Fix it

Recipe Storage System: 2.0

Person

place(List ingredients, Pan p)

Placeable

place(Pan p)

Vegetable

place(Pan p)

Beef

place(Pan p)

Pepper

place(Pan p)

Onion

place(Pan p)

more complex system, actions are scattered further

Clever Worker1 pound fresh asparagus

1. Place asparagus in a saucepan.

2. Reserve a few asparagus tips for garnish

3. Garnish with reserved asparagus tips.

2 tablespoons all-purpose flour

1.2.3. Sprinkle flour into the butter.

Do not let the flour brown.

1 tablespoons butter

1.2.3. Melt butter in the pan that was

used for simmering the asparagus and onions…stir.

1 teaspoon salt

1.2.3. Sprinkle salt into the butter.

1 cup soy milk

12.3. Stir the milk into the saucepan.

1/2 cup yogurt

1.2.3.4. Whisk yogurt into the mixture

Person

sprinkleItems(List ingredients)

Clever Worker1 pound fresh asparagus

1. Place asparagus in a saucepan.

2. Reserve a few asparagus tips for garnish

3. Garnish with reserved asparagus tips.

2 tablespoons all-purpose flour

1.2.3. Sprinkle flour into the butter.

Do not let the flour brown.

1 tablespoons butter

1.2.3. Melt butter in the pan that was

used for simmering the asparagus and onions…stir.

1 teaspoon salt

1.2.3. Sprinkle salt into the butter.

1 cup soy milk

12.3. Stir the milk into the saucepan.

1/2 cup yogurt

1.2.3.4. Whisk yogurt into the mixture