minimal instrumentation for software feature location sharon simmons dennis edwards norman wilde...
TRANSCRIPT
Minimal Instrumentation for Software Feature Location
Sharon Simmons
Dennis Edwards
Norman Wilde
Eric Daniels
UWF - Simmons 2
Software Reconnaissance
Fall 2009 Showcase
Where in this program is feature X implemented?
100’s KLOC, 10’s MLOCModification requiredInteractions forgotten
2. Run tests with the featureFeature is
active
Feature is NOT active
~Set difference
Solution: track execution
3. Compare what was executed
1. Run tests without the feature
Components executed in the first group of tests, but not in the second groupMarker ComponentsBegin exploration here!
UWF - Simmons 3
(demo of version 3) http://www.cs.uwf.edu/~recon/recon3/r3wDemo.htm
TraceGraph 4
Fall 2009 Showcase
RowInstrumentation point
Columntest case
Col 1without feature
Col 2+with feature
Identifies “markers”
Gives big picture
Subroutine (En… 0 be_aas_bspq3.c /research/ope… 147
UWF - Simmons 4
Software Recon Status
Fall 2009 Showcase
Developed for several years at UWF– Stable, usable, and free– Published works enumerate benefits and limitationsBrief Benefits– Identifies a small number of marker components– Identifies good places to start efficient code exploration– Can decrease software maintenance time
UWF - Simmons 5
Instrumentation AlternativesAtom (Digital Equipment Corp)
– Instrumentation at load time
Pin (Intel Corp)– Instrumentation at dynamic link time
CodeTEST (Freescale Semiconductor)– Instrumentation of preprocessed source code
Recon (UWF)– Instrumentation of source code
Gcov (GNU)– Instrumentation by (GNU) compiler
Fall 2009 Showcase
None completely successful
UWF - Simmons 6
Problems Encountered
Fall 2009 Showcase
Newer systems time-sensitiveDeadlines must be metTracing can not be intrusiveTarget must execute normally
Initialization was hours instead of minutesDeadlines were not metTracing intrusive and prevented normal execution
Hard / soft real-timeRace conditionsTiming faults
UWF - Simmons 7
• Instrument preprocessed source• Efficient code coverage tracing• Build IDB prior to execution• Provide facilities to partition execution into
intervals• Simplify installation and use
Fall 2009 Showcase
Our Approach
UWF - Simmons 8
Information Flow
Fall 2009 Showcase
Driver
Collection Engine
Target
IDB
Bitstring Traces
Trace Records
Data Joiner
Source
Instrumenter
Interval Delimiter
Test
Cases
During Compilation
During Execution
After Execution
UWF - Simmons 9
MinInst Overview
Fall 2009 Showcase
Instrumentation point represented as bit– 1MB RAM ≈ 8.4M instrumentation points
Ordered set of bits (bit string) represents code– Bit index identifies IDB record– Bit string cleared at interval start– Bit set when instrumentation point executed
0 not executed; 1 executed– Execution order not retained– Execution frequency not retained– Bit string saved at interval end
UWF - Simmons 10
MinInst Details
Fall 2009 Showcase
Two bitstrings allocated in shared memory– Active (A): currently accessed by target– Passive (P): prepared for next interval
Collection Engine– Started as first instruction in main()1. Wait for signal (no overhead here)2. Exchange A & P bitstrings (pointer swap)3. Save P bitstring to trace file (I/O overhead)4. Clear P bitstring for next interval5. Go back to step 1
UWF - Simmons 11
Version A• Instruments C/C++• Linux platform• CodeTEST
– Identifies instrumentation points– Builds IDB
• MinInst– Inserts instrumentation code– Builds bit index IDB map
• Driver– Delimits trace intervals– Performs test cases
• Collection Engine– Records bitstring trace at runtime
• Joiner– Combines IDB, MAP and bitstring trace– Creates usable trace file
Fall 2009 Showcase
UWF - Simmons 12
Information Flow
Fall 2009 Showcase
Driver
Collection Engine
Target
IDB
Bitstring Traces
Trace Records
Data Joiner
Source
Instrumenter
Interval Delimiter
Test
Cases
During Compilation
During Execution
After Execution
Map
CodeTEST
MinInst
UWF - Simmons 13
Test CasesGenerated Code
– 1 file, simple functions (5 stmts, 5 inst pts)– 15KLOC, 30KLOC, 45KLOC, 60KLOC– CodeTEST: +200% MinInst: +11%
Apache web server (httpd)– 550 files, 300KLOC, multithreaded, multiprocess– CodeTEST: +717% MinInst: +1%
Openarena & OpenGL– 1K files, 700KLOC, time sensitive, multiplayer, networked– CodeTEST: +9371% MinInst: +6%
Fall 2009 Showcase
UWF - Simmons 14
Shortcomings• Complexity of use• Reliance on CodeTEST• CodeTEST failed some test cases• Execution differences between platforms• Licensed software• Unsupported for a while
Fall 2009 Showcase
UWF - Simmons 15
Version B• Prototype written in C for C/C++ source
– Removed CodeTEST dependency – Tested in Linux– Windows port is pending
• Simplified installation and use– Installation/removal scripts– Integrated into GCC/Make in Linux– Integrated into Visual Studio IDE
• Interval options– Timed or user controlled
Fall 2009 Showcase
UWF - Simmons 16
Information Flow
Fall 2009 Showcase
Driver
Collection Engine
Target
IDB
Bitstring Traces
Trace Records
Data Joiner
Source
Instrumenter
Interval Delimiter
Test
Cases
During Compilation
During Execution
After Execution
MinInst
UWF - Simmons 17
SummaryVersion B status
– Execution time unaltered from Version A– Linux and Windows XP– GCC and Visual Studio– Test cases in progress
Next steps– Conclude test cases– Northrop Grumman large system testing– Complete distribution package
Future Work– Modify TraceGraph for bit string operation
Fall 2009 Showcase
UWF - Simmons 18
Information Flow
Fall 2009 Showcase
Driver
Collection Engine
Target
IDB
Bitstring Traces
Source
MinInst
Interval Delimiter
Test
Cases
During Compilation
During Execution
After Execution
UWF - Simmons 19
Contact Information
Sharon Simmons [email protected]
Dennis Edwards [email protected]
Norman Wilde [email protected]
Fall 2009 Showcase