nasa osma sas ‘03
DESCRIPTION
NASA OSMA SAS ‘03. Software Requirements Analysis As Fault Predictor Dolores R. Wallace SRS Technologies Software Assurance Technology Center http://satc.gsfc.nasa.gov [email protected] William M. Wilson SRS Technologies/SATC [email protected]. - PowerPoint PPT PresentationTRANSCRIPT
SAS03/FaultPrediction 1
NASA OSMA SAS ‘03
Software Requirements Analysis As Fault Predictor
Dolores R. WallaceSRS Technologies
Software Assurance Technology Centerhttp://satc.gsfc.nasa.gov
William M. WilsonSRS Technologies/SATC
SAS03/FaultPrediction 2
Software Requirements Analysis As Fault Predictor
Presentation Outline1. Research Rationale & Objective2. Requested Project Data3. Planned Approach4. Data Acquisition & Analysis Obstacles5. Results & Findings6. Conclusions7. Recommendations
SAS03/FaultPrediction 3
Research Rationale & Objective
Rationale• The earlier in the lifecycle that software faults can be
found the less expensive it is to correct them.• The earliest opportunity to preclude software faults is
during the development of the system's requirements.
Objective• To determine if software requirements analysis results can
be used as a predictor of software faults.
SAS03/FaultPrediction 4
Requested Project Data
• Requirements Specification– Accessible in electronic media in MS Word format
for use with the ARM tool– Conforming to NASA-STD-2100– Specification statements identified by hierarchical
numbers
SAS03/FaultPrediction 5
Requested Project Data – Cont.
• Verification Test Matrix– Requirements to Test or vice versa
- OR -• Traceability Matrix
– Requirements to Design Elements– Design Element To Software Component
- AND -• Test Plans
– Tests vs. Software Components.
SAS03/FaultPrediction 6
Requested Project Data – Cont.
• Failure Reports/Data - Preferably in DDTS format– Including: test id, module id, CSCI id, release/build id,
suspected cause of failure and resolution (e.g., actual cause of failure - what was fixed)
• Configuration Control Reports– Baseline item changed -e.g., Requirement, Design,
Code– Reason/Source of change – e.g., RID, Failure Report,
PR, etc.
SAS03/FaultPrediction 7
Planned Approach
1. Use ARM tool to analyze projects’ software requirements.
2. Collected projects’ failure data in a DDTS database.
3. Organize the data to represent same components for each project.
4. Conduct correlation analysis.5. Study results of step 4 to prove or disprove
hypothesis.
SAS03/FaultPrediction 8
ARM TOOL Reports
Identifies and Counts:• Size of Requirements Document
– Lines of Text– Numbered Paragraphs– Number of Specification Statements– Number of Unique Specification Subjects
• Number of Specifications Containing– Weak, Optional and/or Incomplete Phrases.– Compound and/or Complex Statements.
• Requirements Document Structure– Depth and Distribution of Specifications
SAS03/FaultPrediction 9
Data Acquisition & Analysis Obstacles
• Common Problems– No commonality or standardization of data or formats
across project.– Current projects are reluctant to provide data to outside
organizations.– Data from completed projects is incomplete, not in a
usable form or is not accessible.
SAS03/FaultPrediction 10
Data Acquisition & Analysis Obstacles Cont.
• Specific Problems– No project staff available to guide through the mass of
documents in the collection– Documents locked on a WEB site– Documents in “.pdf” format– Variation of format and media within project– Complete set of data for a specific Build/Release not
available– Data held by support contractor and released only on a
“Project Need-To-Know” basis
SAS03/FaultPrediction 11
Results & Findings
• Data provided by four projects– Projects A and B
• Major subsystems of operational information processing system
– Projects C and D• Flight instrumentation packages
SAS03/FaultPrediction 12
Results & Findings Cont
Summary Of Project's Data
PROJECT Tot
al S
peci
ficat
ions
Wea
k Sp
ecifi
catio
ns
Inco
mpl
ete
Com
poun
d
Max
Dep
th
Spec
s at
Low
est L
evel
Tot
al P
robl
ems/
Cha
nge
Req
uest
s
Req
uire
men
ts
Des
ign
Cod
e
Tes
t
Doc
umen
tatio
n
Enh
ance
men
t
Oth
er
Non
e
A 849 256 10 241 5 226 3066* 0 183 2013 9 22 0 267 572B 226 47 13 39 5 38 1132** C 356 75 0 100 6 117 323*** 12 34 91 3 5 69 102 7D # 121##
* System Failure Reports** Details Not Yet Available*** Software Change Requests# System Detailed Design Spec not Requirement## Unformatted Text Problem reports
SAS03/FaultPrediction 13
Conclusions
• Data from many more projects is required to improve the possibility of finding a number of data sets with sufficient commonality to support analysis.
• Analysis approach needs to be expanded to include determining if failures attributed to documentation and design flaws are in reality related to deficiencies in requirements.
SAS03/FaultPrediction 14
Recommendations
Advocate and support:• The use of NASA-STD-2100 documentation
standard• The use of a format to report problems and
failures that is compatible with DDTS• The creation and use of centralized Center
repositories for data from completed projects.