quality leadership forum
DESCRIPTION
Quality Leadership Forum. Software Quality Assurance at GSFC Dr. Linda H. Rosenberg Chief Scientist for Software Assurance Office of Systems Safety and Mission Assurance 301-286-0087 [email protected]. Discussion Areas. NASA Software Activities Software Working Group - PowerPoint PPT PresentationTRANSCRIPT
July 18, 2001 Mission Success Begins With Safety
Quality Leadership Forum
Software Quality Assurance at GSFC
Dr. Linda H. Rosenberg
Chief Scientist for Software Assurance
Office of Systems Safety and Mission Assurance
301-286-0087
July 18, 2001 Mission Success Begins With Safety
Discussion Areas
NASA Software Activities
Software Working Group
NASA Software Initiative Implementation Plan
GSFC Software Assurance Activities
Software Process Improvement – CMMI
Software Safety and Reliability
Software Quality Metrics
IV&V
July 18, 2001 Mission Success Begins With Safety
July 18, 2001 Mission Success Begins With Safety
SWG Initial Tasks
1 - Define criteria for use of IV&V on a project
2 - Standards evaluation
– Review IEEE 12207 for potential NASA use
– Review draft of NPG 2820
– Review draft NPD for IV&V
3 - Prepare a plan for improving software process
– Implementation of software metrics program
– Implementation of process improvement model
– Establishment of Center Software Engineering Process Groups (SEPG)
July 18, 2001 Mission Success Begins With Safety
SWG Task 3 - Metrics
Set of metrics finalized summer 2000
Objectives
Provide project managers with usable information
Provide agency with information on software trends
Provide a measure to assess improvement
2 test projects per Center started Fall 2000 for 1 year
(GSFC projects AURA & AQUA)
Developing database for metrics entry and analysis
July 18, 2001 Mission Success Begins With Safety
NASA Software Initiative Implementation Plan
Goal: Advance software engineering practices (development, assurance, and management) to effectively deliver the scientific and technological objectives of NASA.
Strategies:1. Develop and implement Agency-wide and Center plans for continuous software
process and product improvement in NASA and Contractor developed software; also establish infrastructure and measurement system
2. Improve safety, reliability, and quality of software products through the integration of sound software engineering principles and standards.
3. Provide input for research based on identified software problem areas and infuse research results
4. Improve software engineering knowledge base in NASA, and implement strategies for attracting, retaining software engineers
July 18, 2001 Mission Success Begins With Safety
July 18, 2001 Mission Success Begins With Safety
GSFC Software Development Process Improvement
Purpose - improving the processes and practices in use at GSFC using the Capability Maturity Model Integrated (CMMI) levels of maturity (ML) as a measure of progress.
Scope - process improvement effort that will be undertaken with the goal of raising GSFC from its current state to a CMMI Defined maturity level (L3). All projects defined by NPG 7120.5 or otherwise identified by GSFC’s Center Director will participate in this effort.
July 18, 2001 Mission Success Begins With Safety
Capability Maturity Model Integrated (CMMI)
Level Process Areas
Organization innovation and deploymentCausal analysis and resolutionOrganizational process performanceQuantitative project managementRequirements developmentTechnical solutionProduct integrationVerificationValidationOrganizational process focusOrganizational process definitionOrganizational trainingIntegrated project managementRisk managementDecision analysis and resolutionRequirements managementProject planningProject monitoring and controlSupplier agreement managementMeasurement and analysis
5 Optimizing
4 QuantitativelyManaged
3 Defined
2 Managed
1 Initial
SoftwareDevelopment
SW
SystemsSE
Software Acquisition
SA
CMMI
For Pilots:Emphasis - SW CMMAs appropriate - SE CMM
SA CMM
GSFC
GOAL
July 18, 2001 Mission Success Begins With Safety
Pilot Project Selection
Project W
FLT SW GND SW
Instr 1 Instr 2
Project X
FLT SW GND SW
Instr 1 Instr 2
Project Y
FLT SW GND SW
Instr 1 Instr 2
Project Z
FLT SW GND SW
Instr 1 Instr 2
July 18, 2001 Mission Success Begins With Safety
Schedule
GSFC Implementation plan to HQ July 2001Management Oversight Group
Member identification July 2001Initial meeting August 2001Training in CMMI September (1/2 day or 3 day option)
Engineering Process GroupMember identification July 2001Training in CMMI September 2001(3 day course)Training in Risk Management October 2001
Pilot ImplementationPilot identification by October 1, 2001Pilot study complete October 1, 2002
Evaluation of Pilot and roll out January 1, 2003
July 18, 2001 Mission Success Begins With Safety
What is meant by “safety”
A system/product is Safe when:There is little to no chance for it to blow up, break, malfunction, or
otherwise fail in such a way as to potentially injure someone
Something is Critical when there is a potential for:
Serious injury or deathSerious impact to the bottom line, or Bad publicity, public reputationVital information is accessible to the
wrong folksA system/product is Not Safe when:Someone could die or be seriously injured
NASA includes possible destruction of vital equipment as well
July 18, 2001 Mission Success Begins With Safety
Standards
NASA Standards (http://standards.nasa.gov/sitemap.htm)NPG 8715.3 NASA Safety Manual NSTS-1700-7B Safety Policy and Requirements for Payloads (Shuttle and ISS) NASA-STD-8719.13A NASA Software Safety Standard NASA-GB-A302 Software Formal Inspections Guidebook NSTS-22254 Methodology for Conduct of Space Shuttle Program Hazard Analyses SSP-50038 Computer-Based Control System Safety Requirements, ISS ProgramNPD/NPG 8730 “NASA IV&V Processes “
IEEE StandardsIEEE 12207 Information Technology - Software Life Cycle Processes IEEE 830-1998 Recommended Practice for Software Requirements SpecificationsIEEE 1016-1998 Recommended Practice for Software Design DescriptionsIEEE 1228-1994 Standard for Software Safety Plans
Other StandardsMIL-STD-882D System Safety Program Requirements (C version January 19, 1993)DO-178B Software Considerations in Airborne Systems and Equipment Certification (Federal Aviation Administration).ISO 9000-3 Guidelines For The Application Of ISO 9001 To The Development, Supply, Installation And
Maintenance Of Computer Software
July 18, 2001 Mission Success Begins With Safety
Software Reliability
The probability that software will not cause the failure of a system for a specified time under specified conditions. The probability is a function of the inputs to and use of the system, as well as a function of the existence of faults in the software. The inputs to the system determine whether existing faults, if any, are encountered. [AIAA] [IEEE 982]
IEEE 982.1-1988 Software Reliability Management : “The process of optimizing the reliability of software through a program that emphasizes software error prevention, fault detection and removal, and the use of measurements to maximize reliability in light of project constraints such as resources, schedule and performance.”
July 18, 2001 Mission Success Begins With Safety
Hardware vs. Software Reliability
Burn in Useful Life Wear out Integration Useful Life Obsolete & test
Hardware Failure Rate Software Failure Rate
Hardware reliability == Software reliability
July 18, 2001 Mission Success Begins With Safety
Definitions: Safe vs. Reliable
A system is:
safe if it doesn’t kill anyone, or the system itself, while either performing its normal operations or, when unable to perform correctly, “fails-safe” .
reliable if it performs the required functions within specified parameters/environment and within predicted working timeframe consistently
Some consider Software to be very reliable, in that it does just what its programmed to do, over and over and over again. It doesn’t wear out or ‘break’. However, Linda
will give you the real picture on Software reliability!
July 18, 2001 Mission Success Begins With Safety
Requirements
How Do We Assure / Measure Software Quality?
Design
Coding
Testing
•Can you test the requirements?•Are the requirements complete?•Are you testing each requirement?
•How much testing is necessary?•How hard is it to fix the components?•Can I reuse any components?•What is the quality and complexity of the code?
How many errors still remain?What is the reliability?When can I stop testing?
July 18, 2001 Mission Success Begins With Safety
IV&V Approach
Req Design Code Test (Verification & Validation)Unit Integration Acceptance
Req Design Code Testing Unit
Test (Verification & Validation) Integration Acceptance
Clean Room Approach
Traditional Software Development
V&V
iV&V
Req Design Code Test (Verification & Validation)Unit Integration Acceptance
IV&V Implementation
IV&V