Charlie Robinson / NCASQuality Assurance and Risk Management Services, Inc.

28-30 October 2008

Goddard Space Flight Center - Greenbelt, MD

PROCESS Preparing for an Assessment

RESULTS Using Historical Data

LESSONS-LEARNED Document Control Contracts Software Internal Audits

SUMMARY

AGENDA

GeneralInformation

AS9100Assignments

AssessmentPlan

BlankAssessment Card

Assessors InputTemplate

E-mails toTeam & NASA Rep.

EvaluationForm

In-Briefing AttendanceSheet

Preparing for an Assessment

The Assessment Team Takes Pre-Assessment Planning Seriously

Tea

m C

onfe

renc

e C

all

NCAS – 55% (18 out of 33) have Software problems (NOV 06 – SEP 08)SAC – 89% (32 out of 36) had Software problems (FEB 02 - JUN 05)

10 Suppliers have been assessed 2 to 4 times since FEB 2002

Using Historical Data

Using Historical Data (continued)

HIGH

LOW

RISK

AverageSmallMediumLargeUniversity

Items produced & delivered before

identifying the Environment &

Contamination Requirements

Contractually required Stress Relieve

Procedure not developed.

Unaware of DPAS rating

Wiring ‘Crimped” – Contract requires

“Welding”

Identify Requirements

HIGH

LOW

RISK

AverageSmallMediumLargeUniversity

No Configuration Management Process

Amendment to Instrument Coding Standard

made without Peer-Review

Life-cycle metrics not distributed as

required by contract

No Software Safety Program

Coding Standards in SDP not used.

A “Draft” Satellite Simulator Software is

being used.

Software

Using Historical Data (continued)

Lessons-LearnedDocument Control

47 Non-compliances & 37 Observations

Internal Conflicts between proceduresMatrix of procedure inter-relationships is a way to avoid these conflicts

Industry Specifications not currentA consolidated listing of all industry specifications that are maintained

Procedures not maintained to reflect current practicesSupervisor responsibility & Internal Audits should provide direction and assurance that procedures are current

Lessons-LearnedContracts

46 Non-compliances & 45 Observations

There is not a reliable internal flow-down of requirements Clean-room, DPAS, Calibration, QMS, Record Control, etc.

The review is flawed as suppliers agree to requirements they do not meet.Suppliers often do not know their basic system and therefore are not able to identify those contractual requirements that they do not satisfy.

CDRL Items are either late or revised without updating the customer.Assure that a single person is responsible for tracking the status of all CDRLs

Lessons-LearnedSoftware

38 Non-compliances & 24 Observations

The software controls are not all encompassingControl of software includes deliverable, design, test, and operational software.

There is not a compliant Software Safety Program (SSP)Evaluate the SSP against NASA-Std-2100

The individual who developed the software integration are developing the integration test.The need for objective independence is compromised by having the individual who developed the software integration also develop the integration test.

Lessons-LearnedInternal Audits

18 Non-compliances & 14 Observations

Contractual compliance and procedural support are not always part of the internal audit process.Without auditing the contractual requirements the management will believe the Quality Management System is appropriate when in fact it does not support contractual requirements.

Internal Audits are behind schedule.This is a discipline that is necessary to avoid requirement creep or slippage. As Dwight D Eisenhower said “The uninspected inevitably deteriorate.”

The internal auditors do not audit or are not qualified to audit special processes.Assure that subject mater experts are used when auditing software, soldering, heat treat, plating, etc.

Summary

Individual commitment to a group effort - that is what makes ateam work,company work,society work,civilization work.

Vince Lombardi