11

FSO Assessment ofFermilab QA Program Status

September 14 – 18, 2009

2

Purpose of Assessment

• Evaluate the adequacy of Fermilab’s QA program documents

• Evaluate the status of Fermilab’s efforts to implement their QA program

3

Assessment Scope• Program• Personnel Training & Qualification• Quality Improvement • Documents & Records• Work Processes• Design• Procurement• Inspection & Acceptance Testing• Management Assessment• Independent Assessment• Suspect/Counterfeit Items Prevention

4

Assessment Scope (continued)• Accelerator Division• Particle Physics Division• Technical Division• Computing Division• Business Services Section• Workforce Development & Resources Section• Facilities Engineering Services Section• Environment, Safety & Health Section• Office of Quality & Best Practices

55

Assessment Team• Judy Malsbury, PPPL• Jessie Wilke, BNL• Bob Arthurs, ANL• Margaret Legel, NBL• Matt Cole, SC-31.1• Tom McDermott, SC-CH• Karl Moro, SC-CH• Tracy Sims, SC-CH• John Adachi, SC-CH

6

Strengths

• The Laboratory Director understands the importance of QA and is providing leadership to drive the culture change necessary to institute the Fermilab Integrated Quality Assurance program. Director communicates directly with Lab

staff to convey the importance of the IQA Transition from expert-based to process-

based culture

7

Strengths (continued)• The function and composition of the

Fermilab Assurance Council, under the leadership of the COO, ensures upper management involvement in resolving significant cross-cutting issues.

• OQBP is applying project management discipline to the Laboratory’s efforts to develop and implement quality assurance throughout Fermilab; demonstrated by the Integrated Quality Assurance project plan.

8

Strengths (continued)

• The professional QA expertise of the OQBP QAEs, and the dedication of the line organization’s QARs is evident.

• Fermilab’s plans for a Consolidated Assessment Schedule should provide a mechanism to ensure appropriate assessment coverage of Laboratory operations.

9

Strengths (continued)• The As-Is Process provided significant

value to the efforts to institute the IQA– Established the base-line of the Lab’s

conformance with the requirements of the IQA and FICAP

– Developed integrated CAPs for the institution and line organizations

– Piloted a more formal and rigorous assessment and corrective action process with full participation of the line organization

10

Strengths (continued)

• As part of the procurement process for complex items, vendors are often requested to complete a Technical Questionnaire, or submit a prototype, to test the vendor’s knowledge and capability.

11

Strengths (continued)• The draft Fermilab Assessment Manual requires

Management System Owners and D/S/C management to verify closure and validate effectiveness of corrective actions.

• The CD Problem Management Process and Procedure is an effective mechanism for the identification, analysis and resolution of problems.

• AD has piloted an innovative use of an iTouch device to communicate procedural steps through video to operators in the field.

12

Concerns• The draft Fermilab Management

Assessment Procedure does not require the identification of the assessment criteria, or provide direction or guidance on how to scope assessments.

• The criteria for determining the need for a complex or simple root cause analysis, contained in the Appendix of the Fermilab Corrective and Preventive Action Procedure, lacks sufficient direction for consistent application.

13

Concerns (continued)Terms in the draft Fermilab Assessment Manual: • Minor findings, significant findings, special

assessments, Fermilab Director’s Assessments, and third party assessments are not sufficiently defined to allow for consistent application.

• Some terms in the Definitions Section that are useful for characterizing the significance of assessment issues are not used in the body of the document nor in other documents germane to assessments (IQA, FICAP, draft Management Assessment and Corrective & Preventive Action procedures).

14

Concerns (continued)

• While there is a high level of awareness of the subject of S/CI at Fermilab, and S/CI have been effectively segregated from use, there is some confusion in the field regarding the requirements for final disposition of S/CI in the draft procedure being developed.

15

Concerns (continued)

• Integrated Quality Assurance Chapter Six Design indicates that the Laboratory will staff the position of Fermilab Chief Engineer, however interviews indicated that the decision to create and fill such a position has not been finalized.

16

Significant Concern• The draft Fermi National Accelerator Laboratory

Engineering Manual, Revision 0.2, does not provide sufficient detail and rigor to effectively implement Criterion 5(a) Work Processes and Criterion 6 Design of DOE O 414.1C Quality Assurance, Section 5.3 Work Process Control and Chapter Six Design of the FNAL Integrated Quality Assurance program document, or to adequately address the inadequacies identified in the Root Cause Analysis for the Large Hadron Collider Magnet System Failure.

17

Conclusions• With the exception of the areas of

concern, Fermilab’s efforts to institute the IQA are on track.

• Achieving the necessary culture change is the most difficult and challenging aspect of successfully instituting the Integrated Quality Assurance program; sustained emphasis by top management is necessary to successfully drive the culture change.

1818

Next Steps

• Assessment team members will provide the write-ups of their sections to the Assessment Team Leader by October 2, 2009.

• Assessment Team Leader will produce draft assessment report by October 28, 2009.

• FSO provides draft report to Fermilab for factual accuracy review.

• Assessment team considers Laboratory’s comments, revises report as necessary, and finalizes assessment report.

• FSO issues assessment report to Fermilab.

19

Thanks

• FSO, Fermilab management, QAEs, QARs, Fermilab staff were open and cooperative; this greatly facilitated the efforts of the assessment team.

• The team members did the work necessary to serve on this assessment in addition to doing their “regular jobs”.

• The management of the organizations of the assessment team members allowed them to take time away from their “regular jobs” to serve on this assessment.