radar cross section measurement facilities … the radar committee of the signature ... range...

Radar Cross Section Measurement Facilities Documentation Standard

May 2002

(31 May 2002 Draft)

Prepared by

Radar Committee

Signature Measurements Standards Group

RANGE COMMANDERS COUNCIL

Published by

Secretariat

Range Commanders Council

U.S. Army White Sands Missile Range

New Mexico 88002-5110

TABLE OF CONTENTS

ACRONYMS 3

CHAPTER 1 - INTRODUCTION 4

1.1 General 4

1.2 Scope 5

1.3 Purpose 5

1.4 Limitations 5

1.5 Reference Documents 6

1.6 Definitions 6

CHAPTER 2 - ESTABLISHING THE PROCESS 12

2.1 Overview 12

2.2 Roles and Responsibilities 12

2.3 Timeline and Reports 16

CHAPTER 3 - MAINTAINING THE PROCESS 21

3.1 Maintaining the Certification Body Pool of Reviewers 21

3.2 Maintaining Continuity and Consistency 22

3.3 Follow-up Assessments and Re-Certification 22

APPENDIX A - BIBLIOGRAPHY 24

ACRONYMS

ANSI American National Standards Institute DoD Department of Defense

DSN Defense Switched Network IEC International Electrotechnical Commission

ISO International Standards Organization NCSL National Conference of Standards Laboratories

NCSLI National Conference of Standards Laboratories International NIST National Institute of Standards and Technology

NISTIR National Institute of Standards and Technology Interagency Report RCC Range Commanders Council

RCS Radar Cross Section SMSG Signature Measurements Standards Group

CHAPTER 1 INTRODUCTION

1.1 General 1.1.1 The Radar Committee of the Signature Measurements Standards Group (SMSG) of the Range Commanders Council (RCC) has prepared this voluntary standard to foster improved quality in measurements of radar signatures at RCC member ranges and associate facilities. The range commanders encourage other organizations engaged in the practice of radar cross section (RCS) measurements to adopt this standard. Because the life-cycle measurement requirements of signature design, development, production, and maintenance spans a universe of facilities that are not under the purview of the RCC, fostering universal adoption of this standard will ensure more uniformly improved processes and measurement data products.

1.1.2 The RCS measurement community set out in 1994 to uniformly raise the quality, reliability, and portability of measurements made throughout the community. Working with the National Institute of Standards and Technology (NIST), the Radar Committee of the SMSG researched existing standards and related documents that might serve as a starting point. That course of action was taken because much work related to quality measurement systems had been accomplished at the international level. Eventually, American National Standard for Calibration – Calibration Laboratories and Measuring Test Equipment – General Requirements Z540-1-19911 was adopted as the foundation for the National RCS Measurement Facilities Certification Program. The ANSI Z540 standard is the United States implementation of International Standards Organization (ISO) Guide 25.

1.1.3 Following the principles practiced by the international standards-setting community, the Radar Committee developed and distributed through four revision cycles evaluation criteria against which participating measurement facilities would be assessed. A scoring system was devised and included with the evaluation criteria. A certification process was developed and shared with the community for comments and recommendations. Annual conferences were widely advertised in the RCS community and open discussion was encouraged over a span of more than four years.

1.1.4 Three significantly different Department of Defense (DoD) RCS measurement facilities participated in a demonstration program to exercise and refine the certification process and evaluation criteria. Some adjustments and clarifications were made, with a fifth version of the evaluation criteria published for subsequent use. The process (including evaluation criteria) and a summary of the DoD demonstration program were published in 2001 by the RCC in two volumes (see references 1.4.5 and 1.4.6).

1.1.5 This document represents the culmination of that process and experience to-date. Periodic revisions may be prudent should the environment or lessons-learned suggest substantive

1 See references 1.4.1 and 1.4.2; hereinafter ANSI Z540 or simply Z540

changes. Readers are encouraged to review the most recent versions of the Key Documents (listed below at paragraph 1.4 References) as they constitute the core information for which this standard serves as a framework.

1.2 Scope 1.2.1 This standard prescribes the certification system and certification scheme that form the orderly implementation and application of the processes that formally record the operations and capabilities of RCS measurement ranges. The scope of capabilities attainable with the utilization of this standard requires a careful consideration of tradeoffs. Guidance concerning such tradeoffs is provided herein and in the Key Documents.

1.2.2 This standard is not a stand-alone document. It relies heavily on existing standards, guides, and related documents. The approach utilized by the Radar Committee was the identification and adaptation of an existing, internationally recognized standard, combined with a clarifying handbook, recommended minimum evaluator qualifications, and evaluation criteria. See 1.5, Reference Documents, for details.

1.3 Purpose 1.3.1 This standard provides the necessary criteria on which to base a documentation set that describes the operations and capabilities of an entity engaged in RCS measurements. The ultimate purpose is to ensure uniformly improved RCS measurements across member ranges, other DoD facilities, industry, and academia. That purpose is achieved by establishing common terms and definitions, formats for reports of measurements, and shared practices such as inter-facility (inter-range) comparisons of calibration artifacts.

1.3.2 The policy of the SMSG is to promulgate minimal changes to this core standard as is possible, relying on updates of the supporting documents to keep information current. Those documents will be updated independently as lessons are learned, concepts refined, and the science of RCS measurements advances. To determine the current revision status, contact the RCC Secretariat at White Sands Missile Range, New Mexico at (505) 678-1107 or DSN 258-1107.

1.4 Limitations 1.4.1 Compliance with this standard is voluntary.

1.4.2 The RCC is not a certifying body, nor is the SMSG or any other RCC entity or representative. The RCC SMSG serves as a facilitator of the certification process, coordinating activities and communications by and between the parties through formal and informal dialog and correspondence.

1.4.3 Certification is awarded by a certification body comprised of volunteers from the RCS community who have been vetted against published minimum qualifications.

1.4.4 Achieving certification does not indicate a facility has a capability to measure radar signatures to a specific or finite degree of accuracy or certainty.

1.5 Reference Documents 1.5.1 A full understanding of the concepts of calibrations, calibration artifacts, and uncertainty of measurements is necessary to effectively implement this standard. The documents listed below constitute the core information for which this standard serves as a framework. Additional references are provided in the bibliography at Appendix A.

1.5.1.1 Guide 25 – General Requirements for the Competence of Calibration and Testing Laboratories, International Standards Organization, Geneva, Switzerland, 1990. Note: Guide 25 is being superceded by ISO/IEC 17025, which achieved the status of “International Standard to be revised” on 8 March 2001. This standard will rely on Guide 25 until at least 31 December 2005.

1.5.1.2 American National Standard for Calibration – Calibration Laboratories and Measuring and Test Equipment – General Requirements, ANSI/NCSL Z540-1-1994, NCSL, 1994.

1.5.1.3 American National Standard for Expressing Uncertainty – U.S. Guide to the Expression of Uncertainty in Measurement, ANSI/NCSL Z540-2-1994, NCSL, 1994.

1.5.1.4 Handbook for the Interpretation and Application of ANSI/NCSL Z540-1-1994, NCSL, 1995.

1.5.1.5 RCS Certification for Static and Dynamic RCS Measurement Facilities, Volume I – Certification Process, Radar Committee, Signature Measurements Standards Group, Range Commanders Council, Document Number 804-01, Volume I (revised), August 2001.

1.5.1.6 RCS Certification for Static and Dynamic RCS Measurement Facilities, Volume II – Demonstration Program Results, Radar Committee, Signature Measurements Standards Group, Range Commanders Council, Document Number 804-01, Volume II (revised), August 2001.

1.6 Definitions 1.6.1 Before addressing specific definitions, some discussion about relative terminology is in order. The terms certification, registration, and accreditation are sometimes misconstrued, misunderstood, or misapplied.

1.6.1.1 The assessment of a quality system against the requirements of one of the ISO standards and the subsequent issuance of a certificate that confirms a quality system is in conformance with a quality standard’s requirements is variously referred to in different countries and technical communities as certification or registration. The terms certification and registration are used interchangeably in some countries and technical communities – a practice that reflects different national or business culture preferences. Likewise, the bodies that issue ISO 9000 certificates – certification bodies – are referred to in some countries as registration bodies, or registrars. Again, these different appellations refer to the same type of body.

1.6.1.2 For an RCS measurement facility to become certified, it must first become registered. To become registered, a facility forms its own internal review committee to review and verify its documentation set (see Range Book below) against published evaluation criteria. This process is called an internal self-review audit or registration in ISO vernacular. Once a facility has registered its documentation set, range management personnel normally signify registration is

complete by placing the names of the internal review committee on the opening page of the document.

1.6.1.3 Accreditation is another term which, in the ISO 9000 context, is also sometimes used – wrongly – as a synonym for certification or registration. Accreditation is the procedure by which an authoritative body gives formal recognition that a body or person is competent to carry out specific tasks. In the ISO 9000 context, it relates to the work of national accreditation bodies that have been set up in a number of countries to provide some measure of control over the activities of quality system or environmental management system certification bodies. An accreditation body will accredit – or, in simpler language, approve – a certification body as competent to carry out ISO 9000 certification of quality management systems, or ISO 14000 certification of environmental management systems, in specified business sectors. The National RCS Measurement Facilities Certification Program is not an accreditation program—the RCC does not have the authority to certify, accredit, or endorse that a body or person has complied with a standard or is competent to carry out specific tasks. The National RCS Measurement Facilities Certification Program certifies—through independent third-party reviewers using published evaluation criteria—that a measurement facility has thoroughly documented its systems, process, staffing, results, and other factors appropriate for the facility being reviewed.

1.6.1.4 Compilation of a Range Book, followed by a rigorous and successful internal review to complete the registration process are important steps in the formal documentation of range performance. For an RCS measurement facility to become certified, the facility must undergo an external review by third-party reviewers. The RCC SMSG is a body that offers guidance in the preparatory phase and administers the third-party review process.

1.6.2 Most of the definitions that follow are a subset of the definitions related to standardization and related activities established in ISO/IEC Guide 2:1996, which is available for purchase through the ISO web site (www.iso.org, approximately US $105). Some of the terms and definitions below have been adopted subsequent to the publication of some of the Key Documents (see 1.4 Reference Documents above); therefore, minor discrepancies may occur between this standard and those documents. Some terms defined below do not appear in this document, but may appear in related or referenced documents. In some cases, terms are defined herein to illustrate what is not part of this standard or the National RCS Measurement Facilities Program. Definitions specific to the National RCS Measurement Facilities Certification Program are annotated with [SMSG]. Notes have been included or added for clarification; if added for purposes of this standard, the [SMSG] annotation is applied.

1.6.2.1 Accreditation: Procedure by which an authoritative body gives formal recognition that a body or person is competent to carry out specific tasks.

1.6.2.1.1 Note: [SMSG] The National RCS Measurement Facilities Certification Program does not provide for accreditation.

1.6.2.2 Assurance of Conformity: Procedure resulting in a statement giving confidence that a product, process or service fulfills specified requirements.

1.6.2.2.1 Note: For a product, the statement may be in the form of a document, label or other equivalent means. It may also be printed in or applied on a communication, a catalogue, an invoice, a user instructions manual, etc. relating to the product.

1.6.2.3 Authority: Body that has legal powers and rights.

1.6.2.3.1 Note: An authority can be regional, national or local.

1.6.2.4 Body: Legal or administrative entity that has specific tasks and composition. 1.6.2.4.1 Note: [SMSG] For the National RCS Measurement Facilities Certification Program, the RCC SMSG is the administrative body that facilitates the exchange of information related to the program and the formation of review panels comprised of independent third-party reviewers.

1.6.2.5 Certificate of Conformity: Document issued under the rules of a certification system, indicating that adequate confidence is provided that a duly identified product, process or service is in conformity with a specific standard or other normative document.

1.6.2.6 Certification: Procedure by which a third party gives written assurance that a product, process or service conforms to specified requirements.

1.6.2.7 Certification Body: Body that conducts certification conformity assessments.

1.6.2.7.1 Note: A certification body may operate its own testing and inspection activities or oversee these activities carried out on its behalf of other bodies. [SMSG] In the National RCS Measurement Facilities Certification Program, the independent third-party reviewers are the certification body. The terms review panel, review committee, or reviewers may appear in related or Key Documents.

1.6.2.8 Certification Pool [SMSG]: Individuals whose qualifications have been assessed by the SMSG Radar committee as meeting or exceeding the minimum qualifications for certification body members. From the certification pool are drawn volunteers to perform certifications of specific facilities.

1.6.2.9 Certification System: System that has its own rules of procedure and management for carrying out certification of conformity.

1.6.2.9.1 Note: Certification systems may be operated at, for example, national, regional or international levels. The central body that conducts and administers a certification system may decentralize its activities and rights to certify conformity.

1.6.2.10 Certification Scheme: Certification system as related to specified products, processes or services to which the same particular standards and rules, and the same procedure, apply.

1.6.2.10.1 Note: The term certification program is used in some countries to cover the same concept as certification scheme.

1.6.2.11 Conformity: Fulfillment of specified requirements.

1.6.2.11.1 Note: The above definition is valid for the purposes of quality standards. The term conformity is defined differently in ISO/IEC Guide 2.

1.6.2.12 Conformity Assessment: Any activity concerned with determining directly or indirectly that relevant requirements are fulfilled.

1.6.2.12.1 Note: Examples of conformity-assessment activities are sampling, testing and inspection, evaluation, verification and assurance of conformity (supplier's declaration, certification), registration, accreditation and approval as well as their combinations.

1.6.2.13 Conformity, Evaluation for: Systematic examination of the extent to which a product, process or service fulfills specified requirements.

1.6.2.14 Conformity Surveillance: Evaluation for conformity to determine the continuing conformity with specified requirements.

1.6.2.15 Document Set (or Documentation Set) [SMSG]: The material employed by facilities to meet the documentary requirements of ANSI Z540, and by extension, this standard. See Range Book. The material may consist of one or more media, including digital, paper or other forms, so long as the totality of the documentation set is clearly identified and accessible by the intended users and certification body. Generally, classified data is not a part of a facility’s documentation set for purposes of meeting the requirements of this standard.

1.6.2.16 Evaluation Criteria [SMSG]: the published criteria employed by a certification body to perform an evaluation for conformity with this standard.

1.6.2.17 Grade: Category or rank given to entities having the same functional use but different requirements for quality.

1.6.2.17.1 Note: Grade reflects a planned or recognized difference in requirements for quality. The emphasis is on the functional use and cost relationship.

1.6.2.17.2 Note: [SMSG] In the National RCS Measurement Facilities Certification Program, for each evaluation criteria, the following Grades apply:

S = Fully Satisfactory - meets or exceeds standard (No review comments required)

N = Marginally Satisfactory but needs improvement (Marginally meets standard but may be improved using mandatory reviewer comments as guidance)

U = Unsatisfactory (Does not meet standard and should be re-accomplished using mandatory reviewer comments as guidance)

N/A = Not Applicable (Reviewer comments needed if the N/A reason is non-obvious)

1.6.2.18 Inter-range (or inter-facility) comparison [SMSG]: a process whereby a calibration artifact is measured at two or more facilities.

1.6.2.18.1 Note: Measurement results may optionally be maintained as confidential by the participating facilities. Calibration artifacts constructed to standardized dimensional tolerances and materials may be substituted for unique calibration items shipped from one facility to another.

1.6.2.19 License (For Certification): Document, issued under the rules of a certification system, by which a certification body grants to a person or body the right to use certificates or marks of conformity for its products, process or services in accordance with the rules of the relevant certification scheme.

1.6.2.19.1 Note: [SMSG] Licenses are not used in the National RCS Measurement Facilities Certification Program.

1.6.2.20 On-site Review [SMSG]: That portion of the certification process during which the certification body visits a subject facility for purposes of verifying the totality of the documentation set and assesses conformity with this standard as measured against published evaluation criteria.

1.6.2.21 Quality: Totality of characteristics of an entity that bear on its ability to satisfy stated and implied needs.

1.6.2.22 Range Book [SMSG]: A common reference to the documentation set employed by facilities to meet the documentary requirements of ANSI Z540, and by extension, this standard. See Documentation Set.

1.6.2.23 RCS community [SMSG]: All government, industry, and academic persons or entities engaged in the practice of radar signature measurements.

1.6.2.24 Recognition Arrangement: Agreement that is based on the acceptance by one party of results, presented by another party, from the implementation of one or more designated functional elements of a certification system.

1.6.2.24.1 Note: Typical examples of recognition arrangements are testing arrangements, inspection arrangements, and certification arrangements. Recognition arrangements may be established at, for example, national, regional, or international level.

1.6.2.24.2 Note: [SMSG] An example of a recognition agreement within the context of the National RCS Measurement Facilities Certification Program is the substitution of calibration artifacts constructed to standardized dimensional tolerances and materials for unique calibration items shipped from one facility to another.

1.6.2.25 Registration: Procedure by which a body indicates relevant characteristics of a product, process, or service, or particulars of a body or person, in an appropriate, publicly available list.

1.6.2.25.1 Note: [SMSG] Specific to the National RCS Measurement Facilities Certification Program, registration indicates an RCS measurement facility publicly states it has completed an internal review of its documentation set (i.e., Range Book) against published evaluation criteria; this step is required to initiate certification.

1.6.2.26 Review Panel (or Review Committee) [SMSG]: Term used in RCS community to refer to the certification body empanelled for an evaluation for conformity at a specific RCS facility. A Review Panel is comprised of three volunteers from the Certification Pool, plus one alternate, plus one or more representatives from the Radio-Frequency Technology Division (813) at the Boulder Laboratory of the National Institute of Standards and Technology (NIST).

1.6.2.27 Reviewer [SMSG]: Common reference to a review panel (or certification body) member who has been vetted and approved by the SMSG to participate in a certification.

1.6.2.28 Standard: Document, established by consensus and approved by a recognized body, that provides for common and repeated use, rules, guidelines or characteristics for activities or their results, aimed at the achievement of the optimum degree of order in a given context.

1.6.2.28.1 Note: Standards should be based on the consolidated results of science, technology, and experience, and aimed at the promotion of optimum community benefits.

1.6.2.29 Standards Body: Standardizing body recognized at national, regional or international level, that has as a principal function, by virtue of its statutes, the preparation, approval or adoption of standards that are made available to the public.

1.6.2.29.1 Note: A standards body may also have other principal functions. [SMSG] In addition to other principal functions, the RCC is a standards body.

1.6.2.30 Super-Facility [SMSG]: Where an entity has multiple measurement facilities, for purposes of certification, those facilities may be grouped such that common operational or organizational elements suggest greater efficiencies can be had by combining or merging certifications.

1.6.2.31 Suppliers Declaration: Procedure by which a supplier gives written assurance that a product, process or service conforms to specified requirements.

1.6.2.31.1 Note: To avoid confusion, the expression self-certification should not be used in conjunction with Suppliers Declaration.

1.6.2.32 Test: Technical operation that consists of the determination of one or more characteristics of a given product, process or service according to a specified procedure.

1.6.2.33 Testing: Action of carrying out one or more tests.

1.6.2.34 Test Method: Specified technical procedure for performing a test. 1.6.2.35 Test Report: Document that presents test results and other information relevant to a test.

1.6.2.35.1 Note: [SMSG] In the National RCS Measurement Facilities Certification Program, a formal Report of Measurement is a crucial Test Report used for communicating test results, which will be evaluated by the certification body.

1.6.2.36 Third Party: Person or body that is recognized as being independent of the parties involved, as concerns the issue in question.

1.6.2.36.1 Note [SMSG]: the certification body of the National RCS Measurement Facilities Certification Program is comprised of third party persons. Employees or others paid by a government, industry, or academic entity with multiple facilities engaged in the certification process may serve in a third party capacity so long as the individual’s affiliation is made known and the chain of authority for the individual is different from the instant facility being certified.

CHAPTER 2 ESTABLISHING THE PROCESS

2.1 Overview 2.1.1 The concept of establishing and meeting a standard for RCS measurement facilities is based on documenting the operations and capabilities of such facilities. Meeting this standard does not imply or certify that a facility can and does make measurements to some finite degree of accuracy, certainty, or signal level. The degrees of accuracy, certainty, signal levels, or other metrics are established by the facility based on the conditions at the facility and the abilities of the personnel engaged in RCS measurements given the hardware and software systems at their disposal and the physical constraints of their measurement environment. The processes, procedures, instrumentation, environment, and the fidelity and rigor to which they are managed determine the level of success achieved by the facility. This standard addresses documentation.

2.1.2 The fundamental requirement of this standard is one of documentation. This is in keeping with the concepts of the ISO and its 9000 series of quality standards. By documenting what should be and what is, and comparing the two on periodic bases, incremental improvements in quality will evolve as the differences approach zero. By raising the bar over time, quality continually improves.

2.1.3 Therefore, a methodical approach to documenting the operations and conditions at an RCS measurement facility forms the basis for conformity with this standard. The approach should be tailored to fit the practical realities faced by a facility seeking to adopt this standard. If a practice does not make sense for the facility, it should not be adopted. Defensible logic for not adopting a recommended practice may be required by customers, however.

2.1.4 Customers, and customer focus, are important elements of this standard, as it is generally the customer who is demanding higher quality RCS data products.

2.1.5 Other important elements of this standard include uncertainty analyses, calibrations, calibration artifacts, inter-range comparisons, and reports of measurement. The activities associated with these elements support the documentation process. The most challenging is the conduct of the uncertainty analysis (or analyses if multiple measurement systems or configurations are utilized).

2.2 Roles & Responsibilities 2.2.1 There are four distinct roles for the participants in the National RCS Certification Program: a standards body, an administrative body, a certification body, and facilities participating in the program.

2.2.1.1 The standards body is the RCC, with the SMSG proposing and maintaining this standard (and others). It is the mission of the RCC to ensure technical standards are established and disseminated.

2.2.1.2 The administrative body of this standard, and the body that facilitates the process of certification, is the SMSG, with the Radar Committee of the SMSG performing many of the

duties. It is the responsibility of the SMSG Chairperson to review and approve the activities of the Radar Committee. Duties of the SMSG include:

2.2.1.2.1 Soliciting volunteers to participate as the certification body in the form of review panel members;

2.2.1.2.2 Reviewing (through the Radar Committee) nominees for the certification body;

2.2.1.2.3 Approving (or disapproving) through the Radar Committee individuals who meet published minimum qualifications (see 2.2.1.3);

2.2.1.2.4 Maintaining a published list of approved certification body members including particulars related to employment affiliation, contact information, date approved, facilities reviewed and when reviewed;

2.2.1.2.5 Contacting certification body members when stand-up of a review panel is required to ascertain availability and potential conflicts of interest;

2.2.1.2.6 Providing to the SMSG Chairperson and subject facilities the names and affiliations of proposed review panel members to vet potential conflicts of interest;

2.2.1.2.7 Drafting formal letters of invitation to vetted review panel members for the purpose of standing up a review panel for a specific registered facility, providing contact information to all parties involved in the instant certification;

2.2.1.2.8 Notifying the Radio-Frequency Technology Division (813) at the NIST Boulder Laboratory of pending certifications, providing contact information to all parties involved in the instant certification;

2.2.1.2.9 Coordinating correspondence by and between all parties to the certification process, including the confidential delivery of certification documentation sets from facilities to review panel members and NIST;

2.2.1.2.10 Responding to inquires by providing documentation or sources of information relating to certification to the RCS community;

2.2.1.2.11 Coordinating dissemination of information about certification to the RCS community through symposia, conferences, teleconferences, web sites, or other fora or media;

2.2.1.2.12 Preparing and delivering formal documentation, including final certification reports, letters of congratulation, certificates, letters of thanks, and other formal correspondence to parties of specific facility certifications.

2.2.1.3 The certification body consists of volunteers solicited by the SMSG Radar Committee to participate in the certification process as the body that reviews and evaluates the documentation sets of participating facilities. Volunteers are vetted by the SMSG to assess qualifications against published minimum qualifications and potential conflicts of interest. Minimum qualifications for certification body volunteers are (adopted 15 June 1998):

2.2.1.3.1 Education: Bachelor of Science degree in Math, Physics, Electrical Engineering, Computer Science, Systems Engineering, or closely aligned degree program (emphasis in electromagnetics or scientific data processing desired, though not required).

2.2.1.3.2 Work Experience: at least 10 years direct experience in RCS-related discipline (i.e., measurements, design, and analysis) with at least 5 years of recent RCS measurement

experience. Specific experience in RCS range engineering may include measurements planning and execution, model design/build, calibration and characterization of RCS measurement systems and auxiliary systems.

2.2.1.3.3 Characteristics: Candidates are expected to be extremely familiar with RCS measurements, sources of error and uncertainty, various range characterization methods, mathematics related to RCS data processing, mounting techniques, calibration methods, and general customer requirements. Candidates are also expected to be familiar with (or be willing to become familiar with) the RCS range certification standard, RCS range procedures, and the ANSI-Z540 standard.

2.2.1.3.4 Corporate Commitment: participation requires a written statement from candidate's management to sponsor up to three trips in one year, or four trips in two years in order to forward the certification process2. Trips will generally be 2-3 days and will be in the contiguous United States3. A candidate’s management authority must also commit to provide approximately two man-months worth of labor to support certification activities for one year (minimum) or three man-months across two years, or the candidate must be willing to donate this time after hours on a time-available basis within the timeline of the base review schedules.

2.2.1.3.5 Personal Commitment: candidates must express in writing a desire to assist candidate ranges in achieving certification and express commitment to forward the process, to working productively with other board members, and to provide timely response and assistance to certification on related matters. Upon appointment to the review board, members will be required to sign and honor non-disclosure agreements.

2.2.1.3.6 Other Requirements: must be nominated; may nominate self. Nominations should address each of the topical areas above. Nominees must be approved by a consensus of four existing members once the board is established. The SMSG Radar Committee Chairperson will endorse the initial board candidates and forward them to the SMSG Chair, who will make the final selections.

2.2.1.4 The procedure for establishing the certification body follows:

2.2.1.4.1 Candidates are nominated (or self-nominated) by submitting resumes and individual commitment letters.

2.2.1.4.2 Corporate Commitment letters may follow approval of the nominees by the Radar Committee. This event-driven sequence permits entities with more complicated approval requirements some flexibility and certainty that a specific nominated individual is acceptable to the administrative body prior to seeking internal signatures certifying corporate commitment.

2 Under normal circumstances, each SMSG-approved reviewer will participate in two separate facility reviews. In the event a certification does not follow the nominal process or timeline and requires multiple trips to the same facility by the review panel, the SMSG Radar Committee may recommend to the SMSG Chair, or the SMSG Chairperson may independently make a determination, that a reviewer has met the spirit of the commitment. 3 Facilities outside the United States may participate through a Memorandum of Understanding executed between the facility and the SMSG. Contact the SMSG Chairperson for further information.

2.2.1.4.3 Initially, nominees to the certification body pool are to be reviewed and approved (or disapproved) individually by simple majority vote by the SMSG Radar Committee indicating minimum qualifications have been met. Once four (or more) individuals have been approved, those four may collectively approve nominees.

2.2.1.4.4 Information on the certification body pool will be maintained by the Radar Committee Secretary. The certification body pool and the status of each nominee and approved certification body member will be made available to facilities and the RCS community.

2.2.1.4.5 Facilities notify the RCC SMSG Chairperson that they have completed or will complete the registration process within thirty days.

2.2.1.4.6 The SMSG Chairperson tasks the Radar Committee Secretary to query the pool for volunteers who (a) will be available in the timeframe of the subject facility’s certification review and (b) have no conflicts of interest. Three primary volunteers and one alternate are identified and their names and corporate affiliation are provided to the subject facility so that the facility may approve or veto the individuals selected for that facility’s certification body.

2.2.1.4.7 If a facility vetoes one or more candidates, the process is repeated until three primary reviewers and one alternate reviewer are acceptable to the facility to be certified.

2.2.1.4.8 Once three primary reviewers and one alternate reviewer are approved by the subject facility, contact information is forwarded to the SMSG Chairperson who formally invites them to participate in the subject certification.

2.2.1.4.9 Observers from the certification pool may participate in the on-site review portion of the certification process, provided the subject facility approves of the observer. Participation by observers is encouraged to enhance the experience level of the certification pool.

2.2.1.4.10 The Radio-Frequency Technology Division (813) at the NIST Boulder Laboratory participates as a formal advisor to the SMSG and to Review Panels on specific technical topics (uncertainty analysis, measurement and calibration procedures, and inter-range comparisons).

2.2.1.4.11 Formal letters of invitation are prepared and sent by the SMSG to all participants advising them of the particulars of the subject certification and contact information of all parties.

2.2.1.4.12 The three volunteers confer via email and/or telephone to elect a chairperson and initiate activities.

2.2.1.5 Review panel members, once notified by the SMSG Secretary of their selection for a specific facility certification, must promptly confer with one another to elect a chairperson. All review panel members are responsible for maintaining strict confidentiality of all documents and information relating to the facility undergoing certification. Transmittal of documents via email or other electronic means must employ encryption. Review panel members are responsible for the detailed review, comment, documentation, and rating of a facility’s documentation set in accordance with published evaluation criteria. Review panel members are responsible to the review panel chairperson for developing and meeting a detailed timeline of activities and milestones associated with the conduct of the subject certification review. Review panel members are responsible for proactively participating in all aspects of the certification review, including the on-site review.

2.2.1.6 The review panel chairperson is responsible for coordinating all activities of the review panel and serves as the single point of contact with the facility, NIST, and SMSG,

promulgating schedules and other information affecting parties outside the review panel’s deliberations. Review panel chairpersons are responsible for on-going dialog with the designated facility point of contact. Review panel chairpersons are responsible for providing a mid-term report to the designated facility point of contact, said mid-term report being comprised of preliminary findings and guidance in addressing deficiencies (see 2.3.6.1). Review panel chairpersons are responsible for compiling information for an on-site review, scheduling a 2-to-3-day on-site review, leading the on-site review as prescribed in published evaluation criteria, and conducting an out-briefing for the facility, said out-briefing to include a grading report with comments for unsatisfactory or needs-improvement scores, if any. Review panel chairpersons are responsible for developing a final report in draft form and submitting it to the SMSG Secretary no later than 30 days following the conclusion of the on-site review (see 2.3.11.1).

2.2.1.7 Facilities wishing to participate in certification are responsible for nominating to the SMSG individuals to act as review panel members as part of the certification body. Said facilities whose nominees are approved by the SMSG must commit to supporting their nominees as described in 2.2.1.4.2.

2.2.1.8 Facilities electing to participate in the certification process are responsible for proactively seeking information about the certification process, understanding the process, developing and maintaining their respective documentation sets, including the activities necessary to develop said documentation like uncertainty analyses and inter-facility comparisons of calibration artifacts. Facilities electing to participate in the certification process are responsible for notifying the SMSG secretary of their registration status (no sooner than 30 days prior to when registration is to be completed). Facilities are responsible for designating to the SMSG primary and secondary certification points of contact. Facilities are responsible for reviewing the names and affiliations of proposed review panel members to assess and report to the SMSG potential conflicts of interest, if any. Facilities are responsible for on-going dialog with the designated review panel point of contact. Facilities are responsible for preparing an On-Site Closure Plan in response to the review panel’s Mid-Term Certification Report. Facilities are responsible for hosting and supporting on-site reviews held by their respective certification review panels. Facilities are responsible for participating with the review panel in preparing a Remedial Plan if a satisfactory grade is not awarded by the review panel during the on-site review. Facilities are responsible for notifying the SMSG of any irregularities or concerns in the process or conduct of their certification, or if the integrity of the process is questioned in any way.

2.2.1.9 The role of the RCC Secretariat is to review, edit, publish, and disseminate this standard in keeping with the RCC Operations and Policy Document.

2.3 Timeline and Reports 2.3.1 The nominal timeline for the certification process is six months. Prior to the certification phase, a facility prepares its documentation set and conducts its internal review (registration). The timeline of the preparatory phase can vary considerably depending upon the complexity of the facility and the level of existing processes and systems and the extent to which those processes and systems are documented. In particular, efforts surrounding several key processes can be very time consuming. Among the key processes requiring larger expenditures

of effort are the uncertainty analysis, calibration processes and procedures, and inter-facility (inter-range) comparisons.

2.3.2 In this section, the timeline is annotated as R+xx, where xx is the nominal number of days since registration; e.g., R+0 is the Registration Event, R+30 as the Commencement of the Review Panel activities, and so on.

2.3.3 Registration is the event that triggers the formal timeline of the certification process (R+0).

2.3.4 Upon receipt by the SMSG Chairperson of notification by a facility that registration will be complete within 30 days, or that registration is actually complete, the SMSG Chairperson will task the SMSG Secretary to initiate the stand-up of a review panel (see 2.2.1.2.4 through 2.2.1.2.8 above). The complete stand-up of review panels must be accomplished as quickly as practicable, but in no case should the stand-up take longer than 30 days (R+30).

2.3.5 Upon notification by the SMSG of the name and contact information for the review panel members, the facility should promptly forward to the review panel its documentation set (R+35). The facility should send to NIST only those portions of the facility’s documentation set that addresses uncertainty analyses, measurement and calibration procedures, and inter-range comparisons, noting that NIST prefers a paper copy (in lieu of a digital version).

2.3.6 The review panel, under the guiding hand of the review panel chair, has 60 days to complete its preliminary review and Mid-Term Certification Report of the facility’s documentation set (R+90). During this period, the review panel should conduct dialog with the facility through the review panel chairperson on any matters requiring clarification. Advance notice of any discrepancies or potential discrepancies, coupled with guidance from the review panel can ensure the facility has time to address issues as they are discovered, thereby facilitating the timely completion of the certification.

2.3.6.1 The Mid-Term Certification Report shall contain, as a minimum:

2.3.6.1.1 Degree of completion of the review in terms of the portions of the document set reviewed to-date;

2.3.6.1.2 The preliminary grades for the portions of the document set reviewed to-date; 2.3.6.1.3 Comments to all portions of the document set reviewed to-date that have received preliminary grades of U (Unsatisfactory) or N (Needs Improvement);

2.3.6.1.4 Guidance for improving or enhancing any portions of the document set that may enhance the facility’s chances of success, and to the maximum extent practicable, guidance for raising the overall grade of the facility’s documentation set.

2.3.7 Upon receipt of the Mid-Term Certification Report, the facility should promptly review the findings, develop an On-Site Closure Plan to address issues reported in the Mid-Term Certification Report, and deliver the On-Site Closure Plan within fourteen days to the review panel chairperson (R+105).

2.3.8 Following receipt of the On-Site Closure Plan, the review panel shall continue to address issues not already addressed, and confer with NIST on its findings related to uncertainty analyses, measurement and calibration procedures, and inter-range comparisons, and assist the facility in addressing identified issues prior to the on-site review. The on-site review should be

coordinated and scheduled when the facility has assessed it will be ready to successfully engage the review panel and meet the objectives identified in the published evaluation criteria (R+150).

2.3.9 At on-site review and facility out-briefing, if a satisfactory grade is not awarded by the review panel, the review panel and facility representatives shall devise a Remedial Plan to address those issues preventing award of a satisfactory grade. That Remedial Plan shall include a timeline sufficient to minimize risk and support successful completion of the activities necessary to achieve a satisfactory grade. The SMSG Chairperson shall be promptly notified of the on-site review findings, grade, and Remedial Plan, if any (R+150).

2.3.10 If a satisfactory grade is awarded at the conclusion of the on-site review, the review panel chairperson shall prepare a draft Final Certification Report and deliver within 30 days after the on-site review a digital encrypted copy to the SMSG Secretary for final review and formatting (R+180).

2.3.11 Table 2.1 provides a summary of the nominal certification timeline:

Table 2.1: Nominal Certification Timeline Summary MILESTONE REFERENCE

EVENT

R+0 Facility Registration complete; Facility notifies SMSG R+30 SMSG Secretary stands up Review Panel and disseminates points of contact

information R+35 Facility disseminates documentation set to review panel and NIST R+90 Review panel reviews documentation set, submits Mid-Term Certification

Report to facility R+105 Facility submits to review panel On-Site Closure Plan R+150 On-site review. R+180 Review panel chairperson submits draft Final Certification Report to SMSG

Secretary R+200 SMSG Secretary submits Final Certification Report to SMSG Chairperson

for signature and forwarding to the Facility

2.3.11.1 The draft Final Certification Report shall contain, at a minimum:

2.3.11.1.1 Proprietary / Confidentiality notice on each page (e.g., “In keeping with the requirements of ANSI Z540, the contents of this report are to remain privileged between the certification body and the facility reviewed, unless the facility formally releases the results. For more information, or in case of inadvertent release, contact [name and contact information of review panel chair].”

2.3.11.2 A title page with:

2.3.11.2.1 Title (Final Certification Report).

2.3.11.2.2 Subtitle (ANSI-Z540 Conformity Assessment of [facility name]).

2.3.11.2.3 Date of the report. 2.3.11.2.4 Names, affiliations, and signatures of the certifying body review panel members (with chairperson identified).

2.3.11.3 An Executive Summary with: 2.3.11.3.1 The names, affiliations, and titles/positions of the certifying body review panel members.

2.3.11.3.2 The final overall grade. 2.3.11.3.3 A table summarizing the prioritized evaluation criteria compared to the grades awarded, similar to the example shown in Table 2.2.

Table 2.2: Sample ANSI Z540 Final Assessment Scores

PRIORITY EVALUATION CRITERIA ASSESSED GRADES Priority 1 S ≥ 85% S = 90 % N ≤ 15% N = 10 % U = 0 % U = 0%

Priority 2 S ≥ 75% S = 90 % N ≤ Balance N = 5 % U ≤ 10% U = 5%

Priority 3 S ≥ 75% S = 90 % N ≤ Balance N = 7 % U ≤ 10% U = 3%

2.3.11.4 A text or graphical timeline showing the start and stop dates of the major activities or miles milestones of the certification.

2.3.11.5 The evaluation criteria with grades for each criterion, with comments as required for grades of N (Needs Improvement) and U (Unsatisfactory).

2.3.11.6 Optionally, the Final Certification Report may contain:

2.3.11.6.1 Lessons-learned or comments about the certification process or the facility. 2.3.11.6.2 Identification of individuals who demonstrated exceptional support for the registration or certification efforts at the facility.

2.3.11.6.3 Resumes of the certification body review panel members.

2.3.12 The SMSG Secretary will review the draft Final Certification Report for the purposes of ensuring compliance with this standard and for aesthetics, spelling, grammar, and obvious errors and/or omissions. Any anomalies or discrepancies will be promptly reported to the review panel chairperson for resolution.

2.3.13 The SMSG Secretary will incorporate any corrections identified in 2.3.2 and prepare the final version of the Final Certification Report for publication.

2.3.14 The SMSG Secretary will prepare for signature the formal congratulatory letter and certificate, formal report cover page, and formal letters of thanks to the reviewers. Those

documents requiring signatures will be transmitted in triplicate via round-robin mail or courier service to collect the original signatures of the participants. The final package will be assembled by the SMSG Secretary and transmitted formally through the SMSG chairperson to the facility (R+200).

2.3.15 If a Remedial Plan is required, facility personnel and the review panel shall actively pursue the completion of all activities necessary to achieve a satisfactory grade following the agreed-to timeline. The review panel chairperson will assess whether or not an additional on-site review is required, or if the transmittal of documentation will be sufficient to assess the outcomes of the Remedial Plan activities. If an additional on-site review is required by the review panel chair, the review panel chairperson will conduct the review in person, with other panel members participating via teleconference. Upon achievement of a satisfactory grade, the steps described at 2.3.10 and 2.3.11 shall be carried out promptly.

CHAPTER 3

MAINTAINING THE PROCESS

3.1 Overview 3.1.1 Having established the process of certifying RCS measurement facilities that have achieved substantial compliance with this Z540-based documentation standard, maintaining the process becomes the focus of efforts over the long-term. The critical elements of maintaining the certification process relate to maintaining the pool of volunteers to continue certification, maintaining continuity and consistency in how the certification process is applied, and establishing a process of re-certifying facilities.

3.2 Maintaining the Certification Body Pool of Reviewers 3.2.1 Crucial to maintaining the certification process is the depth of the pool of individuals who serve as certifying body review panel members. To that end, the SMSG Secretary will maintain a publicly available list of volunteers and their respective status with respect to numbers of reviews completed. The list of Certification Body Pool Reviewers will be updated by the SMSG Secretary no less often than quarterly.

3.2.2 The SMSG Secretary will maintain a publicly available list of facilities to be reviewed, their probable certification schedules.

3.2.3 The SMSG Secretary will provide to the SMSG Radar Committee projections for the numbers of individuals who will be needed to satisfy all certification reviews.

3.2.4 The SMSG Radar Committee Chairperson will assess the available reviewers against the projected facilities to be reviewed and their probable certification schedules. If that assessment indicates too few reviewers might be available, then:

3.2.4.1 The SMSG Radar Committee Chairperson will advise the full committee and the SMSG Chairperson of the situation.

3.2.4.2 The SMSG Radar Committee Chairperson and committee members will solicit from the RCS measurement community nominations for additional review panel member candidates, and process those nominations as described at 2.2.1.3 of this standard.

3.2.4.2.1 In the event the solicitation for nominations results in too few nominees, or the nomination procedure (at 2.2.1.3) results in too few approved candidates, then extraordinary action will be required.

3.2.4.2.1.1 The Radar Committee Chairperson will seek volunteers from the pool of approved candidates who have completed the terms of their obligatory reviews (see 2.2.1.3.4, 2.2.1.3.5, and associated footnotes).

3.2.4.21.2 In the event insufficient volunteers are identified, the Radar Committee Chairperson will petition the SMSG Chairperson to appoint review panel members from RCC Member ranges.

3.2.4.2.1.3 If funding becomes an issue, the SMSG Chairperson will develop a cost estimate and petition the RCC Secretariat for funding support.

3.2.4.3 The list of Certifications to be Completed will be updated by the SMSG Secretary no less often than quarterly.

3.3 Maintaining Continuity and Consistency 3.3.1 To be effective across the entire RCS community, the certification process must maintain a high degree of continuity and consistency. This standard, and adherence thereto by participating RCC Member Ranges, RCC Associates, and other members of the RCS measurement community is the primary method for achieving continuity and consistency.

3.3.2 Participants are encouraged adhere to this standard in its entirety, not selected portions.

3.3.2.1 In the event any party to RCS measurement facility certification determines the standard contains errors or requirements that are not practicable, said party may petition the SMSG Radar Committee Chairperson to consider modifications to the standard.

3.3.2.2 In the case of a petition to modify the standard, the SMSG Radar Committee will review the petition, conduct discussions with the petitioner to develop a full understanding of the basis of the petition and the conflict with this standard. The SMSG Radar Committee will conduct a formal vote to approve or deny the petition for a waiver. The results of the vote will be formally transmitted to the SMSG Chair, who will formally transmit the results to the petitioner.

3.3.2.3 In the case of an approved change to this standard, the SMSG Radar Committee Secretary will, within 60 days of the vote, draft the change language, circulate the change language for review, edit, and approval by the Radar Committee. Upon receipt of consensus approval of the change language, this standard will be revised through the existing RCC Secretariat publication process.

3.3.3 Facilities shall adhere to guidance provided by ANSI Z5404 to the maximum extent practicable, though are permitted discretion in determining which elements are congruent with their respective operations. If deviations from ANSI Z540 are adopted by a facility, the facility should be prepared to explain to review panel members the reasoning behind the deviation, the potential impact, and mitigating actions taken or planned (if any).

3.4 Follow-up Assessments and Re-Certification 3.4.1 The process of re-certifying facilities is being developed as this standard is being finalized. Some of the issues to be addressed include the time between certification and re-certification, or other triggering events that might indicate a need for re-certification. The criteria for re-certification are also under discussion (same criteria, a subset of the certification criteria, or a hybrid).

4 Depending on the time frame, ANSI Z540 may be supplanted by ISO 17025 or the ANSI derivation thereof.

3.4.2 This standard will be revised no later than December 2003 to define the processes relating to re-certification of RCS measurement facilities.

APPENDIX A

BIBIOGRAPHY

The references that follow provide additional insight into the concepts related to developing and implementing successful processes and practices associated with measurement uncertainties, calibrations, and Radar Cross Section measurements.

A.1 Analysis for Engineers. 2nd ed. New York: John Wiley & Sons, 1999, ISBN 0-471-12146-0

A.2 Andersh, Dennis, Brian M. Kent, Dean L. Mensa, J. P. Skinner, and Ron C. Wittmann. “Normalization and Interpretation of Radar Images,” submitted to IEEE Transactions, Antennas Propagation, 1996.

ANSI/ASME US Standard Z540-2, NCSL, Boulder CO, 1998

A.3 ASME PTC 19.1, “Measurement Uncertainty, American Society of Mechanical Engineers,” NY, 1985

A.4 Balanis, Constantine A., Advanced Engineering Electromagnetics, New York: John Wiley & Sons, 1989. 62.

A.5 Black, D.N., and Ron C. Wittmann, “Antenna/RCS Range Evaluation Using A Spherical Synthetic-Aperture Radar,” Proceedings of 18th Antenna Measurement Techniques Association Symposium, Seattle. 30 Sept.-3 Oct. 1996. 406-410.

A.6 Bleszynski, E., M. Bleszynski, and T. Jaroszewicz. “AIM: Adaptive Integral Method for Solving Large-Scale Electromagnetic Scattering and Radiation Problems,” Radio Science, 31 (1996): 1225-1251.

A.7 Borchardt, B., W.T. Estler, and S.D. Phillips. The Estimation of Measurement Uncertainty of Small Circular Features Measured by CMMs: NISTR 5698., Gaithersburg, MD: NIST, 1995.

A.8 Canino, L.F., et al. “Numerical Solution of the Helmholtz Equation in 2D and 3D Using a High-Order Nyström Discretization,” Journal of Computational Physics, 146 (1998): 627-663.

A.9 Catalogue of Intrinsic and Derived Standards, NCSL International Business Office, 1800 30th Street, Suite 305B, Boulder, CO 80301.

A.10 CEN/CENELEC, EN45001, “General Criteria For the Operation of Testing Laboratories,” Brussels, 1989.

A.11 Chizever, H. M., Brian M. Kent, and R.J. Soerens. “On Reducing Primary Calibration Errors in RCS Measurements.” Proceedings of 18th Antenna Measurement Techniques Association, Seattle. 30 Sept.-3 Oct. 1996. 383-388.

A.12 Coleman, Hugh W., and W. Glenn Steele. Experimentation and Uncertainty Analysis for Engineers. 2nd ed. London: John Wiley & Sons, 1999.

A.13 Dietrich, C. F. Uncertainty, Calibration and Probability: The Statistics of Scientific and Industrial Measurements. Bristol: Adam Hilger Ltd., 2nd ed. May 1991. ISBN: 0750300604.

A.14 Doiron, Ted, and John Stoup, “Uncertainty and Dimensional Calibrations.” Journal of Research of the National Institute of Standards and Technology 102(6). Gaithersburg, MD: NIST. 1997. 647-676. http://nvl.nist.gov/pub/nistpubs/jres/102/6/j26doi.pdf

A.15 Doiron, Ted, Keith R. Eberhardt, W. T. Estler, M. S. Levenson, and Steven D. Phillips. “A Careful Consideration of the Calibration Concept.” Journal of Research of the National Institute of Standards and Technology 106(2). Gaithersburg, MD: NIST. 2001. 371-379. http://nvl.nist.gov/PDF/jres/106/2/j62phi.pdf

A.16 Donaldson, Robert, “Error Budget,” Chapter 3 of Large Optics Diamond Turning Machines, Lawrence Livermore National Laboratories, 1978.

A.17 EAL-4/02, “Expression of the Uncertainty of Measurement in Calibration,” EAL, 1999

A.18 Eberhardt, Kieth R., Steven D. Phillips, and Barry Parry. “Guidelines for Expressing the Uncertainty of Measurement Results Containing Uncorrected Bias,” Journal of Research of the National Institute of Standards and Technology 102(5). Gaithersburg, MD: NIST. 1997. 647. http://nvl.nist.gov/pub/nistpubs/jres/102/5/j25phi.pdf

A.19 Estler, W.T. Accuracy Analysis of the Space Shuttle Solid Rocket Motor Profile Measuring Device: NISTR 89-4171. Gaithersburg, MD: NIST, 1989

A.20 Estler, W.T. and S.D. Phillips. “Calculation of Measurement Uncertainty using Prior Information,” Draft, Gaithersburg, MD: NIST, 1997

A.21 Fink, Donald G., and H. Wayne Beaty. Standard Handbook for Electrical Engineers. 11th Ed. New York: McGraw Hill, 1978, 4-6.

A.22 Fischer, Brian E. (Capt.), Brian M. Kent, Byron M. Welsh, William D. Wood (Maj.), and Thomas M. Fitzgerald. “Moment Method inter-Code Comparisons and Angular Sensitivity Studies For NIST Calibration (Squat) Cylinders,” Proceedings of 20th Antenna Measurement Techniques Association Symposium, ___ 1998, Montreal. 414-420

A.23 Francis, Mike H., Lorant A. Muth, and R. L. Lewis, Ron C. Wittmann. Proposed Uncertainty Analysis For RCS Measurements. National Institute of Standards and Technology Interagency Report, NISTIR 5019, Jan. 1994.

A.24 Francis, Mike H., Lorant A. Muth, R. L. Lewis, and Ron C. Wittmann. “Proposed Analysis of RCS Uncertainty,” Proceedings of 16th Antenna Measurement Techniques Association Symposium, 3-7 Oct. 1994. Long Beach, CA. 1994. 51-57.

A.25 Hawkins, C.F., and R.H. Williams. “The Economics of Guard Band Placement,” Proceedings of the 24th IEEE International Test Conference, Baltimore. 1993

A.26 ILAC, G8, “Guidelines on Assessment and Reporting of Compliance with Specification,” International Laboratory Accreditation Committee, 1996

A.27 Instone, Ian. “Calculating the Uncertainty of a Single Measurement.” Proceedings of IEE Colloquium on Uncertainties in Electrical Measurements, London. 11 May 1993. http://metrologyforum.tm.agilent.com/one_meas.shtml

A.28 ISO TC-213 Working Group 3, “Uncertainties of Dimensional Measurements Made at Temperatures Other Than 20 °C.” draft prepared by TSO TC 213 WG3. Geneva: ISO, 1997

A.29 ISO. Guide to the Expression of Uncertainty in Measurement. BIPM, IEC, IFCC, ISO, IUPAC, OIML. ISBN 92-67-10188-9. BSI Equivalent: “Vocabulary for Metrology, Part 3. Guide to the Expression of Uncertainty in Measurement.” BSI PD 6461: 1995.

A.30 ISO/IEC 17025, “General Requirements for the Technical Competence of Testing Laboratories.” Geneva: ISO, 1999.

A.31 ISO/IEC Guide 43, “Development and Operation of Laboratory Proficiency Testing.” Geneva: ISO, 1984.

A.32 ISO/IEC, IS0 10012-1, “Quality Assurance Requirements For Measuring Equipment - Part 1: Metrological Confirmation System For Measuring Equipment TC 176.” Geneva: ISO, 1992

A.33 ISO/TC 213, ISO/DIS 14253-1, Part 1: “Decision Rules for Providing Conformance or Nonconformance with Specification.” Geneva: ISO, 1998

A.34 ISO/TC 213, ISO/DIS 14253-2, Part 2: “Guide To the Estimation of Uncertainty in GPS Measurement, in Calibration of Measuring Equipment and in Product Verification.” Geneva: ISO, 1998.

A.35 Johnson, Richard C., and Henry Jasik. Antenna Engineering Handbook (2nd Edition). New York: McGraw Hill, 1989.

A.36 Kent, B. M., “Improvements in Static Radar Cross Section Calculations for RCS Calibration Artifacts.” Electromagnetic Code Consortium (EMCC) Conference, 9-11 May 2000, St Louis.

A.37 Kent, B. M., “Improvements in Static Radar Cross Section Calibration Processes and Artifacts – Initial Measurement Results and Validation Through Inter-range Comparisons.” Proceedings of 21st Antenna Measurement Techniques Association, 4-8 Oct. 1999, Monterey, CA. 97-102.

A.38 Kent, Brian M., and Byron M. Welsh. “An RCS Uncertainty Analysis and Calibration Certificate for AFRL Calibration Cylinders.” Proceedings of 22nd Antenna Measurement Techniques Association, 16-20 Oct. 2000, Philadelphia.

A.39 Kent, Brian M., and W.D. Wood, Jr. “The Squat Cylinder and Bicone Primary Static RCS Range Calibration Standards.” Proceedings of 19th Antenna Measurement Techniques Association Symposium, 17-20 Nov. 1997, Boston. 319.

A.40 Kent, Brian M., Lorant A. Muth, and Ron C. Wittmann. “Measurement Assurance and Certification of Radar Cross Section Measurements.” Proceedings of 1997 National Conference of Standards Laboratories Workshop and Symposium, 28-31 July 1997. Atlanta. 555-566.

A.41 Kent, Brian M., Lorant A. Muth, James D. Tuttle, and Ron C. Wittmann. “Radar cross section range characterization,” Proceedings of 18th Antenna Measurement Techniques Association Symposium, 30 Sept.-3 Oct. 1996, Seattle, WA. 267-272.

A.42 Kuyatt, Chris E., and Barry N. Taylor. Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results: NIST Technical Note 1297. Gaithersburg, MD: NIST, 1994.

A.43 Lewis, R. L., Lorant A. Muth, and Ron C. Wittmann. “Polarimetric Calibration of Reciprocal-Antenna Radars,” Proceedings of 17th Antenna Measurement Techniques Association Symposium, 13-17 Nov. 1995. Williamsburg, VA. 3-8.

A.44 Lewis, R. L., Lorant A. Muth, and Ron C. Wittmann. Polarimetric Calibration of Reciprocal-Antenna Radars, National Institute of Standards and Technology Interagency Report, NISTIR 5033. March 1995.

A.45 Muth, Lorant A., and Ron C. Wittmann. “Calibration of Polarimetric Radar Systems,” IEEE Antennas and Propagation Society International. Symposium, 13-18 July 1997. Montreal. 830-833.

A.46 Muth, Lorant A., William Parnell, and Ron C. Wittmann. “Polarimetric Calibration of Nonreciprocal Radar Systems,” Proceedings of 18th Antenna Measurement Techniques Association Symposium, 30 Sept.-3 Oct. 1996, Seattle, WA. 389-393.

A.47 NAMAS M3003, “The Expression of Uncertainty and Confidence in Measurement for Calibrations,” NAMAS, England, 1995

A.48 NAMAS NIS 80, “Guide to the Expression of Uncertainties in Testing,” NAMAS, England, 1994

A.49 NCSL RP12, “Determining and Reporting Measurement Uncertainties, Recommended Practice” NCSL, Boulder CO, 1995

A.50 NF E 02-204, “Verification Des Tolerances Des Produits, Declaration De Conformite,” AFNOR, Paris, France, 1993

A.51 Putnam, J.M, and L. N. Medgyeshi-Mitschang. Combined Field Integral Equation Formulation for Axially Inhomogeneous Bodies of Revolution (Combined Field Formulation of CICERO): Report MDCQA003. St. Louis: McDonnell Douglas Research Laboratory, , Dec. 1987.

A.52 Putnam, J.M, and Medgyeshi-Mitschang, L.N., “Combined Field Integral Equation Formulation for inhomogeneous Two and Three Dimensional Bodies: the Junction Problem," IEEE Antenna and Propagation Transactions 39(5). May 1991.

A.53 Quality assurance requirements for measuring equipment, Part 1. Metrological confirmation system for measuring equipment, ISO 10012-1:1992.

A.54 Salsbury, James G., “Uncertainty of the Coordinate Measurement Process Due to Unwanted Asperities,” American Society for Precision Engineering, Volume 14, 1996

A.55 SINGAS Technical Guide 1, “Guidelines on the Evaluation and Expression of the Measurement Uncertainty,” SINGAS, Singapore July 1995

A.56 Swyt, Dennis A. “Uncertainties in Dimensional Measurements Made at Nonstandard Temperatures.” Journal of Research of the National Institute of Science and Technology 99(1). 1994. 31.

A.57 Taylor, John R. An Introduction to Error Analysis: the Study of Uncertainties in Physical Measurements University Science Books; ISBN: 0935702423; 2nd edition (May 1997)

A.58 Treib, T., “Error Budgeting - Applied to the Calculation and Optimization of the Volumetric Error Field of Multi-axis Systems,” IWF-ETHZ, Vol. 36 Annals of the CIRP, 1987

A.59 Zipen, R.B. “The Risks Due to Measurement Errors.” American Society of Mechanical Engineers,” Winter annual meeting, Phoenix, 1982.