ncsli 2011 conference calibration software uncertainty implementation philosophies and practices...
TRANSCRIPT
NCSLi 2011 Conference
Calibration Software Uncertainty Implementation Philosophies and Practices
Speaker/Author : Jorge Martins
NCSLi 2011 Conference
Laboratory to Document and Approve
• Calibration procedures• Uncertainty analysis• Uncertainty budgets• Calibration procedures validation plan• Calibration procedures validation results• Traceability chain for each measurement method
NCSLi 2011 Conference
Early Developments of Calibration Procedure Software•Lack of uncertainty standardization•Focus on the needs of a particular organization•No international uncertainty standards to follow
NCSLi 2011 Conference
ISO/IEC 17025 and GUM
NCSLi 2011 Conference
Requires that to issue an accredited calibration certificate a laboratory must:
•Estimate expanded uncertainty for each measurement•Estimation is compliant with GUM•Certificate shall report numerical value of uncertainty for each measurement•Implicit references to uncertainty are NOT acceptable
ILAC-P14: 12/2010 Policy for uncertainty in calibration
NCSLi 2011 Conference
NCSLi 2011 Conference
Uncertainty Implementation in Software
GroupMeasurement Uncertainty
Type of SW
I Not supported• Open source • Procedure can be modified by the laboratory
II Budget is supported • Open source • Procedure can be modified by the laboratory
III User defined per point• Locked by developer • Procedure cannot be modified by the laboratory
IV Developer defines budget • Locked by developer • Procedure cannot be modified by the laboratory
V Developer defines budget • Locked by developer • Procedure cannot be modified by the laboratory• Unc. Budget ISO 17025 Accredited by 3rd party
NCSLi 2011 Conference
NCSLi 2011 Conference
NCSLi 2011 Conference
NCSLi 2011 Conference
Document maintenance and validation responsibilitiesGroup I Group II Group III Group IV Group V
Uncertainty analysis (UA)
External maintained
External maintained
External maintained
External maintained
Not available to user
Uncertainty budget (UB)
External maintained
Internal user available
External maintained
Internal user available
InternalNot available to
userRepeatability
(type A) collection
Not availableOriginal
readings user available
Not available Original readings user available
InternalNot available to
userUncertainty
analysis approval
User resp. User resp. User resp. Developer responsibility
Dvlp. Resp.3rd party IT test
accreditedUnc. budget calculations validation
User resp.Not need SW
framework validated
User resp.Not need SW
framework validated
3rd party IT test accredited
Calibration Procedure validation
User resp. User resp.Not need SW
framework validated
Not need SW framework validated
Not need SW framework validated
Traceability Req.
User defined User defined User defined Developer defined
Developer defined
NCSLi 2011 Conference
Laboratory Calibration Procedure implementation Efforts
NCSLi 2011 Conference
Major CharacteristicsI II III IV V
User has complete control of measurement process √ √ X X XUncertainty validation is independent of measurement procedure
√ X √ X X
Merged data (measurement and uncertainty) is available to issue calibration certificate
X √ √ √ √
Validation of uncertainty engine calculations X √ X √ √Uncertainty budget is not directly synchronized with measurement procedure
X √ X √ √
Type A data collection in real time X √ X √1) √1)
Noisy units detection in real time X √ X √1) √1)
User technically challenging to make changes on the measurement process
√ √ X2) X2) X2)
Requires metrologist with SW knowledge √ √ X2) X2) X2)
User defined environmental conditions √ √ X X XUser defined Uncertainty analysis constraints and assumptions
√ √ X X X
NCSLi 2011 Conference
Calibration SW Validation Efforts
NCSLi 2011 Conference
When the software is:•Commercially available•Documented •Locked by the development house
The laboratory procedure validation: •Trained technicians•Operational Capabilities•Resources
Commercially Available Software
NCSLi 2011 Conference
Calibration SW Development ProcessPublished
Calibration Procedure
Published Calibration Procedure
Measurement Uncertainty
Analysis
Measurement Uncertainty
Analysis
SW Code ImplementationSW Code Implementation
SW Test OK?SW Test OK?
SW ValidationSW Validation
SW Validation OK?
SW Validation OK?
Y
N
N
Y
SW ReleaseSW Release
Procedure Revision?Procedure Revision?
NCSLi 2011 Conference
Calibration SW Development Deliverables
SW House Calibration Final Report:•Reference to Documented calibration procedure implemented•Validation Report summary•Uncertainty analysis Summary
Detailed SW specifications
NCSLi 2011 Conference
NCSLi 2011 Conference
Conclusion
• Significant difference in Laboratory: Validation efforts
Calibration procedure control
Uncertainty analysis control
Group I
Group II
Group III
Group IV
Group V
Estimation Laboratory Approval Efforts
100% 90% 55% 25% 20%
NCSLi 2011 Conference
Questions?