cbr– 2019 (91) proficiency testing program report

45
CBR Proficiency Testing Program 2019 (91) Copyright: LabSmart Services Issue 13 May 2020 Page 1 of 45 www.labsmartservices.com.au CBR– 2019 (91) Proficiency Testing Program Report Accredited for compliance with ISO/IEC 17043 Copyright: LabSmart Services

Upload: others

Post on 01-Jan-2022

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 1 of 45

www.labsmartservices.com.au

CBR– 2019 (91)

Proficiency Testing Program Report

Accredited for compliance with ISO/IEC 17043

Copyright: LabSmart Services

Page 2: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 2 of 45

Report

This report is available on the LabSmart Services website. The issue of this proficiency report was authorised by Jeffrey Mulholland, General Manager, LabSmart Services, May 2020. Contact Details

Email: [email protected] Mobile: 0439 208 406

Program Coordinator

The program coordinator for this program was Jeffrey Mulholland with assistance from Chris Milne, LabSmart Services. Contact Details

Email: [email protected] Mobile: 0439 208 406

Please note that any technical questions regarding this program are to be directed to the program coordinator.

Z-scores Summary

A z-scores summary for this program was issued in 18th of March 2020. This technical report supersedes the z-sores summary.

Accredited Proficiency Testing Provider

LabSmart Services is accredited by NATA to ISO/IEC 17043, Conformity assessment – General requirements for proficiency testing. Accreditation number 20650. The accreditation provides additional assurance to participants of the quality and importance we place on our proficiency testing programs.

LabSmart Services

Please see our website for further details: www.labsmartservices.com.au

Copyright

This work is copyright. No part of this publication may be reproduced in any form, transmitted or stored in any repository (e.g. mechanical, digital, electronic or photographic) without prior written permission of LabSmart Services. Please contact LabSmart Services should you wish to reproduce any part of this report.

Amendment History

Reports may be downloaded from the LabSmart Services website. Version 1 – Issued 13 May 2020

Page 3: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 3 of 45

Contents

1. Program Aim .......................................................................................................................... 5

2. Performance .......................................................................................................................... 5

2.1 Performance assessment .................................................................................................................... 5

2.2 Identified outliers ................................................................................................................................. 6

2.3 Program Summary .............................................................................................................................. 7

3. Technical Comment ............................................................................................................... 9

3.1 General ................................................................................................................................................ 9

3.1.1 Measurement Uncertainty ............................................................................................................................. 9

3.1.2 Good laboratory practice ............................................................................................................................. 10

3.1.3 Supply of test information ............................................................................................................................ 10

3.1.4 Errors .......................................................................................................................................................... 10

3.2 Statistics ............................................................................................................................................ 11

3.2.1 Accuracy of data ......................................................................................................................................... 11

3.2.2 Variation in CBR results .............................................................................................................................. 11

3.2.3 “Set s.d limit” ............................................................................................................................................... 13

3.2.4 Repeatability ............................................................................................................................................... 13

3.3 CBR Results ...................................................................................................................................... 13

3.3.1 CBR results ................................................................................................................................................. 13

3.3.2 Identification of inconsistencies and errors.................................................................................................. 13

3.4 Direct Influences ................................................................................................................................ 14

3.4.1 Load cell ...................................................................................................................................................... 14

3.4.2 Seating load ................................................................................................................................................ 15

3.4.3 Penetration rate ........................................................................................................................................... 15

3.4.4 Test (penetration / load) data ...................................................................................................................... 16

3.4.5 Accuracy of the graph prepared .................................................................................................................. 17

3.4.6 Zero-point correction ................................................................................................................................... 18

3.4.7 Rounding of CBR ........................................................................................................................................ 19

3.5 Indirect Influences ............................................................................................................................. 19

3.5.1 Pre-compaction curing ................................................................................................................................ 20

3.5.2 CBR compaction ........................................................................................................................................ 20

3.5.3 OMC & MDD ............................................................................................................................................... 22

3.5.4 LDR and LMR ............................................................................................................................................. 22

4. Statistics: Z-Score & Graph ...................................................................................................24

Page 4: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 4 of 45

5. Program Information .............................................................................................................26

5.1 Z score Summary ....................................................................................................................... 26

5.2. Program Design ......................................................................................................................... 26

5.2.1 Design ......................................................................................................................................................... 26

5.2.2 Selection of material for the program .......................................................................................................... 27

5.2.3 OMC & MDD ............................................................................................................................................... 27

5.2.4 Role of proficiency testing ........................................................................................................................... 28

5.2.5 Participant assessment ............................................................................................................................... 28

5.2.6 Reporting of results - Significant figures ...................................................................................................... 29

5.2.7 Additional test information requested .......................................................................................................... 29

5.2.8 Data checks ................................................................................................................................................ 30

5.3. Sample preparation .................................................................................................................... 30

5.4. Packaging and instructions ......................................................................................................... 31

5.5. Quarantine ................................................................................................................................... 31

5.6. Sample despatch ........................................................................................................................ 31

5.7. Homogeneity testing ................................................................................................................... 31

5.8. Participation................................................................................................................................. 32

5.9. Statistics ...................................................................................................................................... 33

5.9.1 Z-score summary ........................................................................................................................................ 34

5.9.2 Comparing statistics from one program to another ..................................................................................... 35

5.9.3 Measurement uncertainty ............................................................................................................................ 35

5.9.4 Metrological traceability ............................................................................................................................... 35

5.10 Non-statistical .................................................................................................................................. 36

6. Summary of Participants Results ..........................................................................................37

Appendix A: Instructions for testers ...........................................................................................42

Appendix B: Results Log ...........................................................................................................44

Page 5: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 5 of 45

1. Program Aim

The proficiency program was conducted in November/December 2019 with participants throughout Australia. The program involved the performance of:

AS 1289.6.1.1 (2014) – Determination of the California Bearing Ratio of a soil – Standard laboratory method for a remoulded specimen.

The program provides feedback and confidence to the construction materials testing industry regarding the competency of participants (and the industry) to perform this test. Each participant’s performance is statistically assessed and used as a measure of competency relative to all those who participated. Other measures of performance are also used (Section 5.2.5).

This report has been prepared using robust statistics. In addition, test data has been reviewed for consistency. A comprehensive technical comment is provided to assist participants to improve the overall performance of this test (Section 3). Information regarding the conduct and design of the program etc. can be found under section 5.

2. Performance

2.1 Performance assessment

In discussing the outcome of this program, the following have been used to determine aspects of testing performance that needs to be investigated or reviewed.

Statistical • Z-scores based on submitted CBR results

Non-statistical • Errors

• Identification of inconsistences

• Non-adherence to test method

• Accuracy of calculations

• Accuracy of graphing

See section 5.2.5 & 5.10 for further detail.

Page 6: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 6 of 45

2.2 Identified outliers

In most proficiency testing programs, the identification of outliers is relatively straight forward. This is not the situation with CBR testing due to the large standard deviation in CBR results obtained. There are also many steps in the testing process that contribute to the quality of the CBR result.

Participants with statistical outliers, a departure from the test method or errors need to investigate the aspect of testing shown in Table 1. Those with significant departures compared to other participants need to review the aspects summarised in Table 1.

Participants, where there is a concern regarding the accuracy of the results or summited data, are requested to investigate their submissions. Others have been identified as able to benefit from reviewing their submissions where it is felt the quality of testing may be improved.

In Table 1, there are no participants listed under some sections as needing to investigate (e.g. ‘penetration rate’ and ‘test data’). This is not because there are no concerns identified, only that the test method does not necessarily identify, address or quantify these aspects of the test affecting the accuracy/precision for this test.

The more times a participant’s code appears in Table 1, the greater the need for follow up.

Page 7: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 7 of 45

Table 1: Participants identified where investigation or review follow up is warranted

Aspect of testing Section Investigate Review

CBR results 3.3.1 - K3, B3

Identification of inconsistences and errors 3.3.2 Z8 -

Load cell 3.4.1 - -

Seating load 3.4.2 - -

Seating load set to zero 3.4.2 - -

Penetration rate 3.4.3 - -

Test (Penetration / load) data 3.4.4 - M3, F3, W4

Accuracy of graph 3.4.5 - W4

Zero-point correction 3.4.6 - K3

Pre-compaction curing 3.5.1 F4, Y5 -

CBR compaction 3.5.2 -

J2, L3, A8, Y5, R3, V3, F9,

L8, X2, Y7, F4, W9, U5, P9

OMC & MDD 3.5.3 - -

LDR 3.5.4 M8, Y7 -

LMR 3.5.4 W4, M3, F4,

Y5, C8, K3, P9 -

2.3 Program Summary

Based on LabSmart Services programs, there has been an observable improvement in CBR testing over the last ten years as measured by the coefficient of variation (CV). The last four programs have levelled out in the mid to low 20’s with an average CV of around 23%. (Table 3)

To the many participants and organisations who have participated “well done” and “thank you” for your participation.

Page 8: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 8 of 45

The program identified those aspects of the test that affect accuracy (direct influences) and those aspects of the test that are less controllable (indirect influences).

The CBR graph would appear to be the only way of checking the validity of the results obtained. In many cases, the graphs prepared do not adequately fulfil this function.

A continued reduction in testing errors, better graphing and supervisor checking would greatly improve the accuracy of CBR testing.

Improvements to the test method, by better defining the test process (e.g. graphing), limits and expected outcomes would also significantly improve the accuracy of the test.

It should be noted that most of the above comments relate to the accuracy of the test. It is unlikely that improving the accuracy of testing is going to improve the current variation in CBR results shown.

Enough proficiency programs have now been conducted to show that the current spread (variation) in results is both a reliable and accurate estimate.

This proficiency program provides increased understanding of current test practices and potential sources of variation. It also allows monitoring of improvements in testing and provides the opportunity for participants to improve their competency. A summary of the program statistics is shown in Table 2.

Table 2: Summary of statistics for the CBR program. Some results have been rounded. *Min, Max & Range are with outliers excluded

Statistic CBR

Number of participants 29

Median 58.7

Normalized IQR 15.2

Minimum* 24.6

Maximum* 90.0

Range* 65.4

CV (%) 26

Page 9: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 9 of 45

3. Technical Comment

3.1 General

3.1.1 Measurement Uncertainty

Aspects of the test can be split into whether they have a direct (measurable) or indirect (not

measurable) impact on the calculated CBR result. This is part of the process taken when

calculating measurement uncertainty.

Direct influences can be measured or estimated (section 3.4). Direct influences generally involve

participant errors or inconsistencies in testing. Testing can be improved by improving the

accuracy with which these aspects of the test are performed. For example:

• Accuracy of the load cell

• Accuracy of seating load

• Accuracy of penetration

• Accuracy of the rate of penetration

• Accuracy of recording force readings

• Number of data points selected

• Accuracy of the graph prepared

• Accuracy of the zero correction

• Rounding of results

Indirect influences cannot be practically measured or improved easily (section 3.5). Indirect

influences generally involve non – compliance to the test method requirements or limits. Test

variation is minimised by strict adherence to the test method. For example:

• OMC & MDD

• Moisture content

• LDR & LMR

• Compaction, i.e. layer thickness, compaction pattern, number of blows

• Curing of sample

Page 10: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 10 of 45

3.1.2 Good laboratory practice

Proficiency program participants are expected to comply with the requirements of the program and meet basic laboratory standards. Good laboratory practices cover those aspects of laboratory operations that are in keeping with NATA accreditation. Some aspects that are particularly relevant for this program are:

• Supervision of testing

• Following the test method

• Following proficiency testing instructions

• Correctly filling out paperwork, i.e. PT log sheet

• Checking of calculations and data, i.e. free of errors

• Reality check of results, i.e. does it fit the type of material submitted

Compared to earlier CBR proficiency testing programs, there has been significant improvement in most of the above areas. However, as detailed in subsequent sections, there is still room to improve the performance of CBR testing.

If participants are not meeting the above basic requirements, it also raises concern as to what other omissions or errors are occurring during testing that remains undetected.

3.1.3 Supply of test information

Most participants supplied all the testing details requested. This additional information (see section 5.2.7) is important as it is used to validate the results received and to assist in providing the feedback given in the following sections.

Participants are always welcome to contact the program coordinator if they require further explanation as to what information is required or how to proceed with testing.

There were several participants that did not supply all the data requested or supplied incorrect data. These participants are encouraged to review what they have submitted against other participants in order to improve data supplied in future.

3.1.4 Errors

Errors may arise from several sources, an incorrect calculation, transcription error, the wrong methodology used, not following the test method etc. Many of the comments in the following sections relate to errors.

Although some of these may have only a small impact, they do accumulate. Others can have a large impact such as incorrect graphing technique and zero correction.

Page 11: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 11 of 45

3.2 Statistics

The use of statistics are a very useful and practical means of analysing test data. Below are some aspects that affect statistical outcomes.

3.2.1 Accuracy of data

If the test data is in error, then any statistics calculated may also be in error. Any interpretations made, based on the statistics, may therefore also be in error. Most proficiency programs can handle a few inaccurate results without any concern about the veracity of the program outcome. Most of the technical comment concerns the accuracy of the CBR test results.

3.2.2 Variation in CBR results

Enough proficiency programs have now been conducted to show that the current assessment of the spread (variation) in results is both reliable and accurate (Table 3).

The standard of CBR testing has improved, and the spread in results (variation) has decreased over the last 10 years.

The coefficient of variation (CV) has settled at around 23%. Note that the ‘gold’ highlighting in (Table 3) shows programs conducted to the current 2014 test method.

Page 12: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 12 of 45

Table 3: Comparison of CBR program results for the last ten years

Year Program Median CV

2019 91 59 26

2018 81 51 23

2017 74 52 21

2016 67 155 21

2015 59 140 20

2014 54 74 31

2013 46 37 29

2012 37 44 20

2011 48 61 35

2009 16 30 32

The industry has expressed concerns that from an engineering “End User” perspective that such large variations in CBR results are impractical. It is also undesirable from a laboratory testing perspective. However, without changing the test method, the variation is what the current method produces.

As has been indicated in previous proficiency programs, it is the middle 50% of participants results that is far larger than it should be. It is this group of results, therefore that is of primary interest when considering ways in which to reduce the spread of results.

Much of the technical feedback relates to improving the accuracy of CBR testing. This will not improve the spread of results, but they will become more accurate.

Further work on improving the test method is needed in order to improve (decrease) the spread shown by the middle 50% of participants.

Page 13: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 13 of 45

3.2.3 “Set s.d limit”

In previous CBR proficiency programs, the z-score statistics have been recalculated using a “Set s.d limit” or “Target s.d”. The purpose of which was to bring the variation (spread) in results back to something useful to geotechnical engineers and others in the industry. It appeals as a very practical way of approaching the problem.

There is no reason however to suspect that, based on the outcome of this program, or other programs that any result within the middle 50% is better than any another result from the middle 50%. The “Set s.d limit” outcome then does not provide any useful information.

If the accuracy of many CBR test results is questionable along with the median value, then results may lie above or below any ‘set limits’. It does not identify problems or inaccurate results; worse, it could indicate results as being satisfactory when they may not be.

The “set limits” is therefore no longer used in CBR PT programs.

3.2.4 Repeatability

This program focuses on the variation (spread) of results between laboratories (reproducibility).

It is questionable that with the large variation is shown in CBR results that an estimate of repeatability (performance of two identical samples) would yield reliable information.

For some tests, the homogeneity data can be used as a guide to the repeatability. However, for CBR such an estimate may be unreliable as the precision may be good (same machine and pace rate etc.), but it is unknown if the overall accuracy is good or poor.

3.3 CBR Results

3.3.1 CBR results

Z-scores and associated statistics were calculated on the CBR results as submitted and are detailed in section 4. While not outliers, Participants with z-scores 2 or l -2 l and above should review the results obtained (B3 and K7).

3.3.2 Identification of inconsistencies and errors

There are many steps within the conduct of the test (methodology) that can become a source of error or where inconsistencies can occur. As well there are limits posed by the test method itself that may also contribute. For example, compaction and moisture content. See section 3.4 and 3.5 that explore these aspects further.

Page 14: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 14 of 45

The use of a detailed CBR graph is a quick and reliable means of checking results. A rough mathematical check was undertaken by the program coordinator for all participants. Those with significant differences were regraphed, and the CBR recalculated.

It should be noted that during these steps a common issue encountered was the use of incorrect units of measurements and rounding. In most cases once adjustments were made many of these issues were resolved. It is recommended that all participants take time to review their results before submission. A summary of all results ‘as submitted’ can be found in section 6.

Z8, is of note in this program as the test data supplied within the ‘Results Log’ were transcribed incorrectly by a factor of 10 from the participants internal worksheet (Supplied as additional information).

3.4 Direct Influences

The following sections cover many aspects of test methodology. From previous programs, it has been noted that even with corrections resulting from re-graphed data and using unrounded results, it has only a marginal effect on the middle 50% of participants. In other words, the corrections tend to be random with some corrected CBR values increasing while others decrease.

Overall, it suggests that while the accuracy of testing can and should be improved, there may be little change to the overall spread of results obtained for the CBR test.

3.4.1 Load cell

In section 6, the load values are shown for each participant. Some laboratories used more data points than requested (great to see).

Most participants in this program used load cells with four participants (Y8, C8, K3 and X2) that used load rings. Most devices were calibrated to ‘Class ‘A’ or a combination, e.g. A/B/C. Participants generally used a 50 kN load device.

Selection of the correct load cell capacity is dependent on the experience of the laboratory and where possible prior knowledge of the material to be tested. Unfortunately, due to the large range of CBR results possible from participants, this information cannot be given by the program organisers prior to testing.

If a load cell or ring does not have enough capacity during testing, it is important that testing is stopped on approach to the maximum capacity of the load cell. Exceeding the capacity of a ring can cause permanent damage (not visually obvious to a user).

Another consideration is the resolution at the lower end of the load scale to accurately measure the seating load. For load cells used in this program that are on the larger side (e.g. 50kN), it may be difficult to accurately measure small loads.

Page 15: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 15 of 45

Often this is not a lack in ability of the load cell, but a reflection of the normal calibration practise where the calibration may not extend to the low load values required for seating loads or low CBR values. Laboratories may need to request calibration facilities, where possible, to specifically cover the seating loads required when undertaking the load cell calibration.

3.4.2 Seating load

The test method requires that the least amount of force be used for the seating load. It is important that the piston is in contact with a stable surface. The seating load is considered the ‘zero-point’ from which the load values and penetration commence.

In this program, the CBR was greater than 30%, and a seating load of 250 N should have been used. Thirteen participants (68%) used the correct seating load. The test method indicates that 50N should be used for CBRs less than 30%.

At high CBRs, the seating load has only minimal effect on the CBR obtained but does influence where the penetration points fall. For this type of material, any effect of incorrect assignment of the zero penetration is usually cancelled out with the zero-point correction offset if performed correctly.

All participants indicated that the seating load was set to zero except for participants Y8, M3, C8 and X2. Participant Y8 and X2 used a load ring where setting the dial gauge to zero could introduce an error in the force calculated.

Not setting ‘back to zero’ again can lead to an inaccuracy in the load scale, creating an offset.

However, errors in both processes (seating load applied and resetting back to zero) may influence the CBR. An error in the penetration of ± 0.5mm could lead to a change of ± 4.5% CBR. This may not seem much, but in the rounding process when reporting this may cause a difference of 10% in the CBR result.

3.4.3 Penetration rate

The test method indicates that the machine used must be capable of “…forcing the penetration piston into the specimen at a uniform (not pulsating) rate of 1.0± 0.2 mm/min during the complete test….”.

NATA indicates that the penetration rate is checked every two years.

Page 16: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 16 of 45

It is not entirely clear, based on input from previous programs, if the standard means an ‘average rate’ or if it means it must be met at ‘all times’. If it is taken as an average rate then theoretically you could have half the penetration at 0.5 mm/min and the other half at 1.5 mm/min and still arrive at the average rate of 1.0 mm/min.

For ‘hand’ operated devices, it is hard to check other than an overall average. A motorised platform was used by most participants with one participant (M8) using a ‘hand’ operated unit.

With load cell units, they usually allow the rate to be checked as you go on a ‘per 0.5 mm of travel’ etc. This can be done on a ‘test by test’ basis so is a very good record of meeting the requirements of the standard.

In previous programs, the rate was requested with most participants reporting the test method requirement rather than the actual rate achieved. For this program, more detailed information was requested from participants, i.e. average, minimum and maximum rates achieved.

Around 52% of participants accurately completed this section of the log sheet. Those that did provide the information all meet the requirements of the test method.

The penetration rate is linked to the slope of the load/penetration curve. It is therefore significant in determining the CBR and hence the set limits placed on the rate of travel by the test method.

3.4.4 Test (penetration / load) data

The number of penetration points selected is extremely important. Most participants recorded the requested additional load/penetration data, and some recorded more. A very good outcome. Participants M3, F3 and W4 should review the number of data points submitted.

The test method specifies a minimum data set (0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 4.0, 5.0, 7.5, 10.0 and 12.5 mm penetrations).

The ‘key’ word in the test method is “at least”. In other words, if you know the material well (i.e. have a CBR history of the material) then you should be able to use fewer points; otherwise you need to record loads at more points.

Additional data points are needed to:

• Allow for the discount of an abnormal data value

• Have sufficient points left so that the discounting of a point does not compromise the test result

• Have sufficient points to fit a straight line and a curve

• Have sufficient points above the straight section of the graph.

• Have sufficient points to be able to tell that you have an abnormal data point

Page 17: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 17 of 45

It is evident also that two few data points can have a measurable difference in the result that are obtained, as much as suspect data. Greater confidence in the result and accuracy is obtained when more points are taken.

3.4.5 Accuracy of the graph prepared

In the previous CBR proficiency program, all participants results were re-graphed. In this program, only a selected group (approximately 50%) have been re-graphed. Consequently, any inconsistencies found will not need to be investigated.

Graphing is discussed in this program due to its importance in deriving an accurate result and being able to check the CBR result obtained. The CBR test method does not emphasise this aspect.

Graphing of results has been an issue for the last ten CBR proficiency programs. Overall graphing has improved vastly over this time.

Regardless of what graph is submitted to the client, a detailed graph for use by the laboratory is important as it is the primary method of checking that a reasonable result has been obtained.

The test method is also not very descriptive regarding the quality of the graph prepared. In previous proficiency programs, considerable feedback was given.

Nine participants did not submit a graph (N2, W4, W9, T5, F9, L8, U5, R3 and P9.). These participants need to review the lack of a graph. Five participants submitted hand-drawn graphs (A8, Z8, N5, V4 and X2), with all other participants submitting computer-generated graphs.

Table 4 shows how a variation in the final CBR result can be affected by the accuracy of

graphing. The differences in CBR results shown in Table 4 are noted as significant. Participants

shown in Table 4 need to review their results.

Page 18: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 18 of 45

Table 4: Selected participants - Re-graphed CBR data results

Participant Recalculated Review Participant Recalculated Review

Participant Code

Zero Correction

(mm)

Zero Correction

(mm) Difference

Unrounded CBR (%)

Unrounded CBR (%)

Difference

W4* 2.9 3.0 0.1 37.3 51.9 14.6

K3 0 2.4 2.4 24.6 49.6 25.0

*W4 did not supply their graph or the additional test data requested, limiting the amount of feedback available.

3.4.6 Zero-point correction

Overall, most participants calculated a zero-point correction and applied it.

Seating load and zero correction combined generally result in small changes. However, sometimes small changes can have a significant effect on the CBR result and particularly when a BR value is to be rounded either up or down to the nearest 10%. A variation of ± 10 % CBR is not unrealistic.

As seen with participant K3, Not applying the zero-point correction can have a significant

impact, as shown in Table 4.

.

Page 19: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 19 of 45

3.4.7 Rounding of CBR

The reason for rounding is not entirely clear in the Australian Standard. It perhaps acknowledges that CBR values are quite variable and rounding makes the results easier to use and compare when grouped together, i.e. takes out some of the fluctuations.

Laboratories were asked for the unrounded Bearing Ratio rather than rounded CBR results. Part of the design consideration of this program was to try and isolate as well as minimise sources of variation. The process of ‘rounding’ was identified as adding to the variation of determining CBR. The statistics associated and test variation with the CBR results will often increase slightly if rounded results are used. At other times it may slightly decrease the variation shown.

3.5 Indirect Influences

The following aspects of the testing methodology are difficult to relate to the final CBR test result. They can be measured individually, but the influence it has on the CBR result is more difficult due to the ‘unknown interactions’ they have on each other.

It is clear however, that more accurate measurement of these aspects of the test in conjunction with the better definition within the test method should assist with improving the overall accuracy of the test.

Page 20: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 20 of 45

3.5.1 Pre-compaction curing

There were a range of curing times used by participants. The majority used 24 hours and above. The curing times specified by the test method are minimums. More curing, if done correctly, is better than less.

The test method now requires laboratories to select the appropriate curing time based on material, Liquid Limit and departure from OMC.

There were a wide range of ‘liquid limit’ values used and hence a wide range of curing times. The 2017 amendment to the test method allows for the ‘liquid limit’ to be estimated based on experience. Most participants (approximately 85%) based the curing on their ‘estimate’ of ‘liquid limit’.

An estimate of the MC of the material ‘as received’ and whether within 2% of the OMC was not specifically requested as part of this program.

Section 6c of the test method requires the material to be within ±0.5% of the specified moisture. For this program, OMC was 12.0%. The allowed range than was 11.5 to 12.5 %. The following participants (F4 & Y5) indicated MCs outside this range and should investigate; However, it is suspected that these participants incorrectly filled out their result logs.

3.5.2 CBR compaction

The proficiency program required participants to perform the CBR compaction using the OMC and MDD values provided and 100 % standard compaction.

Test methods relating to compaction are very specific about the energy input into the process. This is largely governed by the spread of hammer blows and the number of blows used. The revised CBR method now stipulates the pattern to be used when compacting the CBR mould. The test method, however, does not specifically require the number of blows delivered to be recorded, as it is an important part of the test it should be recorded.

It is expected that by compacting a calculated amount of material to a set height that the desired density will be achieved. The blows will vary depending on the material type and moisture. Depending on how this is done a variation in the number of blows per layer is the typical outcome. However, between layers, these should remain reasonably close.

For determination of OMC/MDD using standard compaction 25 blows per layer is used. To achieve the same energy input around 53 blows is required for the larger CBR mould.

Page 21: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 21 of 45

More or less blows than 53 may be needed for a variety of reasons:

• The inaccuracy of the OMC and MDD initially

• Blows not delivered in a regular pattern

• Nature of the material may cause it to move around the mould excessively

• Material added is higher or lower than the prescribed layer depth

The blows delivered provides an insight into whether any of the above issues may have had an effect.

Relying on the dry density calculated is useful, but it is a calculated value and dependant largely on how representative and accurate the moisture determination was.

How much variation is reasonable? This is at present unknown, but for this program, a variation of 40 to 60 has been used with a variation between layers of 5 blows.

The following participants shown in Table 5 do not meet this criterion and should review their results.

Table 5: Participants with high or low number of compaction blows

CBR Compaction

< 40 Blows per layer

> 60 blows per layer

Difference in blows greater than 5 between layers

No result

J2, L3, A8, Y5, R3

- J2, V3, F9, L8, X2 Y7, F4, W9, U5,

P9

It may not affect the dry density obtained, but there is concern that it may influence the CBR obtained.

For low compaction, it may influence:

• segregation of particles,

• uneven compaction,

For high compaction, effects such as:

• orienting the soil particles,

• segregation of particles,

• causing fissures,

• breaking up of particles,

• uneven compaction,

All of which could influence the CBR without affecting the dry density value achieved. CBR results may be higher or lower depending on the influence. It is unclear if this has been investigated in recent times.

Page 22: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 22 of 45

3.5.3 OMC & MDD

Different determinations of OMC & MDD by different laboratories will give rise to a spread of results (Variation). To limit the effect of this variation on the CBR testing in this proficiency program, the OMC & MDD have been predetermined. This information was supplied to participants (See instructions Appendix A) so that all participants used the same OMC & MDD values.

3.5.4 LDR and LMR

Calculation of LDR & LMR

Participants were requested to submit:

• The sample moisture immediately prior to compaction (w1) in accordance with clause 6(c) of the standard.

• Moisture content variation (Wv)

• The Laboratory Moisture Ratio (LMR)

• The Laboratory Density Ratio (LDR) and

• Dry Density (before soaking)

These intermediate results are noted in the test method as needing to be reported or required to determine compliance with the test method.

The reported LDR and LMR values were recalculated using the reported moisture from clause 6(c) and density (before soaking). It is suspected that some participants had incorrectly reported the moisture of the sample as being that of ‘as received’.

There were several participants that had difficulty in calculating the intermediate results detailed above.

The participants listed in Table 6 showed inconsistencies in the values submitted, throwing doubt on compliance with the test method and should be investigated.

Page 23: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 23 of 45

Table 6: Participants with inconsistencies in calculating LMR and LDR

Information submitted Investigate*

Moisture (Clause 6c) F4, Y5

Variation in moisture content (not) reported F4, Y5, C8

LMR does not match reported moisture W4, M3, F4, Y5, C8, K3, P9

LDR does not match reported dry density M8, Y5 *It suspected that in many cases, these participants incorrectly filled out their result logs. Additionally, LabSmart asked for serval results to be reported to a greater accuracy then that of the standard, it is possible these participants may not have complied with this request and in effect suppled rounded results. In either case, this should be investigated by the participant

Achievement of OMC & MDD

Participants were requested to compact the sample to 100 % standard compaction at 12.0% moisture.

Overall most participants achieved the desired range for OMC and MDD, which was a very good outcome. Achieving the LMR and LDR is a requirement of the test method and must be met for the results to be valid and hence reportable.

Participants with results outside these limits as detailed in Table 7.

Table 7: Participants that are outside the limits set for LMR and LDR OR W1

OMC ± 0.5%

Moisture Range %

LMR Range %

Investigate*

11.5 95.8

C8* 12.0

12.5 104.2

MDD t/m3

Density Range t/m3

LDR Range ± 1%

Investigate

2.044 99.0

- 2.065

2.086 101.0

*It suspected that in this case the LMR would have been shown to be achieved had the participant calculated them correctly, Moisture Before Compaction (W1) was recorded as 11.9%.

Page 24: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 24 of 45

4. Statistics: Z-Score & Graph

% %

V9 77.5 1.24 P9 33.2 -1.68

N2 59 0.02 B5 NR

B3 90 2.06

M8 60 0.09

L3 60.3 0.11

W4 37.8 -1.38

F3 60 0.09

J2 58.7 0.00

Y8 57.6 -0.07

W9 70 0.74

M3 60 0.09

Y7 30 -1.89

F4 48 -0.70

M9 47 -0.77

A8 71.5 0.84

Y5 48.7 -0.66

C8 50 -0.57

K3 24.6 -2.24

T5 60 0.09

Z8 46.7 -0.79

V3 30 -1.89

F9 70.4 0.77

L8 68.6 0.65

U5 69.4 0.70

R3 67.5 0.58

N5 55.8 -0.19

U4 53.5 -0.34

X2 47 -0.77

Number of results 29

Median 58.70

Median MU 3.53

First Quartile 47.00

Third Quartile 67.50

IQR 20.50

Normalised IQR 15.20

CV (%) 25.9

Minimum 24.60 ()

Maximum 90.00 ()

Range 65.40 ()

Note: A # indicates an outlier where the z-score obtained is either greater

then 3 or less than -3. Codes for all participates are shown. The results

column shows a blank entry or 'NR' for those participants that did not submit

a result for this test. Results in green have been calculated by the program

coordinator. An R indicates an abnormal result rejected by the program

coordinator. Minimum, Maximum and Range are calculated with outliers

excluded, those in brackets include outliers.

CBR - Sample A: Z - Scores

Code Z Score Code Z Score

Statistic Value

Test

Result

Test

Result

Page 25: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 25 of 45

23

25

5

4

7

11

19

2

8

9

26

27

17

16

13

14

28

20

6

29

12

21

18

ReviewWeak

Consensus

Weak

ConsensusReview

Z-score

Strong Consensus

CBR - Sample A: Z - Score Graph

B3

V9

A8

F9

W9

U5

L8

R3

L3

M8

F3

M3

T5

N2

J2

Y8

N5

U4

C8

Y5

F4

M9

X2

Z8

W4

P9

Y7

V3

K3

-3 -2 -1 0 1 2 3

Page 26: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 26 of 45

5. Program Information

5.1 Z score Summary

The proficiency program was conducted in November/December 2019. A ‘Z-score Summary’ summary was issued on the 13th of March 2020. A copy was e-mailed to all participants who submitted results. The summary is intended as an early indicator of participant performance. This program report supersedes the z - score summary. Further information can be found in section 5.9 ‘Statistics’.

The z-scores generally do not vary significantly between the “summary” and the “Final Report”.

5.2. Program Design

5.2.1 Design

This program is one of a series of CBR programs conducted by LabSmart Services over the last ten years.

The CBR test is a complex test from a measurement uncertainty perspective despite its apparent technical simplicity. Unfortunately, the CBR test method does not provide guidance about some aspects of the test, such as repeatability or reproducibility. There also appears a lack of guidance on both the performance and the interpretation of the test within the industry. The range of test results obtained in a proficiency program, for any given sample, has been far wider than is generally acceptable to the industry. This adds to the difficulty in interpreting the outcome of CBR proficiency testing programs.

Part of the design of each program involves asking for the right information. The correct analysis of the data collected then allows feedback to be offered to enable participants to improve in the performance of this test.

The program was designed to provide technical feedback regarding performance as well as possible improvements in performance. Other considerations involving the design of the program are detailed throughout section 5.

Page 27: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 27 of 45

5.2.2 Selection of material for the program

The test in this proficiency program is operator skill/experience dependant.

Different materials are selected for each program to mirror the range of materials encountered in practice and hence the results obtained. The higher the CBR value, the greater the variation encountered.

This program provides a sample that gives results in the range that would be commonly tested by laboratories. It is expected that the level of experience/skill needed to perform these tests will present a reasonable assessment of the overall competency of the tester and industry performance.

5.2.3 OMC & MDD

The determination of OMC and MDD is usually an initial stage undertaken prior to performing a CBR test. The determination of these two parameters can show a significant variation. In turn, having an impact on the variation obtained for CBR results.

The intention of the program is to minimise the influence on the CBR results that could arise from laboratories determining these values in-house and reduce the likelihood of different OMC and MDD values being applied.

To assist in reducing this variation, participants were requested to use 100% standard compaction and use:

• OMC = 12.0%

• MDD = 2.065 t/m3.

These values were determined prior to the program.

This approach has been used to minimise variation; however, other aspects may still contribute to the variation observed. OMC/MDD values may vary from person to person, but this may not be so important if the same person determines OMC/MDD and CBR. That is a low compaction on the OMC/MDD should give the same compaction on the CBR. Overall, it is still considered that a set OMC/MDD will contribute the least variation.

Page 28: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 28 of 45

5.2.4 Role of proficiency testing

The determination of outliers is an important task of this proficiency program. A secondary function is to provide feedback that can help those with outliers identify possible areas to investigate as well as assist all participants to improve.

In addition to the statistics, proficiency programs often obtain other information that is not normally available to a laboratory. It allows for a better understanding of the testing and can provide information that can lead to improvements in the testing process or test method.

Proficiency testing enables participants to measure competency against others. It is also a measure of staff performance and the equipment used. Apart from ‘measurement uncertainty’ it is the next most useful tool a laboratory has in better understanding the performance of a test.

5.2.5 Participant assessment

In discussing the outcome of this program, the following have broadly been used to determine outliers and areas for investigation/review.

Statistical • Z-scores based on submitted CBR results

Non-statistical • Errors

• Identification of inconsistences

• Non-adherence to test method

• Accuracy of calculations

• Accuracy of graphing

Participants are asked to “investigate” statistical outliers. Assessment of each participant is based on a z-score that is related to the program consensus value (median). This is used to determine any statistical outliers.

Errors in testing, test method not followed or where test parameters are outside the limits set in the test method all need to be “investigated”. See section 5.10.

Other matters identified are shown as “Review”. These are matters that would help improve testing, and in most cases would be considered outside normal testing parameters. It is sometimes difficult to determine as the CBR test method often does not provide enough guidance.

Page 29: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 29 of 45

Compliance to proficiency program requirements including the correct calculation of results and adherence to program and test method requirements may also be used as part of the assessment process (sees section 5.2.7). Participants may also be asked to investigate any discrepancies detected with the paperwork submitted. See section 5.2.8 for further details.

5.2.6 Reporting of results - Significant figures

The number of decimal places (significant figures) reported for a test has a bearing on the statistical analysis and therefore, the interpretation of the results. There is a need to strike a balance between what is desirable from a statistical viewpoint and test method accuracy while recognising how the results are used in practice.

Too few decimal places (e.g. due to rounding) can cause an increase in the observed spread of results. Increasing the number of decimal places (with respect to normal reporting) can distort the observed spread of results compared to that encountered in actual practice. Large numbers of similar, rounded results can also cause a distortion in the analysis.

For example, rounding to 10 % means that any number between 45 and 54 will become 50%. If the largest value is 45 in a set of results, it is pushed out to 50 through rounding. Rounded results may better reflect the repeatability and reproducibility of the test according to the rounding in the test method but are not as useful when considering laboratory performance.

For this program, it was decided that the benefits of using additional decimal places would complement the aim of the proficiency program. Participants results were analysed as received regardless of whether there were more or less significant figures than the number requested by the program.

5.2.7 Additional test information requested

This program requested additional information as detailed in Section 6 not usually reported. The additional information is, however, consistent with the performance of the test and the records the test method requires laboratories to maintain or is consistent with ‘good laboratory’ practice. The additional information is used to interpret participant’s performance and assist with providing technical comment, including feedback on outliers and possible participant improvement. It is also used to validate the results received.

Page 30: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 30 of 45

Participant results can be rejected if they do not conform to the program requirements. The correctness and quality of the information supplied is assessed as to the veracity of the information or results submitted. An adverse assessment may lead to the whole of the participant’s results being rejected or asked to investigate/review some aspect of what has been submitted.

5.2.8 Data checks

As often observed ‘operator errors’ can occur in the result calculation process. Every participant’s results were verified as reasonable. Checks, however, are only as accurate as the raw data supplied by each participant. These checks also help ensure that the data is comparable. Any inconsistencies identified during this process are identified as possible feedback for participant improvement. In some cases, inconsistencies identified may need to be investigated by participants.

Proficiency testing providers are obligated under their accreditation standard to remove results known to be incorrect or where a participant has not followed the test method, including adherence to prescribed limits. Not providing all data requested, particularly where it is used to assess the validity of the results obtained (e.g. compaction, MC) is also a valid reason to reject a CBR result. These matters are not ‘black & white’ but require some interpretation as to each component’s importance.

Keeping results that may be suspect in the statistical pool may distort the statistical outcome. However, if all the results found to be inaccurate or not meet the test method etc. were rejected from this program, the pool of results would be significantly decreased. A balance must be struck.

Participants need to be aware that the program coordinator performing the checks may not have access to the full set of results for each participant (e.g. significant figures, etc.). This can sometimes cause differences between what the participant has calculated and what the program coordinator calculates.

Also, due to the large amount of data associated with this program, it is entirely possible that the coordinator may not have recalculated some participants results correctly, although a considerable effort is made to prevent this from occurring.

5.3. Sample preparation

Sufficient material of a homogeneous appearance was obtained for the proficiency program. The lot was partially dried then mixed to ensure, as far as possible, a homogeneous material throughout. The material was sampled and placed into numbered plastic bags.

Page 31: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 31 of 45

Ten samples were drawn at regular intervals from the lot for homogeneity testing.

Each participant received a randomly drawn sample from the remaining samples. A unique program code was assigned to each sample.

5.4. Packaging and instructions

Each plastic bag was sealed with a zip tie and placed into a sturdy box. Each participant received one box with a sealed sample labelled ‘2019 (91) CBR Sample’. The sample weighed approximately 10 kg. Instructions and a ‘Results Log’ sheet were enclosed (See Appendix A & B). Participants were instructed to test according to the nominated test method and report to the accuracy indicated on the ‘Results Log’.

5.5. Quarantine

There were no samples that needed to comply with quarantine requirements.

5.6. Sample despatch

Samples were dispatched to participants Mid November 2019 via courier. Dispatched samples were tracked from ‘despatch to delivery’ for each participant.

5.7. Homogeneity testing

Homogeneity samples were selected, evenly spaced, from the prepared participant samples. Samples for homogeneity testing were packed in the same way as those for all participants. The homogeneity samples were tested by a NATA accredited laboratory. To approximate the same conditions, the same instructions were given to the laboratory performing the homogeneity testing.

Ten samples were tested for homogeneity. One sample had to be removed as either outlier or not representative of the statistical sample population. An additional sample could not be reported due to a technical issue encountered during testing.

The overall variability associated with the homogeneity samples was considered satisfactory. The average of the homogeneity samples also lies within 1 s.d of the program median value. This provides confidence that any outliers identified in the program represent statistically valid outliers. A statistical analysis of the homogeneity testing results is provided in Table 8.

Page 32: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 32 of 45

Table 8: Homogeneity results

Code

CBR

%

CBR

%

(Rounded)

H1 52.9 50

H2 50.9 50

H3 53.4 50

H4 55.1 60

H5 - -

H6 57.9 60

H7 57.0 60

H8 - -

H9 54.1 50

H10 50.8 50

Mean 54.0 54

Standard Deviation 2.6 5

Range 7.2 10

Coefficient of Variation (%) 4.8 10

5.8. Participation

Twenty-nine participants from around Australia entered the program. All Twenty-nine participants returned results. Participants were requested to return results by 16th December 2020; However, this was extended into early January 2020 for participants affected by the 2019-2020 summer bushfires.

Page 33: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 33 of 45

5.9. Statistics

Z-Scores were calculated for each test and used to assess the variability of each participant relative to the consensus median. A corresponding z-score graph was produced for each test.

The use of median and quartiles reduces the effect that outliers have on the statistics and other influences. Therefore, z-scores provide a more realistic or robust method of assessment.

Some results were reported by participants to more decimal places than requested as part of the proficiency program and by others to fewer decimal places. In all instances, test results have been used as submitted by participants.

Assessment of participant’s data is undertaken to ensure the data is statistically comparable. Checks are undertaken to ensure the data calculated matches that reported by the participant and that the appropriate corrections etc. have been applied if required. The level of checking required varies from program to program. If inconsistencies are identified, the data may be removed or amended with the discrepancy highlighted.

A z-score is one way of measuring the degree of consensus with respect to the grouped test results. The z-scores in this report are an approximate of the standard deviation. For each test, a z-score graph is shown. Use the graph to visually check statistically how you compare to other participants.

The following bar (Figure 1) is shown at the bottom of each graph. This helps to quickly visualise where each participant’s results falls.

Figure 1: Z-score interpretation bar

Review Weak

Consensus Strong Consensus

Weak Consensus

Review

For example:

• A strong consensus (i.e. agreement) means that your test result is close, i.e. within 1 standard deviation of the median.

• A weak consensus means that your test result is satisfactory and is within 2 standard deviations of the median.

• If you have obtained a test result that is outside 2 standard deviations, then it may be worth reviewing your testing processes to ensure that all aspects are satisfactory. Only those obtaining a z-score approaching 3 (I.e. outside 2.75 range) have been highlighted in the report for review.

Page 34: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 34 of 45

If you have obtained a test result that is outside 3 standard deviations, then you will need to investigate your testing processes to ensure that all aspects are satisfactory.

Participant assessment is not based purely on statistical analysis. Compliance to proficiency program requirements including the correct calculation of results and adherence to program requirements, may also be used as part of the assessment process. Participants may also be asked to investigate any discrepancies detected with the paperwork submitted. See section 5.10.

For further details on the statistics used in this proficiency program can be obtained from LabSmart Services or download the ‘Participant Guide’ from the LabSmart Services website.

5.9.1 Z-score summary

A “Z-Scores Summary” is issued soon after most results are received. It gives participants early feedback as to any program outliers. The summary is available on the LabSmart Services website up until the final report is issued. The final report supersedes the z-score summary.

The final report contains detailed technical feedback regarding the performance of tests and revised z-scores. The inclusion of late results or corrections are at the discretion of the program coordinator. In some instances, this may change some of the z-scores slightly, but generally, the performance outcome remains the same. If there is any significant impact, it will be discussed within section 5.1 of the report.

Page 35: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 35 of 45

5.9.2 Comparing statistics from one program to another

The statistics generated from one proficiency program are not usually comparable to those from another proficiency testing program. Only very general comparisons may be possible. The reason statistics from one program may not be compared to another is due to the range of variables that differ from one proficiency program to another.

These variables include:

• Type of material selected,

• The number of participants,

• Experience of participants,

• Test methodology variations,

• Equipment used,

• Test methods used,

• Experience of supervisors,

• Range of organisations involved.

• Program design and the statistics employed.

The program outcome represents a ‘snap shot’ of the competency within the industry and hence provides an overview of the industry. However, it should be noted that more participants involved in a given program, then the more representative the overview.

5.9.3 Measurement uncertainty

The statistics detailed in this program do not replace the need for laboratories to separately calculated measurement uncertainties associated with each test when required by the client or NATA. The proficiency program does give information useful for calculating the MU and bench marking the MU calculated.

5.9.4 Metrological traceability

The assigned median value used in this proficiency testing program is derived from participant performance and is not metrologically traceable.

Page 36: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 36 of 45

5.10 Non-statistical

One of the issues faced by proficiency testing providers is what to do with an incorrect result even if its z-score is satisfactory. In many cases, they cannot be detected but still can have a significant impact on the statistics calculated. This can cause biased (or unfair) outcomes for other participants.

To limit the effect that erroneous results may have on a program, additional information is requested to allow the main results to be recalculated. In some cases, results shown to be erroneous may be rejected for inclusion in the program. If the result does not add any statistical bias, it is left in the program.

The result, however, is incorrect even though it may have a satisfactory z-score. To highlight that the participant needs to investigate, ‘non-statistical’, erroneous results.

This may also be applied to non-compliance to program requirements, e.g. incorrect reporting of results etc. or incorrect partial calculations/data.

Page 37: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 37 of 45

6. Summary of Participants Results

Code V9 N2 B3 M8 L3 W4 F3

Moisture-Before compaction - W1 (%) 11.9 12.1 11.9 12.0 12.0 11.7 11.9

Moisture Content Variation - Wv (%) 0.1 -0.1 0.1 0.0 0.0 0.3 0.1

Compaction (Manual or Auto) Manual Auto Manual Manual Manual Manual Manual

Compaction Method - Standard (Y/N) Yes Yes Yes Yes Yes Yes Yes

No. blows per layer 48/48/48 53/53/53 55/55/55 53/53/53 25/22/25 55/55/55 53/53/53

Dry Density g/cm3

2.086 2.072 2.079 2.076 2.064 2.081 2.067

Density Ratio (LDR) % 101.0 100.3 100.7 100 100.0 100.7 100.1

Moisture Ratio (LMR) % 99.2 100.8 99.2 100 100.0 97.1 99.2

BR @ 2.5 mm (%) 64.0 44.1 70 40 45.0 30.4 45.0

BR @ 5.0 mm (%) 77.5 59.0 90 60 60.3 37.3 60.0

CBR (%) 77.5 59.0 90 60 60.3 37.3 60.0

Correction (mm) 0.1 1.1 0.8 62.17 1.2 2.9 0.5

Swell (%) -0.1 -0.0 -0.1 0.0 -0.01 -0.1 -0.1

Moisture ww NA 13.5 NA NA 12.2 12.2 12.3

Moisture w30 11.8 12.4 12.1 12.5 12.8 11.8 11.6

Moisture wr 11.6 11.8 12.0 11.6 11.4 11.3 12.1

Date last calibrated 24/09/2018 25/09/2019 18/11/2019 22/03/2018 21/8/2017 23/08/2018 23/01/2019

Calibrated range 0-50 kN 0.05-50kN 0-100 kN 0-50 kN 0-50kN 0-50kN 0.1-50 kN

Load cell (C) or ring (R) C C C C C C C

Calibration Class (AA, A, B, C etc) A A A A B,A A AA

Hand driven (H) or motorised (M) M M M H M M M

Rate of penetration (mm/min)

Average 0.95 Average 1mm/m NA 1.03 NA 1mm/min

Lowest 0.89 NA 0.98mm/m NA 0.97 NA NA

Highest 0.98 NA 1.01mm/m NA 1.08 NA NA

Condition of material Bag intact MC 5.9%Sealed Dry of OMC Good OK Moist 6.1% <OMC Good Moist

Seating load applied (N) 249 250 250 250 250 kN Yes 45

Seating load set to zero (Y/N) Yes Yes Yes Yes Yes Yes Yes

LL determined by clause(5d)1, 2 or 3 3 3 Visual 3 3 3 Visual Tactile

LL value used lowsands and granular Material6 >50% Low(LL<35% No NA

Period Cured (hours) 96 16 24 24 24 2 550.7

Graph computer or hand (C/H) C NA C C C NA C

Loads in ( N or kN ) N kN N N kN kN N

0 0 0.012 0 0 0 0 0

0.5 1444 0.471 473 648 0.422 0.078 642

1 3123 0.972 1544 1540 0.709 0.195 1417

1.5 4842 1.676 2916 2612 1.362 0.397 2398

2 6529 2.515 4531 3948 2.354 0.682 3492

2.5 8141 3.459 6344 5254 3.520 1.058 4644

3 9683 4.474 8061 6592 4.760 1.523 5879

3.5 11153 5.555 9881 8180 5.970 NA 7135

4 12508 6.725 11702 9512 7.153 2.789 8379

4.5 13842 7.951 13453 10820 8.315 NA 9605

5 15096 9.079 15204 12310 9.420 4.426 10764

6 17472 11.376 18251 14958 11.571 NA 12981

7.5 20595 14.733 22868 18610 15.097 9.410 15997

8 21585 15.750 24323 19766 16.425 NA 16953

10 25453 19.792 29960 23806 19.810 14.587 NA

12.5 30037 24.245 36363 28748 23.571 19.096 NA

Code V9 N2 B3 M8 L3 W4 F3

Number 1 2 3 4 5 6 7

6 Participants Test Results - Sample A

Page 38: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 38 of 45

Code J2 Y8 W9 M3 Y7 F4 M9

Moisture-Before compaction - W1 (%) 12.1 11.9 12.0 11.8 12.0 5.8 11.6

Moisture Content Variation - Wv (%) 0.1 -0.1 0 0.2 0> 6.2 0.4

Compaction (Manual or Auto) Manual Manaul Manual Manual Manual Manual Manual

Compaction Method - Standard (Y/N) Y Standard STD Y Y Y Y

No. blows per layer 36/45/46 53/52/50 To Depth/layer 53/53/53 NA NA 53/53/53

Dry Density g/cm3

2.061 2.067 2.062 2.065 2.065 2.062 2.067

Density Ratio (LDR) % 99.8 100.1 99.9 100.0 100 99.9 100.1

Moisture Ratio (LMR) % 100.8 99.2 100.0 98.5 100 99.2 96.7

BR @ 2.5 mm (%) 50 42.4 50 45 20 37.0 35

BR @ 5.0 mm (%) 58.7 57.6 70 60 30 48.0 47

CBR (%) 58.7 57.6 70 60 30 48.0 47

Correction (mm) 0.9 1.6 1.5 2.5 0.5 2.1 1.2

Swell (%) 0 0.0 0.0 -0.2 0.0 -0.1 0.0

Moisture ww 13.1 12.8 12.0 12.7 15.8 12.8 12.6

Moisture w30 13.2 12.1 11.9 12.1 11.8 12.3 12.0

Moisture wr 11.9 11.7 11.4 12.0 11.8 11.8 11.7

Date last calibrated 20/12/2017 20/03/2019 23/07/2019 14/03/2019 9/12/2019 24/10/2017 24/09/2018

Calibrated range 0-50kN 0-50kN 50kN 0-50kN 0-50kN 50kN 0-50kN

Load cell (C) or ring (R) C R NA C C C C

Calibration Class (AA, A, B, C etc) A,B,C B,A A NA A A A

Hand driven (H) or motorised (M) M M M M M M M

Rate of penetration (mm/min)

Average NA 1.00 29.07per/.5 1.001 1mm 1mm/m 1.0

Lowest 0.84 0.95 25.92per/.5 1.001 0.99mm NA 0.9

Highest 1.00 1.09 32.34per/.5 1.004 1.02mm NA 1.2

Condition of material Goodundamage plastic bag 5.9%Good (moist) NA Good intact, moist Moist

Seating load applied (N) 0.05kN 0.045kN 0.250kN 0.253 153 270 250

Seating load set to zero (Y/N) Y No Y No Yes Y Y

LL determined by clause(5d)1, 2 or 3 3 1 3 Visual 3 3 3

LL value used Low 25 Sand - Gravel <35 sand/gravel Sand&gravel NA

Period Cured (hours) 50 48 24 2 2 2 2

Graph computer or hand (C/H) C C NA C C C C

Loads in ( N or kN ) N N N kN kN N kN

0 0 0 0 0.253 0.0 0.0 0.00

0.5 251 133 485 0.687 0.356 115 0.29

1 1009 433 891 1.187 0.772 325 0.62

1.5 1880 901 1465 1.803 1.294 645 1.10

2 3091 1509 2229 2.558 1.854 1046 1.72

2.5 4184 2265 3314 3.527 2.382 1571 2.45

3 5396 3195 4483 4.718 2.952 2218 3.28

3.5 6559 4210 5734 NA 3.480 2961 4.17

4 7830 5303 7021 7.136 4.066 3792 5.09

4.5 8836 6436 8410 NA 4.519 4698 6.04

5 9853 7570 9649 9.499 5.212 5590 7.01

6 11861 9786 12318 NA 6.244 7489 8.89

7.5 14995 13645 16161 15.289 7.886 10240 11.60

8 15822 14570 17288 NA 8.344 11145 12.51

10 19204 18219 21409 NA 10.386 14631 15.98

12.5 22391 22153 26327 NA 12.722 18395 19.64

Code J2 Y8 W9 M3 Y7 F4 M9

Number 8 9 10 11 12 13 14

6 Participants Test Results - Sample A

Page 39: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 39 of 45

Code A8 Y5 C8 K3 T5 Z8 V3

Moisture-Before compaction - W1 (%) 11.9 6.5 11.9 11.9 12.1 12.0 12.2

Moisture Content Variation - Wv (%) 0.1 5.5 98.8 0.1 0.1 0.0 0.2

Compaction (Manual or Auto) Manual Manual Manual Manual Manual Auto Manual

Compaction Method - Standard (Y/N) Y Y Y Y standard Y Yes

No. blows per layer 35/35/35 25/28/30 60/60/60 40/42/38 55/55/55 50/50/50 53/53/40

Dry Density g/cm3

2.069 2.070 2.067 2.076 2.063 2.063 2.061

Density Ratio (LDR) % 100.2 100.0 100.1 100.5 99.9 99.9 99.8

Moisture Ratio (LMR) % 99.2 100.0 90.0 99.6 100.8 100.0 101.7

BR @ 2.5 mm (%) 56.8 41.3 35 8.8 40 34.8 25

BR @ 5.0 mm (%) 71.5 48.7 50 24.6 60 46.7 30

CBR (%) 71.5 48.7 50 24.6 60 46.7 30

Correction (mm) 0.5 0.7 0.2 zero 0 2.5 2.4

Swell (%) 0.0 0.0 4.2 -0.2 0 0.1 0.0

Moisture ww 13.1 12.0 11.9 12.8 12.1 12.4 NA

Moisture w30 12.2 11.8 11.7 12.1 12.7 12.4 12.3

Moisture wr 11.9 12.6 11.4 11.4 11.7 12.4 12.5

Date last calibrated 13/06/2019 1/07/2019 15/11/2018 24/1/2018 11/09/2018 20/09/2018 22/08/2018

Calibrated range 0-50kN 0-40kN 1-50kN 0-50kN 0-50kN 1-50kN 0-50kN

Load cell (C) or ring (R) C C R R C C C

Calibration Class (AA, A, B, C etc) A A A A A A NA

Hand driven (H) or motorised (M) M M M M M M M

Rate of penetration (mm/min)

Average 1mm per minute NA 1.005 0.6 31.7 sec 1 1.05

Lowest NA 1.0mm NA 0.7 30.02sec 1 1.05

Highest NA NA 1.033 0.8 33.5 sec 1 1.01

Condition of material good Good Intact-Moist Bag undamage Good ok dry

Seating load applied (N) 0.045 250 95div-1.00kN 50 NA 50 50

Seating load set to zero (Y/N) Yes Y N Y Y Y Yes

LL determined by clause(5d)1, 2 or 3 3 3 1289.3.9.1 estimate 1 1 3

LL value used low< 35.0 <35 20.0%granular material-2HrsLow <35% NA low <35%

Period Cured (hours) 55 48 24 2 48 2 72

Graph computer or hand (C/H) H C C C NA H C

Loads in ( N or kN ) kN N kN N kN N N

0 0.00 0.0 1.00 156 0.00 0 0

0.5 0.28 632 1.68 179 0.34 16 303

1 1.30 1269 2.26 294 0.87 24 502

1.5 2.18 2036 3.15 477 1.73 40 727

2 4.45 2956 4.04 752 2.81 59 1008

2.5 6.05 3930 5.05 1165 4.16 78 1333

3 7.55 4999 6.09 1716 5.46 101 1692

3.5 9.00 6075 7.30 2358 6.86 173 2081

4 10.38 7188 8.40 3137 8.31 263 2515

4.5 11.70 8269 9.86 3963 9.43 360 2957

5 12.94 9300 11.02 4880 10.63 458 3435

6 15.50 11331 13.18 6899 12.25 649 4510

7.5 18.70 14200 16.25 10110 14.81 925 6312

8 19.80 15170 17.10 11210 15.23 1014 6966

10 23.30 18690 20.53 15293 17.75 1329 9632

12.5 27.70 22750 23.59 20109 20.01 1696 12646

Code A8 Y5 C8 K3 T5 Z8 V3

Number 15 16 17 18 19 20 21

6 Participants Test Results - Sample A

Page 40: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 40 of 45

Code F9 L8 U5 R3 N5 U4 X2

Moisture-Before compaction - W1 (%) 12.3 12.3 12 11.9 11.9 12.0 12.0

Moisture Content Variation - Wv (%) 0.3 0.3 0.0 0.1 0.1 0.0 0.0

Compaction (Manual or Auto) Manual Manual Manual Manual Manual Manual Manual

Compaction Method - Standard (Y/N) Y Y Yes Y STD Y Y Yes

No. blows per layer 35/35/30 35/35/30 NA 25/25/25 53/53/53 53/53/53 53/41/41

Dry Density g/cm3

2.060 2.060 2.065 2.067 2.063 2.055 2.085

Density Ratio (LDR) % 99.8 99.8 100.0 100.1 99.9 99.5 101.0

Moisture Ratio (LMR) % 102.5 102.5 100.0 99.2 99.2 100.0 100.0

BR @ 2.5 mm (%) 61.1 64.5 59.1 58.7 42.4 41.3 44.1

BR @ 5.0 mm (%) 70.4 68.6 69.4 67.5 55.8 53.5 47.0

CBR (%) 70.4 68.6 69.4 67.5 55.8 53.5 47.0

Correction (mm) 1.2 0.7 1.2 0.5 2.6 3.1 1.00

Swell (%) 0.4 0.8 -0.21 0 0.0 0.0 0.0

Moisture ww 13.3 13.1 13.3 12.8 12.1 12.4 12.4

Moisture w30 12.2 12.2 13.2 12.3 12.4 12.5 12.4

Moisture wr 11.2 11.6 12.9 12.1 12.0 12.0 12.1

Date last calibrated 19/09/2019 23/08/2018 3/05/2018 6/05/2019 22/07/2019 22/07/2019 20/05/19

Calibrated range 50kN 50kN 50kN 0-50kN 50kN 50kN 0-50kN

Load cell (C) or ring (R) C C C C C C R

Calibration Class (AA, A, B, C etc) A A A A C C C

Hand driven (H) or motorised (M) M M M M M M M

Rate of penetration (mm/min)

Average 1.1 1.1mm/min 1.0mm/min 1mm/min 1.022 1.022 0.89

Lowest NA NA NA NA 1.000 1.000 0.82

Highest NA NA NA NA 1.133 1.133 0.96

Condition of material Bag Not SealedBag not sealed Good Moist Good GoodBag sealed-sample6% dry of OMC

Seating load applied (N) 250 250 50kN .250 0.01 0.01 250

Seating load set to zero (Y/N) Y Y Yes Y Y Y No

LL determined by clause(5d)1, 2 or 3 3 3 Visual/tactile Assesment 3 3 3 1

LL value used N/A N/A N/A 26 <12% <12% 29%

Period Cured (hours) 24 24 48 48 2.5 2.5 24

Graph computer or hand (C/H) NA NA NA NA H H H

Loads in ( N or kN ) N N N N kN kN N

0 0 0.0 0 0 0.00 0.00 250

0.5 235 198 0.215 .590 0.02 0.01 599

1 801 1256 0.735 1.772 0.06 0.02 1077

1.5 1756 3295 1.520 3425 0.19 0.06 1780

2 2910 5103 2.701 5224 0.43 0.19 2624

2.5 4233 6536 4.076 6750 0.84 0.42 3617

3 5910 7966 5.670 8132 1.46 0.87 4729

3.5 7144 9107 7.141 9314 2.26 1.46 5819

4 8865 10156 8.647 10379 3.18 2.21 7003

4.5 10033 11288 10.194 11398 4.30 3.26 8204

5 11455 12189 11.246 12355 5.38 4.15 9300

6 13148 13245 13.154 14030 7.65 6.34 12223

7.5 15234 16856 15.011 16334 10.84 9.46 15102

8 16207 17254 15.522 17096 11.85 10.46 15996

10 18233 19587 17.060 19899 15.56 14.20 19377

12.5 20123 21899 19.111 23331 19.71 18.64 23298

Code F9 L8 U5 R3 N5 U4 X2

Number 22 23 24 25 26 27 28

6 Participants Test Results - Sample A

Page 41: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 41 of 45

Code P9 B5

Moisture-Before compaction - W1 (%) 12.1

Moisture Content Variation - Wv (%) 0.1

Compaction (Manual or Auto) Manual

Compaction Method - Standard (Y/N) Y

No. blows per layer NA

Dry Density g/cm3

2.062

Density Ratio (LDR) % 100

Moisture Ratio (LMR) % 101

BR @ 2.5 mm (%) 20.9

BR @ 5.0 mm (%) 33.2

CBR (%) 33.2

Correction (mm) 0

Swell (%) -0.1

Moisture ww 12.8

Moisture w30 12.0

Moisture wr NA

Date last calibrated 12/04/2018

Calibrated range 0-50kN

Load cell (C) or ring (R) C

Calibration Class (AA, A, B, C etc) C

Hand driven (H) or motorised (M) M

Rate of penetration (mm/min)

Average 1mm/min

Lowest NA

Highest NA

Condition of material Good

Seating load applied (N) No

Seating load set to zero (Y/N) NA

LL determined by clause(5d)1, 2 or 3 Assumed <35

LL value used <35

Period Cured (hours) 24

Graph computer or hand (C/H) NA

Loads in ( N or kN ) N

0 NA

0.5 588

1 1045

1.5 1552

2 2144

2.5 2765

3 3488

3.5 4218

4 4765

4.5 5778

5 6568

6 8361

7.5 11085

8 11925

10 15219

12.5 18270

Code P9 B5 0 0 0 0 0

Number 29 30 31 32 33 34 35

6 Participants Test Results - Sample A

Page 42: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 42 of 45

Appendix A: Instructions for testers

Page 43: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 43 of 45

Page 44: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 44 of 45

Appendix B: Results Log

Page 45: CBR– 2019 (91) Proficiency Testing Program Report

CBR Proficiency Testing Program – 2019 (91)

Copyright: LabSmart Services Issue – 13 May 2020 Page 45 of 45