1

SCL Simulator version 1.0 (QS 1684)

Software Manual

1 Introduction

“Guide to the expression of uncertainty in measurement” (GUM) [1] defines the GUM

Uncertainty Framework (GUF) for the evaluation of uncertainties by applying the law of

propagation of uncertainty (LPU). However, there are limitations in the application of GUF.

The Supplement 1 to the GUM (GUM-S1)[2] introduces a Monte Carlo method (MCM) to

implement the propagation of the distribution, and also provides a validation method for the

GUF.

The Standards and Calibration Laboratory (SCL) has developed a simulator software (the SCL

simulator) for evaluation of measurement models in accordance with Supplement 1. With the

SCL simulator, users need only to encode the measurement model using the Visual Basic for

Application (VBA) programming language included in Microsoft Excel and to specify relevant

parameters of the uncertainty components (also called input quantities), such as estimates,

standard uncertainties and probability distribution function (PDF), in table form on an Excel

worksheet.

2 Hardware and Software Requirements

For the tool to perform uncertainty calculation, the following hardware and software are

required:

2.1 Hardware Requirements

Personal computer with Intel CPU

2.2 Software Requirements

(1) MS Windows 7 or later version (32-bit or 64-bit version)

(2) MS Excel 2013 or 2016 version (32-bit or 64-bit version)

2

3. Simulator software and the libraries

The SCL simulator, written in Visual C++ and VBA, is tightly integrated with Microsoft Excel

which serves as front-end user interface. It consists of the following 3 files.

a). scl_simulator_vx.x.xls : The SCL simulator in excel format (x.x is the version number),

written in VBA

b). simulator_64.dll : Dynamic Link Library (DLL) for 64 bit Excel

c). simulator2.dll : DLL for 32 bit Excel

Computational intensive routines that require faster execution speed were developed in Visual

C++ and compiled into Dynamic Link Libraries (DLL) for MS Excel in both 32 bit version and

64 bit version. User should copy all the 3 files into the same folder for execution. The software

will detect the version of Excel being used and load the correct DLL for the simulation. No

installation is required.

4. Enabling the Developer Tab and the Macro in Excel

4.1 Enabling the Developer Tab

Microsoft Excel includes a VBA editor, allowing users to employ a subset of the VBA

programming language to develop software routines to enhance and customize the features of

Excel. The SCL simulator makes use of this built-in VBA programming language to encode the

measurement model and the execution of the simulation. The VBA editor is available in the

Developer tab in Excel.

The Developer tab of MS Excel 2016 can be enabled through the following ways:

a). On the File tab, click the Options button. Choose ‘Customize the Ribbon’ in the popup menu

and check the Developer tab in the right column.

3

Fig. 4-1 Developer tab in ‘Customize the Ribbon’ popup menu

b). Enable the Developer through the Enable content button

Fig. 4-2 Enabling the developer

c). Or using the keyboard shortcut key “ALT+F11”.

4.2 Enabling the Macro

To enable the Macro, click the Macro Settings at the Develop tap (Fig. 4-3), enable the Macros

and select ‘Trust access to the VBA project object model’ at the Trust centre (Fig. 4-4).

Fig. 4-3 Macro settings (Develop tap)

4

Fig. 4-4 Macro settings (Trust Centre)

5. Worksheet in the Simulator

The SCL simulator consists of 4 worksheets namely ‘MODEL’, ‘Sheet1’, ‘Sheet2’ and

‘ Sheet3’. The functions of the worksheets are listed below:

Worksheet Function

Model Input of the measurement model

Sheet1 1). Definition of the probability distribution function (PDF)

of the Input quantities

2). Setting up the MCM simulation mode

3). Reporting the MCM simulation results

Sheet2 Storage of the output distribution results

Sheet3 Storage of the output correlation results

6. Setting up the measurement model

User can enter the measurement model starting from row 22 at the ‘MODEL’ worksheet as

shown below.

5

Measurement model

coding area (from row 22)

Model input button

Fig. 6-1. ‘MODEL’ worksheet for the input of measurement model

The input quantities of the measurement model are represented by an array x() with a maximum

of 100 elements, x(1) to x(100). The output quantities are returned in an array y( ). User should

specify the number of output in y(0) and define the measurement models of the output quantities

from y(1) to y(100).

For example, in GUM-S1 example 9.2, the model is

Y1 = X1 + X2 + X3 + X4 - Eq. (1)

There is only 1 output quantity, so y(0) = 1. The output y(1) is modelled by adding the four

input quantities x(1), x(2), x(3) and x(4) as shown in Fig. 6-1.

The following example is an additive measurement model with two output quantities as

described in example 9.2.4 in GUM Supplement 2.

Y1 = X1 + X3 - Eq. (2)

Y2 = X2 + X3 - Eq. (3)

In the SCL simulator, this measurement model is encoded as below

y(0) = 2 y(1) = x(1) + x(3) y(2) = x(2) + x(3)

After encoding the output quantities and the models, press the ‘Model Input’ Button ( )

6

and select the column of codes as shown in Fig. 6-2. Press ‘ok’ after the selection to load the

model(s) into the simulator module.

Fig. 6-2. Loading the model to the simulator

Note: To use this model input function, user should enable the Microsoft Visual Basic For

Applications Extensibility 5.3 in the VBA editor by selecting Tools menu in the VBA editor,

choose References and enable Microsoft Visual Basic For Applications Extensibility 5.3 as

shown in Fig. 6-3.

Fig. 6-3. Enable Microsoft Visual Basic For Applications Extensibility 5.3

7

7. Setting up the input quantities and the PDF

Worksheet ‘Sheet1’ is the main operation page of the SCL simulator. There are 3 areas for

user input and selection as shown in Fig. 7-1.

(a) Parameters of the input quantities

(c) Selection of the Simulation Mode

(b) Correlation of the input parameters

Fig. 7-1. SCL Simulator user input areas: (a) Parameters of the input quantities, (b) Correlation

of the input parameters and (c) Selection of the simulation mode.

7.1 Parameters of the input quantities

The SCL simulator supports a maximum of 100 input quantities. User can specify the

parameters of the input quantities in a table on the SCL simulator (see input area (a) in Fig. 7-

1). An example is shown in Fig. 7-2. The "PDF" column is used to specify the PDF for the

input quantities. The following 8 types of PDF are supported: CTP - curvilinear trapezoid; E -

exponential; G - Gaussian; R - rectangular; TP - trapezoidal; TR - triangular; T - student-t; U -

8

arc sine. The other columns are used for entering the expectation, standard deviation, semi-

range or other parameters relevant to the particular type of PDF. Details are given in Table 7-1.

The histograms generated by the SCL simulator for x1 to x8 in Fig. 7-2 are shown in Figure

Fig. 7-3.

PDF* valueG, T : std.devR,U,TR : semi-rangeCTP, TP : a

T : DOFCTP, TP : b

TPCTP : d

X1 G 0.000E+00 1.000E+00

X2 T 0.000E+00 1.000E+00 5

X3 R 0.000E+00 1.000E+00

X4 U 0.000E+00 1.000E+00

X5 TR 0.000E+00 1.000E+00

X6 CTP -1.000E+00 1 0.3

X7 TP -1.000E+00 1 0.5

X8 E 1.000E+00

Fig. 7-2. Examples of specifying input quantities.

PDF column 1 column 2 column 3 column 4 Remark

G : Gaussian best estimate standard uncertainty N/A N/A

T : student-t average standard deviation degrees of

freedom

N/A

R : rectangular (upper limit +

lower limit)/2

semi-range =

(upper limit - lower

limit)/2

N/A N/A

U : arc sinc (upper limit +

lower limit)/2

semi-range =

(upper limit - lower

limit)/2

N/A N/A

TR : triangular (upper limit +

lower limit)/2

semi-range =

(upper limit - lower

limit)/2

N/A N/A

CTP : curvilinear

trapezoid

N/A a b d rectangular distribution

with lower limit of a d

and upper limit of b d

TP : trapezoid N/A a b refer to GUM

supplement 1 section

6.4.4

E :exponential best estimate N/A N/A N/A

Table 7-1. Types of PDF supported and their parameters

9

(a) x1 : Gaussian (b) x2 : Student-t

(c) x3 : Rectangular (d) x4 : Arc sine

(e) x5 : Triangular (f) x6 : Curvilinear trapezoid

(g) x7 : trapezoid (h) x8 : exponential

-4 -2 0 2 4 -6 -4 -2 0 2 4 6

-1.5 -1 -0.5 0 0.5 1 1.5 -1.5 -1 -0.5 0 0.5 1 1.5

-1.5 -1 -0.5 0 0.5 1 1.5 -1.5 -1 -0.5 0 0.5 1 1.5

-1.5 -1 -0.5 0 0.5 1 1.5 -2 0 2 4 6

Figure 7-3. PDF of x1 to x8 in Table 7-1 generated by the SCL simulator.

7.2 Correlation of the input quantities

The SCL simulator supports the generation of up to a maximum of 10 input quantities with

multivariate Gaussian distribution.

In reference [2] section 6.4.8.1, the uncertainty matrix of N input quantities is defined as

, … ,,

…

, Eq. (4)

⋮ ⋮ ⋱ ⋮ , , …

10

with u(xi, xj) = u(xj, xi) = u(xi) u(xj) r(xj, xi) and r(xj, xi) is the correlation coefficient between

the two input quantities xi and xj [1]. Eq. (4) can further simply as

… 1 , … , … , 1 … , Eq. (5) ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ ⋱ ⋮ … , , … 1

with the second matrix on the right represents the correlation coefficient matrix of the input

quantities.

In the SCL simulator, the correlation coefficients can be entered in a table at input area (b) in

Fig. 7-1. Table 7-2 shows a correlation table for 4 input quantities x1 to x4. The lower triangular

matrix of the correlation coefficient matrix in Eq. 5 is a mirror image of the upper triangular

matrix about the diagonal, therefore the user is only required to input the upper triangular

matrix of these coefficients and the program will generate the lower triangular matrix (the

darkened cells) for computation.

x1 x2 x3 x4

x1 1

Correlation Coefficient

for x1 and x2

Correlation Coefficient

for x1 and x3

Correlation Coefficient

for x1 and x4

x2 Correlation Coefficient

for x1 and x2 1

Correlation Coefficient

for x2 and x3

Correlation Coefficient

for x2 and x4

x3 Correlation Coefficient

for x1 and x3

Correlation Coefficient

for x2 and x3 1

Correlation Coefficient

for x3 and x4

x4 Correlation Coefficient

for x1 and x4

Correlation Coefficient

for x2 and x4

Correlation Coefficient

for x3 and x4 1

Table 7-2 Correlation Table for 4 input quantities. The darkened cells are copied from the

upper triangular matrix by the program.

Fig. 7-4 shows a zoom-in view with the input parameters and the correlation table putting side-

by-side. The correlation coefficient between x3 and x4 is 0.9 and the other pairs of quantities

are treated as uncorrelated with zero correlation coefficient (the empty cells refer).

11

Fig. 7-4. Correlation coefficient between x3 and x4 is set as 0.9 in the correlation table.

7.3 Setting of the Simulation Mode

GUM-S1 describes a MCM for the propagation of distributions. The idea is to make large

number of draws from the PDF of the input quantities and to derive the output quantities for

each draw. There are two ways to select the number of Monte Carlo trials: (a) a fixed number

of trials defined by the user and (b) an adaptive algorithm. An adaptive algorithm will determine

the number of trials on-the-fly based on the dispersion of the simulated output. The user can

specify the precision of the simulation by selecting the number of significant digits ndig [2] for

the output quantities.

User can select the MCM simulation mode in input area (c) in Fig. 7-1. A zoom-in view of

the fixed sample size setting and adaptive setting are shown in Fig.7-5. For the “fixed sample

size” mode, user can specify the number of MCM trials in the “Sample Size” field. The

allowable range of trials is from 20,000 to 3,000,000. For the “adaptive” mode, the user can

specify the number of significant digits (1 or 2) required for the output quantities. User can also

configure the “Confidence Interval Type” to be “Symmetrical” or “Shortest”. Finally, the

“Coverage Probability” of the confidence interval is selectable between 95% and 99 %.

(a) (b)

Fig. 7-5. (a) Fixed sample size mode with 1,000,000 MCM trials selected ; (b) Adaptive

mode selected with output quantities having 2 significant digits.

12

After the completion of the setup procedure in Section 6 and 7, click the ‘Start Simulation’

button to run the simulation program. The simulation time and the total number of trial are

shown in the same area in Fig. 7-5.

8. Reporting the results

Fig. 8-1 shows the display areas of the MCM simulation results in ‘Sheet1’.

(a) Results of the MCM

(b) Histogram of the PDF for y(1) to y(10)

(c) Correlation of the output

quantities

(d) Contour map

Fig. 8-1 SCL Simulator output areas: (a) Results of the MCM, (b) Histogram of the PDF for

y(1) to y(10), (c) Correlation of the output quantities and (d) Contour map.

8.1 MCM result table and GUF validation

After MCM trials, the estimates, standard uncertainties and the confidence intervals (95 % or

99 %) of the output quantities y(i) (i = 1 to 100) are shown in area (a) in Fig. 8-1. The results

of GUM-S1 Example 9.2.2 are shown in Fig. 8-2.

Fig. 8-2. Result table of GUM-S1 Example 9.2.2

The MCM results are compared with the results obtained from GUM uncertainty framework

(GUF). In adaptive simulation mode with assigned number of significant digits ndig, the

software will check the validity of the GUF according to Section 8.1.3 of GUM-S1[2]. If the

absolute differences of the respective endpoints of the two coverage intervals obtained by the

MCM method and the GUF are within the numerical tolerance, the GUF is validated and the

last column of the result table will show as ‘Yes’.

13

8.2 Visualization of the PDF for y(i)

Area (b) in Fig. 8-1 displays the histogram of the PDF of the first 10 output quantities. An

example of the plot is shown in Fig. 8-3 with the blue line showing the PDF obtained by MCM

and the pink line represents the results by GUF.

Fig. 8-3 Histogram of the PDF for the output quantity y(1).

8.3 Correlated output quantities

Section 8.1 and Section 8.2 demonstrate the capabilities of the SCL Simulator for MCM

computation and GUF validation according to GUM-S1 [2]. In addition, the SCL simulator

also calculates the correlations of the output quantities to provide further insight of the results

to the user (in Area (c) and Area (d) of Fig. 8-1). Fig. 8-4 shows the correlation coefficient table

of the output quantities y(1) and y(2) of Eq. (2) and Eq. (3). The correlation coefficient between

y(1) and y(2) is 0.9. The correlation of y(1) and y(2) can be visualized through the contour map

as shown in Fig. 8-5.

Fig. 8-4 Correlation coefficient table of the output quantities

14

Fig. 8-5 Contour map of the joint PDF for the first two output quantities.

9. Practical example from GUM-S1

This example from section 9.2 of Supplement 1 is used to explain the procedure for validation

of GUF using the SCL simulator. It is an additive model described by Eq. (1) in section 6 of

this manual, where

Y1 = X1 + X2 + X3 + X4

Three cases are considered. In case (i) all input quantities X1 to X4 have Gaussian PDF with

standard uncertainty of 1. In case (ii) X1 to X4 have rectangular PDF with standard uncertainty

of 1. Case (iii) is similar to case (ii) except X4 has standard uncertainty of 10.

Step 1 – Setting up the models

The measurement models for Case (i) to (iii) are encoded as below, with y(1) for case (i), y(2)

for case (ii) and y(3) for case (iii). The total number of output quantities y(0) is 3.

y(0) = 3

y(1) = x(1) + x(2) + x(3) + x(4) 'GUM-S1 Section 9.2.2 case (i)

y(2) = x(5) + x(6) + x(7) + x(8) 'GUM-S1 Section 9.2.3 case (ii)

y(3) = x(9) + x(10) + x(11) + x(12) 'GUM-S1 Section 9.2.4 case (iii)

15

Fig. 9-1. Step 1 setting up the models for case (i) to (iii)

Step 2 – Setting up the input quantities

The parameters for the input quantities are entered into the SCL simulator are shown in Figure

9-2.

Fig. 9-2. Step 2 setting up the input quantities

16

Step 3 – Setting up the simulation mode

The “adaptive” simulation mode with 2 significant digits are selected as shown in Fig. 9-3,

with symmetrical confidence interval of 95 % coverage.

Fig. 9-3. Step 2 setting up the simulation model

Step 4 – Results

After the SCL simulator has executed 1.13 million trials, the results are shown in Fig. 9.-4.

Case (i) and case (ii) are validated but not case (iii). The histogram of the 3 outputs are shown

in Fig. 9-5. It is easy to see why case (iii) is not validated.

Fig. 9-4 Results of the validation of GUF using MCM.

17

(a) y1 for case (i) (b) y2 for case (ii) (c) y3 for case (iii)

-10 -5 0 5 10 -10 -5 0 5 10 -40 -20 0 20 40

Fig. 9-5 Histogram of the 3 outputs

10 Reference

1. Evaluation of measurement data — Guide to the expression of uncertainty in measurement,

JCGM 100:2008

2. Evaluation of measurement data — Supplement 1 to the“Guide to the expression of uncertainty

in measurement” — Propagation of distributions using a Monte Carlo method, JCGM 101:2008

First Edition

3. Evaluation of measurement data — Supplement 2 to the “Guide to the expression of uncertainty

in measurement” – Extension to any number of output quantities, JCGM 102:2011