Andy Buffler Department of Physics

University of Cape Town

South Africa

Measuring for knowledge: Teaching the ISO-GUM in the

modern introductory physics laboratory

Test and Measurement 2015

12 October 2015

Why does practical work form part of most introductory physics courses?

… driven by a philosophy of physics which values the coordination of theory and evidence in physics.

Quaestiones quaedam

philosophicae

Physics 1000W University of Cambridge

1661

Amicus Plato, Amicus Aristoteles, sed magis amica veritas

American physicist and Nobel Prize winner in physics,

Richard Feynman:

“Now I’m going to discuss how we would look for a new law. In

general, we look for a new law by the following process. First, we

guess it, then we compute the consequences of the guess, to see

what, if this is right, if this law we guess is right, to see what it would

imply and then we compare the computation results to nature or we

say compare to experiment or experience, compare it directly with

observations to see if it works. If it disagrees with experiment, it’s

wrong. In that simple statement is the key to science. It doesn’t make

any difference how beautiful your guess is, it doesn’t matter how smart

you are who made the guess, or what his name is… If it disagrees with

experiment, it’s wrong. That’s all there is to it.”

“ There has been much discussion in the past few years on the deficiencies and problems of the introductory physics laboratory. ”

J.C. Menzie, “The lost arts of experimental investigation,” American Journal of Physics, 1970.

Goals of the Introductory Physics Laboratory as proposed by a special commission of the American Association of Physics Teachers

I. Development of the appreciation for the art of experimentation

II. Teaching and learning of experimental and analytical skills

III. Provide opportunities for conceptual learning

IV. Illustration of the basis of knowledge in physics

V. Development of collaborative learning skills

American Journal of Physics, June 1998

“Laboratory work is essential in the study of physics.”

What is the unifying theme?

… but make sure that you have

a proper error analysis. Find g

or else …

Amalgam of purposes for laboratory work can lead to a confusing curriculum

… thanks, Saalih, for the idea

“ In the laboratory the student is introduced at once to the difficult subject of measurement, required to make immediate use of such unfamiliar instruments as the diagonal scales, the vernier calliper, and the balance sensitive to a centigram; to report his results in terms of the metric system, to discuss errors, sources of error, percentages of error, averages and probabilities; to deduce laws, many of which he knew before, from data that cannot be made to prove anything …”

… noted at the July 1905 convention of the U.S. Department of Science Instructors.

Our students don’t seem to be learning much in

the lab …

… look at the state of our equipment …

… we should have

computer-based labs !

Modernise equipment ?

There has been much recent effort in designing and evaluating new, innovative laboratory curricula.

… laboratory tasks framed in the form of authentic problems that require experimental investigation for their resolution … provides a purpose for measurement and data analysis. … concept-based laboratory activities with components structured around increasingly complex concepts of measurement.

Learning about measurement and uncertainty features in most introductory physics laboratory courses (both ‘traditional’ and ‘innovative’ curricula) … … and most of us would argue that this is appropriate … … however, there is a growing body of evidence that is suggesting that very few students develop their understanding of the nature of scientific measurement and uncertainty in their introductory lab course …

THE TRUTH IS OUT THERE

… it has been consistently found that about 80% of students from all groups studied arrive at university with a view of measurement (in the physics lab) which may be characterised as ‘point-like’…

… for example, measurements should be repeated so that the recurring value may be identified … … or … so that an average may be calculated.

Studies into students’ understanding of measurement and uncertainty.

A single reading has the potential of being the true value.

See: Buffler et al., International Journal of Science Education, 23, 1137 (2001).

For example, when students are asked to compare two sets of data and their averages …

Data set A Data set B (mm) (mm)

440 444 438 432 433 426 423 433 431 440

433 435 Average:

… or intuitive notions of ‘closeness’ of the averages …

… the data in the sets are examined …

… are used to come to a decision about agreement of the two sets of measurements.

… while students may execute the technical procedures of data analysis adequately, they often appear to lack the foundational basis from which the computational aspects are derived.

… do not display a conceptual understanding of measurement uncertainty.

… for example, typically more than 60% of students resort to subjective notions of acceptable ‘closeness’ of data points, or means, when comparing data sets.

Testing of students’ understanding after their laboratory course shows that …

This finding is independent of the laboratory curriculum, no matter how innovative the design of the practical tasks.

What is the origin of the exact ‘point-like’ view held by introductory students?

‘Approximate’ and ‘exact’ are often

used loosely when measuring, and are related to the notion of ‘good enough for

the purpose.’

Scientific measurement viewed as the

pursuit of exact results.

Everyday domain Scientific domain

Science uncovers the truth

Measurement is exact

At least some hands-on

labwork at school

Teacher did

some

demonstrations

No labwork

experience

Incoming UCT Science students

Students and labwork

Why do we teach data analysis in the lab in the

way that we do? I don’t know. Why …

is there a problem?

Some problems with the traditional approach to teaching measurement and uncertainty in the

introductory laboratory

… traditional data analysis is based upon the ‘frequentist’ approach … offers no formal logical link between data and the measurand.

… ‘systematic’ and ‘random’ errors are estimated through completely different approaches and cannot be combined.

… result is that most introductory lab courses feature an amalgam of rigorous computational methods and arcane rules of thumb when dealing with ‘errors.’

… the theoretical framework is only really valid for large data sets anyway.

Our experiment did not agree

with theory due to human error.

The lecturer’s readings are correct but ours have lots of errors.

I feel really uncertain about getting good

marks.

So 9.80 is 3 significant figures ?

… thanks, Saalih, for the idea

Inconsistencies were also recognised at the technical / research level … … where there was a need to standardise metrology and methods of reporting measurement and uncertainty …

The need for a consistent international language for evaluating and communicating measurement results prompted (in 1993) the ISO (International Organisation for Standardization) to publish recommendations for reporting measurements and uncertainties based on the probabilistic interpretation of measurement. All standards bodies have adopted these recommendations for reporting scientific measurements: BIPM: International Bureau of Weights and Measures IUPAC: International Union of Pure and Applied Chemistry IUPAP: International Union of Pure and Applied Physics OIML: International Organization of Legal Metrology NIST: National Institute of Standards and Technology … and NMISA ...

A number of documents currently serve as international standards, including … VIM ( International Vocabulary of Basic and General Terms in Metrology ) GUM ( Guide to the Expression of Uncertainty in Measurement ) … published by the ISO in 1993 and 1995.

The ISO-GUM has become the de facto international standard for evaluating and expressing uncertainty in measurement.

… introduces the concept of the measurement equation

y = f (x 1 … xn )

… a mathematical description of the process that is used for determining the result of a measurement y from various input quantities x i .

… the uncertainties u (x i) associated with the inputs x i

are components of the final (‘combined’) uncertainty u (y ) for the result of the experiment.

… the uncertainty components from random effects and from the corrections applied for systematic effects are treated in exactly the same way.

Evaluation of uncertainties the ISO-GUM way

Type A evaluations of uncertainty … are based on statistical analyses of current data.

Type B evaluations of uncertainty … when a statistical evaluation cannot be made but profound metrological knowledge is available (data from previous measurements, published data, etc.) … commonly obtained by assigning a state-of-knowledge a priori probability distribution to an input quantity.

…

The ISO-GUM does not clearly advocate the use of classical or Bayesian statistics … with the result that familiar procedures of frequentist statistics are often used for Type A evaluations.

… in classical statistics, the value of the measurand is assumed to be an unknown constant, often called the true value, and the measurement data are random variables each with a sampling probability distribution.

… in Bayesian statistics, the measurement data are

constants and the value of the measurand is a random variable … the probability distribution for the value of the measurand is a state of knowledge distribution that describes the degrees of belief about all possible values that can be reasonably attributed to the value of the measurand.

… therefore the ISO-GUM is consistent only when a Bayesian framework is applied and the Type A evaluations are interpreted as parameters of state-of-knowledge probability distributions.

… then Type A and Type B evaluations have a common probabilistic interpretation and they may be combined through the measurement equation.

… the best estimate of the value is given by the expectation value (most often the centre of the posterior probability density function) and the standard uncertainty is given by the square root of the variance, which is related to the second moment of the pdf.

The ISO-GUM offers genuine pedagogical advantages to the teaching of measurement in the laboratory.

… genuine instance where advances at the research/technical level have led to opportunities for improvements at the introductory/teaching level.

… the case has been made that the probabilistic approach provides a systematic teaching framework at the first year level and beyond.

Uncertainty is defined in the ISO-GUM as … “… a parameter associated with a measurement result, that characterizes the dispersion of the values that could reasonably be attributed to the measurand.”

A key philosophical feature of ISO-GUM is how it views the nature of measurement and uncertainty.

… the purpose of measurement is to improve knowledge about a particular measurand …

… and uncertainty is attached to this knowledge.

… the ISO-GUM approach leads directly to inferences about the quantity being measured in a natural way, in both cases of single and repeated measurements.

… uncertainties associated with single and multiple measurements may be dealt with in a consistent way.

… Type A and Type B evaluations can be combined into one uncertainty.

… no need to appeal to random and systematic errors.

… the use of a measurement equation and uncertainty budget allow all sources of uncertainty to be considered.

The ISO-GUM offers a logical framework for teaching measurement and uncertainty

… challenge was to unpack the GUM documents into a form that could be used by introductory students.

… result was an interactive student workbook …

Teaching the ISO-GUM way at the introductory level

Introduction to Measurement in the Physics Laboratory: A Probabilistic Approach

Andy Buffler, Saalih Allie, Fred Lubben, Bob Campbell

See: Buffler, et al., The Physics Teacher 46, 539 (2008).

… students work through chapters in small groups during special measurement tutorials.

… the chapters are designed to deal with the key ideas about measurement in both a conceptual and technical way … includes hands-on exercises that require simple equipment.

… measurement tutorials are interspersed with ‘authentic’ practical tasks.

… the workbook has been in circulation since 2006, and was used by 1000 first year students at UCT in 2015.

… available online as a pdf … click from www.andybuffler.net

The GUM-compliant student workbook

… a sample of some key interventions follow …

Common everyday views about scientific measurement are challenged …

Thinking visually about data helps conceptual understanding …

The uncertainty budget helps to think about the measurement …

Evaluating the effectiveness of the workbook based on the ISO-GUM approach

… evaluation based on diagnostic testing of the students both before and after the course … and interviews with individual students.

… has been repeatedly found that between 75% to 85%

of the students develop an appropriate understanding of scientific measurement and uncertainty.

… other courses have not reported levels above 50% … no matter how innovative the practical tasks are.

See: Pillay, et al. European Journal of Physics 29, 647 (2008).

Concluding remarks

Axiom: The introductory physics course needs to include a laboratory component … … which involves measurement-based activities.

… in order for students to learn to make meaningful inferences from measurement-based observations, both the technical and conceptual aspects of measurement and uncertainty need to be explicitly featured in the introductory laboratory curriculum.

… the philosophy and mathematics of the GUM approach offer genuine pedagogical advantages.

… the GUM approach can be introduced in the introductory laboratory.

… there is evidence that the approach improves students’ understanding of the nature of scientific measurement.

Concluding remarks

“ The actual science of logic is conversant at present only with things either certain, impossible, or entirely doubtful, none of which (fortunately) we have to reason on. Therefore the true logic of this world is the calculus of Probabilities, which takes account of the magnitude of the probability which is, or ought to be, in a reasonable man’s mind. ”

James Clerk Maxwell (1862) quoted in E.T. Jaynes

Probability Theory: The Logic of Science (2003).

… with acknowledgment of a long and fruitful collaboration with Saalih Allie (UCT) and Fred Lubben (University of York, UK)