principles of measurement lunch & learn oct 16, 2013 j tobon & m boyle
TRANSCRIPT
![Page 1: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/1.jpg)
Principles of Measurement
Lunch & Learn Oct 16, 2013 J Tobon & M Boyle
![Page 2: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/2.jpg)
What is Measurement?
• the process of assigning numerals to variables to represent quantities of characteristics according to certain rules.
• Objective – directly observable (e.g., length)• Subjective – inferred from experience (e.g., sensations, perceptions, behaviour)
![Page 3: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/3.jpg)
What are some examples of concepts you
have/would like to measure?
![Page 4: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/4.jpg)
Measurement Steps
1. Identify and define the concept
2. Specify the objective and conditions of measurement
3. Determine if a useful measure exists
4. Decide whether to use existing measure or develop a new measure
Use Develop
![Page 5: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/5.jpg)
Measurement Steps
5. Create a question/item pool and evaluate the relevance/importance of each item using a formal judgment approach and trim item pool
6. Choose an approach to scaling
7. Pre-test instrument with target respondents (interviews, focus groups, pilot data)
8. Conduct reliability and validity studies
![Page 6: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/6.jpg)
Measurement Steps
Reliability and Validity studies
![Page 7: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/7.jpg)
Basic Measurement Concepts
• Development• Evaluation
![Page 8: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/8.jpg)
Basic Concepts…
What is a Concept?
• A mental representation, an abstract idea, the invented name for an attribute that we cannot observe directly but infer from the data of experience (indicators) e.g. SES, I.Q., anger
• Concepts are usually complex and need to be represented by multiple indicators
![Page 9: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/9.jpg)
Basic Concepts…
What are the Objectives of Measurement?
1. To discriminate between individuals
2. To evaluate change within individuals
3. To classify individuals
What are the Conditions of Measurement?
Practical considerations associated with assessment and data collection (cost/burden)
![Page 10: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/10.jpg)
Basic Concepts…
What is a Useful Measure?
1. Adequately represents the concept of interest
2. Satisfies the reliability/validity (empirical) requirements associated with the objectives of measurement
3. Meets practical requirements
![Page 11: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/11.jpg)
Basic Concepts…
What is Scaling?
• The process of ‘scaling’ is an attempt to upgrade measurement – the numerical representation of attributes (e.g., increasing reliable variance or the meaning of score intervals)
![Page 12: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/12.jpg)
Basic Concepts…
What are the Levels of Measurement?
• Nominal, ordinal, interval and ratio
• These levels determine the numerical operations available in an analysis
• Strength and form of association between variables will depend on capturing their underlying nature
![Page 13: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/13.jpg)
Levels of Measurement
NO SCHOOLING SOME PRIMARY COMPLETED PRIMARY SOME SECONDARY SECONDARY COMPLETED SOME COMMUNITY ETC COMPLETED COMMUNITY SOME UNIVERSITY COMPLETED UNIVERSITY
1 2 3 4 5 6 7 8 9
0 3 6101314151617
IntervalOrdinal
0
1
Nominal
![Page 14: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/14.jpg)
Rate your agreement with the following statements
SA A N D SD
I am knowledgeable about the signs and symptoms of student mental health issues
1 2 3 4 5
I am knowledgeable about appropriate actions to take to support student mental health at school
1 2 3 4 5
I am knowledgeable about legislation related to mental health issues (confidentiality, consent to treatment, etc.)
1 2 3 4 5
I am knowledgeable about school system services and resources for helping students with mental health issues
1 2 3 4 5
What is an Additive Scale?•multiple items, same response, added up
Basic Concepts…
![Page 15: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/15.jpg)
Single Item Multiple Items
Signs & Symptoms MH Literacy
![Page 16: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/16.jpg)
Basic Concepts…
What is a Good Additive Scale?
• It will be unidimensional
• It will cover the full range of meaningful values
• It will be scaled to the highest level of measurement
• It will be reliable and valid for its intended use
![Page 17: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/17.jpg)
Basic Concepts…
What is reliability?• the extent to which measures provide
similar values on replication
What are the types of reliability?1. Internal consistency – alpha (item
sampling)2. Test-retest (b/w respondent)3. Inter-observer (b/w observer)
![Page 18: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/18.jpg)
Levels of reliability constrain the magnitude of associations (Spearman Brown Prophecy)
(Predicted) = (Observed) / (Rel_x)(Rel_y)
0.24 rXY = = 0.50
0.60 x 0.80
Observed Predict
ed
Reliabilities of x & y
Importance of Reliability
![Page 19: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/19.jpg)
Basic Concepts…
What is validity?• the extent to which measures
adequately represent a concept and are useful for its intended objective
What are the types of validity?• Old: face, content, criterion, construct • New: construct – test hypotheses based
on theoretical linkages with other measured concepts (nomological net)
![Page 20: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/20.jpg)
Types of Hypotheses
• Differences between groups (discrimination)
• Sensitivity to change (evaluation)
• Accuracy of classification (prediction)
• Correlations with similar measures
Types of hypotheses should align with the objective of measurement
![Page 21: Principles of Measurement Lunch & Learn Oct 16, 2013 J Tobon & M Boyle](https://reader035.vdocuments.site/reader035/viewer/2022070308/551beb57550346af588b63c0/html5/thumbnails/21.jpg)
Summary
• Concepts define the subject matter (boundary) of different scientific disciplines
• Measurement quality (level) defines the sophistication scientific disciplines
• Measurement is a two-step process that works within practical constraints (objectives and assessment conditions) and scientific prerequisites (reliability and validity)