measurement. measurement “if you can’t measure it, you can’t manage it.” bob donath,...
Post on 19-Dec-2015
240 views
TRANSCRIPT
![Page 1: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/1.jpg)
MEASUREMENT
![Page 2: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/2.jpg)
Measurement
“If you can’t measure it, you can’t manage it.”
Bob Donath, Consultant
![Page 3: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/3.jpg)
Measurement
Selecting measurable phenomena
Developing a set of mapping rules
Applying the mapping ruleto each phenomenon
![Page 4: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/4.jpg)
What is to be measured?
CONCEPTS/CONSTRUCTS/VARIABLES
Operational DefinitionsOperational Definitions- Dimensions- Dimensions- Elements- Elements
ScalesScales
![Page 5: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/5.jpg)
CONCEPT
A GENERALIZED IDEA ABOUT A A GENERALIZED IDEA ABOUT A CLASS OF OBJECTS, CLASS OF OBJECTS,
ATTRIBUTES, OCCURRENCES, ATTRIBUTES, OCCURRENCES, OR PROCESSES.OR PROCESSES.
(e.g., (e.g., Technology, Dynamism, Adoption, Technology, Dynamism, Adoption, Learning)Learning)
![Page 6: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/6.jpg)
CONSTRUCT
An image or idea specifically invented for a given An image or idea specifically invented for a given research and/or theory-building purposeresearch and/or theory-building purpose
Higher level concepts for specialised scientific Higher level concepts for specialised scientific explanatory purposes that are not directly explanatory purposes that are not directly observable and for thinking about and observable and for thinking about and communicating abstractionscommunicating abstractions
Concepts and constructs are used at theoretical levels Concepts and constructs are used at theoretical levels
![Page 7: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/7.jpg)
VARIABLE
Used at empirical levelUsed at empirical level Accept numeral or values for the purpose of Accept numeral or values for the purpose of
testing and measurementtesting and measurement
![Page 8: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/8.jpg)
OPERATIONAL DEFINITION
A definition for a construct stated in terms of A definition for a construct stated in terms of specific criteria for testing or measurementspecific criteria for testing or measurement
Specifies what the researcher must do to Specifies what the researcher must do to measure the concept/construct under measure the concept/construct under
investigationinvestigation
![Page 9: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/9.jpg)
SCALE- Definition
SERIES OF ITEMSSERIES OF ITEMS ARRANGED ACCORDING TO VALUEARRANGED ACCORDING TO VALUE FOR THE PURPOSE OF FOR THE PURPOSE OF
QUANTIFICATIONQUANTIFICATION A CONTINUOUS SPECTRUMA CONTINUOUS SPECTRUM
![Page 10: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/10.jpg)
SCALE PROPERTIES
UNIQUELY CLASSIFIESUNIQUELY CLASSIFIES PRESERVES ORDERPRESERVES ORDER DISTANCE/EQUAL INTERVALSDISTANCE/EQUAL INTERVALS NATURAL ORIGIN (ZERO)NATURAL ORIGIN (ZERO)
![Page 11: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/11.jpg)
Types of Scales
OrdinalOrdinal
IntervalInterval
RatioRatio
NominalNominal
![Page 12: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/12.jpg)
NOMINAL SCALE
![Page 13: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/13.jpg)
Nominal Scales
Mutually exclusive Mutually exclusive and collectively and collectively exhaustive exhaustive categoriescategories
Exhibits the Exhibits the classification classification characteristic onlycharacteristic only
![Page 14: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/14.jpg)
NOMINAL SCALE PROPERTIESUNIQUELY CLASSIFIESUNIQUELY CLASSIFIES Male/FemaleMale/Female Academic/AdminAcademic/Admin Asian/EuropeanAsian/European Strategy typesStrategy types Adopters/Non AdoptersAdopters/Non Adopters
![Page 15: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/15.jpg)
Levels of Measurement
OrdinalOrdinal
IntervalInterval
RatioRatio
NominalNominal ClassificationClassification
![Page 16: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/16.jpg)
ORDINAL SCALE
![Page 17: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/17.jpg)
Ordinal Scales
Characteristics of Characteristics of nominal scale plus nominal scale plus an indication of an indication of orderorder
Implies statement Implies statement of greater than and of greater than and less thanless than
![Page 18: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/18.jpg)
ORDINAL SCALE PROPERTIES
UNIQUELY CLASSIFIESUNIQUELY CLASSIFIES
PRESERVES ORDERPRESERVES ORDER Win, Place & ShowWin, Place & Show Podium/Grid positionsPodium/Grid positions RankingRanking
![Page 19: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/19.jpg)
Levels of Measurement
OrdinalOrdinal
IntervalInterval
RatioRatio
NominalNominal ClassificationClassification
OrderOrderClassificationClassification
![Page 20: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/20.jpg)
INTERVAL SCALE
![Page 21: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/21.jpg)
Interval Scales
Characteristics of Characteristics of nominal and ordinal nominal and ordinal scales plus the scales plus the concept of equality concept of equality of interval. of interval.
Equal distance Equal distance exists between exists between numbersnumbers
![Page 22: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/22.jpg)
INTERVAL SCALE PROPERTIESUNIQUELY CLASSIFIESUNIQUELY CLASSIFIES
PRESERVES ORDERPRESERVES ORDER
EQUAL INTERVALSEQUAL INTERVALS Consumer Price Index (Base 100)Consumer Price Index (Base 100) Fahrenheit TemperatureFahrenheit Temperature Assessment of attitude, beliefs, intentionAssessment of attitude, beliefs, intention
![Page 23: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/23.jpg)
Levels of Measurement
OrdinalOrdinal
intervalinterval
RatioRatio
NominalNominal ClassificationClassification
OrderOrderClassificationClassification
OrderOrderClassificationClassification DistanceDistance
![Page 24: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/24.jpg)
RATIO SCALE
![Page 25: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/25.jpg)
Ratio Scales
Characteristics of Characteristics of previous scales plus an previous scales plus an absolute zero pointabsolute zero point
ExamplesExamples WeightWeight HeightHeight Number of childrenNumber of children
![Page 26: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/26.jpg)
RATIO SCALE PROPERTIES
UNIQUELY CLASSIFIESUNIQUELY CLASSIFIES
PRESERVES ORDERPRESERVES ORDER
EQUAL INTERVALSEQUAL INTERVALS
NATURAL ZERONATURAL ZERO Weight and distanceWeight and distance Age, years of serviceAge, years of service
![Page 27: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/27.jpg)
Levels of Measurement
OrdinalOrdinal
IntervalInterval
RatioRatio
NominalNominal ClassificationClassification
OrderOrderClassificationClassification
OrderOrderClassificationClassification DistanceDistance
Natural OriginNatural Origin
OrderOrderClassificationClassification DistanceDistance
![Page 28: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/28.jpg)
Goodness of Measures
Accurately measuring what Accurately measuring what actually the concept one sets out to actually the concept one sets out to measuremeasure
Should be easy and efficient to useShould be easy and efficient to use
![Page 29: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/29.jpg)
Evaluating Measurement Tools
CriteriaCriteria
Validity
Practicality ReliabilityReliability
![Page 30: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/30.jpg)
Validity
The ability of a scale to measure The ability of a scale to measure what was intended to be what was intended to be
measured.measured.
![Page 31: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/31.jpg)
Validity Determinants
Content
ConstructCriterion
![Page 32: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/32.jpg)
Increasing Content Validity
ContentLiterature Search
Literature Search
Expert Interviews
Expert Interviews
Group Interviews
Group Interviews
Question DatabaseQuestion Database
![Page 33: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/33.jpg)
……Goodness of Measures
ValidityValidity Content Validity Content Validity
Includes Face ValidityIncludes Face ValidityHow well the dimensions and elements How well the dimensions and elements
of a concept have been delineatedof a concept have been delineatedProfessional agreement that a scale Professional agreement that a scale
logically appears to accurately measure logically appears to accurately measure what it is intended to measure what it is intended to measure
![Page 34: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/34.jpg)
Validity Determinants
Content
Construct
![Page 35: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/35.jpg)
Increasing Construct Validity
New measure of trustNew measure of trust
Known measure of trustKnown measure of trust
EmpathyEmpathy
CredibilityCredibility
![Page 36: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/36.jpg)
……Goodness of Measures
Construct Validity – when empirical evidence Construct Validity – when empirical evidence generated by a measure is consistent with the generated by a measure is consistent with the theoretical logic of the concept theoretical logic of the concept
Convergent Validity – the measure should Convergent Validity – the measure should “converge” with other similar measures“converge” with other similar measures
Discriminant Validity – when the measure Discriminant Validity – when the measure has low correlation with measures of has low correlation with measures of dissimilar conceptdissimilar concept
![Page 37: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/37.jpg)
Judging Criterion Validity
RelevanceRelevance
Freedom from biasFreedom from bias
ReliabilityReliability
AvailabilityAvailability
Criterion
![Page 38: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/38.jpg)
Criterion-Related Validity
Concurrent – When the scale discriminates Concurrent – When the scale discriminates individuals who are known to be different; A individuals who are known to be different; A type of criterion validity whereby a new type of criterion validity whereby a new measure correlates with a criterion measure measure correlates with a criterion measure taken at the same time.taken at the same time.
Predictive Validity – Ability to differentiate Predictive Validity – Ability to differentiate among individuals with reference to a future among individuals with reference to a future criterioncriterion
![Page 39: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/39.jpg)
VALIDITY
F A C E O R C O N TE N T
C O N C U R R E N T P R E D IC TIV E
C R ITE R IO N V A L ID ITY C O N S TR U C T V A L ID ITY
V a lid ity
![Page 40: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/40.jpg)
Reliability
The degree to which measures are The degree to which measures are
free from random free from random errorerror and and therefore yield consistent results.therefore yield consistent results.
![Page 41: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/41.jpg)
Reliability Estimates
Stability
InternalConsistency
Equivalence
![Page 42: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/42.jpg)
Reliability Estimates
Stability
InternalConsistency
Equivalence
![Page 43: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/43.jpg)
Stability of MeasuresStability of MeasuresTest-Retest Reliability – the Test-Retest Reliability – the
administering of the same scale or administering of the same scale or measure to the same respondents at two measure to the same respondents at two separate points in timeseparate points in time
Parallel-Form Reliability - when Parallel-Form Reliability - when responses on two comparable sets of responses on two comparable sets of measures tapping the same construct are measures tapping the same construct are highly correlatedhighly correlated
… Reliability
![Page 44: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/44.jpg)
Reliability Estimates
Stability
InternalConsistency Equivalence
![Page 45: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/45.jpg)
Equivalence
Variations at one point in time among Variations at one point in time among observers and samples of itemsobservers and samples of items
![Page 46: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/46.jpg)
Reliability Estimates
Stability
InternalConsistency
Equivalence
![Page 47: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/47.jpg)
… Reliability
Internal Consistency MeasuresInternal Consistency MeasuresInteritem Consistency Reliability - Interitem Consistency Reliability -
where items that are independent where items that are independent measures of the same concept correlate measures of the same concept correlate with one another.with one another.
Split-Half Reliability – reflects the Split-Half Reliability – reflects the correlations between two halves of an correlations between two halves of an instrument instrument
![Page 48: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/48.jpg)
RELIABILITY
TE S T R E TE S T
S TA B IL ITY
E Q U IV A L E N T F O R M S S P L ITTIN G H A L V E S
IN TE R N A L C O N S IS TE N C Y
R E L IA B IL ITY
![Page 49: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/49.jpg)
Understanding Validity and Reliability
![Page 50: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/50.jpg)
Practicality
Economy InterpretabilityConvenience
![Page 51: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/51.jpg)
SENSITIVITY
A measurement instrument’s ability to A measurement instrument’s ability to accurately measure variability in accurately measure variability in stimuli or responses.stimuli or responses.
![Page 52: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/52.jpg)
Rating ScalesUsed to elicit responses with regard to the object, event, or person studies:
Dichotomous scaleDichotomous scale
Category scaleCategory scale
Likert scaleLikert scale
Numerical scaleNumerical scale
Semantic differential scaleSemantic differential scale
![Page 53: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/53.jpg)
…..Rating Scales
Itemised rating scaleItemised rating scale
Fixed or constant sum rating scaleFixed or constant sum rating scale
Stapel scaleStapel scale
Graphic rating scaleGraphic rating scale
Consensus scaleConsensus scale
![Page 54: MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath, Consultant](https://reader036.vdocuments.site/reader036/viewer/2022062313/56649d385503460f94a1176b/html5/thumbnails/54.jpg)
Ranking ScalesTo tap preferences between two or more objects or items
Paired ComparisonPaired Comparison
Forced ChoiceForced Choice
ComparativeComparative