![Page 1: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/1.jpg)
Chapter 4Chapter 4
Calculating and Calculating and Evaluating ValidityEvaluating Validity
![Page 2: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/2.jpg)
When We Have Validity . . . When We Have Validity . . .
• We have an acceptably accurate We have an acceptably accurate measurement.measurement.
• We are measuring what we intend to We are measuring what we intend to measure.measure.
• We are interpreting and applying the We are interpreting and applying the measurement appropriately.measurement appropriately.
![Page 3: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/3.jpg)
Methods for Evaluating Validity Methods for Evaluating Validity and Their Calculationsand Their Calculations
![Page 4: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/4.jpg)
Population and SamplePopulation and Sample
• Population: Population: – The designated group being measured The designated group being measured
• Sample: Sample: – A representative subgroup of the populationA representative subgroup of the population
• Inferential statistics:Inferential statistics:– Generalization about a population based on Generalization about a population based on
what is learned about the samplewhat is learned about the sample
![Page 5: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/5.jpg)
Using t-tests to Compare Using t-tests to Compare Sample MeansSample Means
• Purpose of t-tests: Purpose of t-tests: – Compare two sample meansCompare two sample means– Determine if a sample represents a Determine if a sample represents a
populationpopulation
• Types:Types:– Paired vs. independentPaired vs. independent– One-tailed vs. two-tailedOne-tailed vs. two-tailed
![Page 6: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/6.jpg)
Results of a t-testResults of a t-testfor the VOfor the VO22max-12-minute run examplemax-12-minute run example
![Page 7: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/7.jpg)
Interpreting t-testsInterpreting t-tests
• p value: p value: – The probability that the difference between The probability that the difference between
two means is a coincidence of random two means is a coincidence of random samplingsampling
• Sometimes a t-test comparison of means Sometimes a t-test comparison of means doesn’t give us everything we need to doesn’t give us everything we need to know; we need to compare individual know; we need to compare individual differences in scores.differences in scores.
![Page 8: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/8.jpg)
CorrelationsCorrelations
• Defined: Measurement of the strength of Defined: Measurement of the strength of the relationship between two variables.the relationship between two variables.
• Pearson Product-Moment Correlation:Pearson Product-Moment Correlation:– Compares individual differences between two Compares individual differences between two
methods of measurementmethods of measurement
![Page 9: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/9.jpg)
Results of a PearsonResults of a PearsonProduct-Moment CorrelationProduct-Moment Correlation
for the VOfor the VO22max-12-minute run examplemax-12-minute run example
![Page 10: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/10.jpg)
Methods for Evaluating a Methods for Evaluating a CorrelationCorrelation
• r value (or correlation r value (or correlation coefficient)coefficient)
• TrendlineTrendline
• RR2 2 (or coefficient of (or coefficient of determination)determination)
![Page 11: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/11.jpg)
Slope of a TrendlineSlope of a Trendline
• Defined:Defined:– The change in The change in yy value per unit change in value per unit change in xx
value.value.
yy value = the slope value = the slope ** the the xx value + the value + the yy intercept (point where intercept (point where yy = 0) = 0)
OrOr
y = mx + by = mx + b
![Page 12: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/12.jpg)
Scatter Plot Showing TrendlineScatter Plot Showing Trendline
![Page 13: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/13.jpg)
Bland-Altman AnalysisBland-Altman Analysis
A simple approach for evaluating A simple approach for evaluating criterion validitycriterion validity——through error through error analysis: analysis:
• Data taken from an unusually diverse Data taken from an unusually diverse group of subjects will show an artificially group of subjects will show an artificially high correlation.high correlation.
• Analysis focuses attention on error Analysis focuses attention on error scores.scores.
![Page 14: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/14.jpg)
Spreadsheet for a Spreadsheet for a Bland-Altman AnalysisBland-Altman Analysis
![Page 15: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/15.jpg)
Bland-Altman AnalysisBland-Altman Analysis
![Page 16: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/16.jpg)
Evaluating Validity of Ranked DataEvaluating Validity of Ranked Data
• Ordinal numbers:Ordinal numbers:– Ranked numbers that give a place in Ranked numbers that give a place in
line but no information about distances line but no information about distances between numbers; place holders.between numbers; place holders.
• Interval numbers:Interval numbers:– Numbers that are separated by equal Numbers that are separated by equal
intervals; scalar numbers.intervals; scalar numbers.
![Page 17: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/17.jpg)
Spearman’s rhoSpearman’s rho
• A special correlation used in cases A special correlation used in cases where one variable is an ordinal where one variable is an ordinal number number
• Calculating Spearman’s rho:Calculating Spearman’s rho:– Excel doesn’t offer the calculationExcel doesn’t offer the calculation– Some websites can do it; for example, Some websites can do it; for example,
www.wessa.net/rankcorr.waspwww.wessa.net/rankcorr.wasp
![Page 18: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/18.jpg)
Evaluating the Validity Evaluating the Validity of Criterion-Referenced Testsof Criterion-Referenced Tests
• Validity ratio: The ratio of number of Validity ratio: The ratio of number of scores classified correctly to the total scores classified correctly to the total number of scores.number of scores.
Validity ratio = (CR+NCR)/(CR+CW+NCR)Validity ratio = (CR+NCR)/(CR+CW+NCR)where CR = competent and classified correctlywhere CR = competent and classified correctly
NCW = not competent and wrongly classifiedNCW = not competent and wrongly classified CW = competent but wrongly classifiedCW = competent but wrongly classified
and NCR = not competent and classified and NCR = not competent and classified correctlycorrectly
![Page 19: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/19.jpg)
Example Diagram for Evaluating the Example Diagram for Evaluating the Validity of a Criterion-Referenced Validity of a Criterion-Referenced
MeasurementMeasurement
![Page 20: Chapter 4 Calculating and Evaluating Validity. When We Have Validity... We have an acceptably accurate measurement.We have an acceptably accurate measurement](https://reader030.vdocuments.site/reader030/viewer/2022012910/56649d925503460f94a78c6d/html5/thumbnails/20.jpg)
Your ViewpointYour Viewpoint
• After reading the chapter, go back and read After reading the chapter, go back and read the chapter-opening quote from Henry Clay: the chapter-opening quote from Henry Clay:
““Statistics are no substitute for human Statistics are no substitute for human judgment.”judgment.”
• Think also about the famous quote, “There Think also about the famous quote, “There are three kinds of lies: lies, damned lies, and are three kinds of lies: lies, damned lies, and statistics.”statistics.”
• Do you agree with the sentiments expressed Do you agree with the sentiments expressed by these authors? Why or why not?by these authors? Why or why not?