cut points ite - 695. section one n what are cut points?
TRANSCRIPT
![Page 1: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/1.jpg)
Cut Points
ITE - 695
![Page 2: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/2.jpg)
Section OneWhat are Cut Points?
![Page 3: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/3.jpg)
I. IntroductionA. The more critical the issue (task) the more critical
the cut point (example: programming a machine).1. Interpretation of readouts.2. Tolerances in measurement.
B. Assumption: Test has both of these:1. Validity.2. Reliability.
C. Select instrument that best measures action needed (performance vs. explanation).
![Page 4: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/4.jpg)
Validity Reliability
Definition:
The appropriateness, meaningfulness andusefulness of the specific inferences made formtest scores (Standards of .Educational andPsychological Testing, 1985).
Definition:
The degree to which test scores are free from errors inmeasurement (S.E.P.T., 1985).
Types:
Content - domain adequately represented. Construct - degree of ability in subject. Criterion-related - performance on differentdomains.
Types:
Test-Retest - method of estimating reliability over aperiod of time. Internal Consistency - method of estimating reliabilitywithin the test instrument. Equivalent Forms - method of estimating reliability overdifferent forms of test instruments. Interrater reliability - establishing consistency amongdifferent raters.
![Page 5: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/5.jpg)
II Types
A. Normative-Referenced Testing (NTR)
1. Significance
- Accepted reliability & validity
2. Measurement
a. Common Averages:
- mode
- median
- mean
![Page 6: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/6.jpg)
II Types (cont.)
b. Variability:
- range
- quartile deviation
- standard deviation
3. Reliability
- Historical acceptance
![Page 7: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/7.jpg)
II Types (cont.)
B. Criterion-Referenced testing (CRT)
1. Significance
a. Testing
b. Distribution
2. Measurement
a. Judgements
b. Variables
![Page 8: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/8.jpg)
II Types (cont.)
3. Reliability
a. Criterion not based on normal distribution.
b. Data dichotomous, mastery/non-mastery.
![Page 9: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/9.jpg)
NORM REFRENCED TESTING
1. Separate test takers
2. Seek Normal Distribution Curve
![Page 10: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/10.jpg)
NORM REFRENCED TESTING
1. Test items separate test - takers from one another.
2. Normal Distribution Curve.
![Page 11: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/11.jpg)
MEASURES of CENTRAL TENDENCIES MODE MEDIAN MEAN MEASURES of VARIBILITY or SCATTER
– RANGE– DEVIATION (QUARTILE)– DEVIATION (STANDARD)
![Page 12: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/12.jpg)
CRITERION REFERENCED TESTING
1. Test items based on specific objectives.
2. Mastery Curve
![Page 13: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/13.jpg)
Standard normal curve withstandard deviations
SEE HANDOUT
![Page 14: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/14.jpg)
CRITERION REFRENCED TEST
1. Test Compares to Objectives
2. Mastery Distribution
![Page 15: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/15.jpg)
Norm-Reference Testing
GOALS
RELIABILITY
VALIDITY
ADMINISTRATION
STANDARD
MOTIVATION
COMPETITION
INSTRUCTIONALDOMAIN
Criterion Referenced Testing
Test Achievement
Usually High
Instruction Dependent
Standard
Averages-Based
Avoidance of Failure
Student to Student
Low Level Cognitive
Test Performance Mastery
Usually Unknown
Usually High
Variable
Performance Levels Based
Likelihood of Success
Student to Criterion
Cognitive or Psychomotor
![Page 16: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/16.jpg)
Comparison models?
INPUT PRODUCT
(Instruction) (NRT Results)
Model For NRT Construction
DESIGN TEST INPUT PRODUCT
MODIFY? NOYES
(Instruction) (CRT Results)
(Test, Objectives, or Instruction)
Model For CRT Construction
![Page 17: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/17.jpg)
Mastery curve
SEE HANDOUT
![Page 18: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/18.jpg)
Frequency distributions withstandard deviations of various sizes
SEE HANDOUT
![Page 19: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/19.jpg)
Section II
Establishing Cut Points
Three Primary Procedures
![Page 20: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/20.jpg)
ESTABLISHING CUT-POINT
1. Informed Judgement
2. Conjectural Approach
3. Contrast Group
![Page 21: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/21.jpg)
I. Informed Judgement
A. Significance: Separates mastery from non- masteryB. Procedure:
1. Analyze consequences of mid- classification (political, legal, or operational).2. Gather previous test-taker data.3. Ask other stakeholders.4. Make decision.
![Page 22: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/22.jpg)
II Conjecture Method
A. Significance: “Angoff-Nedeisky Method” - most useful.
B. Procedure:1. Select three informed judges.2. Estimate probability of correct
response.3. Chosen cut-off is average of the three judges.
![Page 23: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/23.jpg)
III Contrast Group Method
A. Significance: Single strongest technique; should still use human
judgement.
B. Procedure:1. Select judges to identify mastery/non-mastery.2. Select equal groups (15 minimum, 30 optimum).3. Administer mastery/non-mastery test to both groups.4. Plot scores on distribution chart.5. Make critical cut-off where two distributions intersect.6. Adjust score between highest non-master and lowest
master. score.
![Page 24: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/24.jpg)
Establishing A Criterion Cut-Point
Mastery Level - (Separates master from non-master)
1. Informed judgement
2. Conceptual Approach
3. Control groups
![Page 25: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/25.jpg)
Establishing A Criterion Cut-Point (cont.)Mastery Level - (Separates master from non master)
1. Informed judgement
2. Conceptual Approach
3. Control groups
![Page 26: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/26.jpg)
Establishing A Criterion Cut-Point (cont.)
Mastery Level - (Separates master from non master)
1. Informed judgement
2. Conceptual Approach
3. Control groups
![Page 27: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/27.jpg)
Contrasting group method of cut-off score selection chart.
SEE HANDOUT
![Page 28: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/28.jpg)
Section Three:
Reliability
![Page 29: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/29.jpg)
I. Types
A. Internal Consistency
1. Kuder-Richardson Method.
2. Computer Statistical Package.
3. Problem: Lack of variance.
4. Problem: Excludes items that measure unrelated objectives.
B. Test-Retest Score Consistency.
![Page 30: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/30.jpg)
Review
Types of Validity: Methods of Establishing Cut-Points1. Content 1. Informed Judgment 2. Construct 2. Conjecture Method3. Criterion-related 3. Contrast Group Method
Types of Reliability:1. Test-Retest2. Internal Consistency3. Equivalent forms4. Interrupter reliability
![Page 31: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/31.jpg)
Section Four: Review Questions Validity cannot exist without reliability. (True or
False) Since CRT relies on judgment rather than normal
distribution for scoring, how is reliability assured? If it becomes necessary for you to establish cut-
point for your training program, which of the three methods would you use and why? (Informed judgment, Conjecture method, or Contrast group method)
![Page 32: Cut Points ITE - 695. Section One n What are Cut Points?](https://reader035.vdocuments.site/reader035/viewer/2022081506/56649ef65503460f94c09c91/html5/thumbnails/32.jpg)
Norm-Reference Testing
GOALS
RELIABILITY
VALIDITY
ADMINISTRATION
STANDARD
MOTIVATION
COMPETITION
INSTRUCTIONALDOMAIN
Criterion Referenced Testing
Test Achievement
Usually High
Instruction Dependent
Standard
Averages-Based
Avoidance of Failure
Student to Student
Low Level Cognitive
Test Performance Mastery
Usually Unknown
Usually High
Variable
Performance Levels Based
Likelihood of Success
Student to Criterion
Cognitive or Psychomotor