unit standardization and technical properties n standardization sample n reliability studies...

UNIT Standardization and Technical Properties

Standardization Sample Reliability Studies

• Internal Consistency• Reliabilities at Decision-Making Points• Test-Retest Stability

Floors, Ceilings, and Difficulty Gradients Validity Studies

Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

Standards of Test Technical Adequacy for School-Aged Children

Internal Consistency• Median subtest internal consistency averaged across ages .80• Total test internal consistency averaged across ages .90• Screening test internal consistency averaged across ages .80

Test–Retest Stability• Total test stability coefficient .90

Floors• Average subtest floor 2 SDs or more below normative mean• Total test floor is 2 SDs or more below normative mean

Ceilings• Average subtest ceiling 2 SDs or more above normative mean• Total test ceiling 2 SDs or more above normative mean

Difficulty Gradients• For subtests and total test, no fewer than three measurement points per standard deviation, or each measurement point equivalent to no more than one-third standard deviation

Validity• Validity studies reported (e.g., group mean differences)Note. Adapted from “Limitations of Preschool Instruments and Standards for Minimal Levels of Technical Adequacy,” by B. A. Bracken, 1987, Journal of Psychoeducational Assessment, 5, pp. 313–326. Copyright 1987 by the Journal of Psychoeducational Assessment.

Table 5.1


UNIT Standardization Sample Normed on 2,100 children and adolescents

aged 5.0–17.11 years 108 sites across 38 states Total of 3,865 children and adolescents tested for

norms development and reliability, validity, and fairness studies

Sample representative of the general USA population based on 1995 census updates


UNIT Standardization Sites


UNIT Sampling Variables Age (5 years 0 months through 17 years 11 months) Sex (Female, Male) Race (African American, Asian American, Native American,

White, Other) Hispanic Origin (Hispanic, Non-Hispanic) Region (Midwest, Northeast, South, West) Community Setting (Urban/Suburban, Rural) Parental Educational Attainment (Less than High School

Degree; High School Graduate or Equivalency; Some College or Technical School; Four or More Years of College)

Inclusion of Representative Exceptional Children and Adolescents


UNIT Standardization Sample Compared with U.S. Population: Sex, Race, Ethnicity, and SES

UNIT Sample % U.S. Population %SexFemale 49.9 48.9Male 50.1 51.1

RaceAfrican American 16.0 16.0White 78.4 78.8Other 5.7 5.2EthnicityHispanic 13.0 13.0Non-Hispanic 87.0 87.0Parent Educational AttainmentLess than High School 14.0 13.7High School Graduate 29.5 29.1Some College 29.2 29.7 4 Years of College 27.3 27.5

Note. N = 175 in each age group.Copyright © 1998 by The Riverside Publishing Company. Permission is granted to reproduce or download pages for informational or educational use.

UNIT Standardization Sample Compared with U.S. Population: Geographic Region and Community Size

UNIT Sample % U.S. Population %

Geographic Region (38 states)Midwest 24.4 24.1Northeast 18.0 18.2South 35.0 35.2West 22.7 22.5

Community SizeRural 27.6 24.8Urban/Suburban 72.4 75.2


UNIT Standardization Sample Compared with U.S. Population: Special Education Services

UNIT Sample % U.S. Population %

Learning Disabilities 5.6 5.9

Speech and Language Impairments 2.3 2.4

Serious Emotional Disturbance 0.9 1.0

Mental Retardation 1.2 1.3

Hearing Impairments 0.2 0.2

Intellectual Giftedness 6.2 6.4

Bilingual Education 1.8 3.1

English as a Second Language 2.0 4.0


UNIT Internal Consistency Split-Half Correlations With Spearman-Brown Corrections

Standard BatteryMemoryReasoningSymbolicNonsymbolicFull Scale

AverageReliabilit

y.88.90.87.91.93

Clinical/Exception

al Sample

.95

.96

.95

.97

.98


UNIT Decision-Making Reliabilities Split-Half Correlations With Spearman-Brown Corrections (Also corrected for restriction or expansion in range)


FSIQ = 70±10

.97

.97

.96

.98

.98

FSIQ = 130± 10

.96

.97

.95

.98

.98


Test-Retest Stability Corrected for variability on the first testing


Corrected r

.81

.87

.78

.84

.88


UNIT Floors and Ceilings

Floors: A term describing the extent to which there are a sufficient number of easy items to distinguish between examinees of delayed to very delayed ability

Ceilings: A term describing the extent to which there are a sufficient number of difficult items to distinguish between examinees of superior and very superior ability


UNIT Floors and Ceilings

Floors: UNIT subtests have very good to excellent floors for low ability examinees across the entire age range, with the exception of the Abbreviated Battery at the youngest ages

Ceilings: UNIT subtests have consistently excellent ceilings, even for the oldest and highest ability examinees


UNIT Item Difficulty Gradients Difficulty Gradients: A term describing the steepness of the

gradations in item difficulty. An increase or decrease in a single raw score point should not result in a standard score change of more than one-third SD

UNIT Difficulty Gradients: Average item gradients for UNIT subtests equal or exceed standards for every age level, indicating UNIT’s sensitivity to subtle differences in ability levels (with the exception of one subtest at the earliest ages)


UNIT Internal Validity Studies

Evidence of Content Validity Structural Evidence of Validity

• Development and Growth Curves• Subtest Unidimensionality Studies• Subtest and Scale Intercorrelation Studies• Comparison among the three UNIT Batteries• Factor Analytic Investigations

• Exploratory Factor Analyses• Confirmatory Factor Analyses


0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

5 7 9 11 13 15

Age in Years

Mea

n Su

btes

t Raw

Sco

re

Symbolic Memory

Cube Design

Spatial Memory

Analogic Reasoning

Object Memory

Mazes

Age Progression of UNIT Subtests:Mean Raw Scores By Age


460

470

480

490

500

510

520

530

540

5 6 7 8 9 10 11 12 13 14 15

16-1

7

Age in Years

Mea

n W

-Sco

re

Symbolic Memory

Spatial Memory

Object Memory

Cube Design

Analogic Reasoning

Mazes

Age Progression on UNIT Subtests:Mean Rasch W Scores By Age


475

480

485

490

495

500

505

510

515

520

5 6 7 8 9 10 11 12 13 14 15 16-17

Age in Years

Mea

n W

-sco

res

Age Progression of UNIT Standard Battery FSIQ:Mean Rasch W Score By Age


Exploratory Factor Analyses for Standard BatterySecond-Order Principal Components Schmid-Leiman Solution for the Standardization and Clinical/Exceptional Samples

Standardization Sample Clinical/Exceptional SampleSecond First Second First

Order Order Order Order g Memory Reasoning g Memory Reasoning

Symbolic Memory .75 .46 –.05 .88 .35 .03

Cube Design .77 .06 .36 .88 .05 .34

Spatial Memory .80 .33 .11 .88 .34 .05

Analogic Reasoning .73 –.05 .45 .88 .03 .36


Confirmatory Factor Analyses for the Standardization Sample Factor Loadings (Standardized Regression Beta Weights)

Memory Reasoning Symbolic

Nonsymbolic

Symbolic Memory 0.66 — — — Cube Design — 0.70 — — Spatial Memory 0.74 — — — Analogic Reasoning — 0.67 — — Object Memory 0.63 — — — Mazes — 0.37 — —


Confirmatory Factor Analyses for the Standardization Sample Factor Loadings (Standardized Regression Beta Weights)

Memory Reasoning Symbolic Nonsymbolic

Symbolic Memory — — 0.66 — Cube Design — — — 0.68 Spatial Memory — — — 0.75 Analogic Reasoning — — 0.65 — Object Memory — — 0.63 — Mazes — — — 0.36


Confirmatory Factor Analyses for the Clinical/Exceptional SampleFactor Loadings (Standardized Regression Beta Weights)

Memory Reasoning Symbolic

Nonsymbolic

Symbolic Memory 0.71 — — — Cube Design — 0.74 — — Spatial Memory 0.78 — — — Analogic Reasoning — 0.73 — — Object Memory 0.68 — — — Mazes — 0.47 — —


Confirmatory Factor Analyses for the Clinical/Exceptional SampleFactor Loadings (Standardized Regression Beta Weights)

Memory Reasoning Symbolic Nonsymbolic

Symbolic Memory — — 0.71 — Cube Design — — — 0.71 Spatial Memory — — — 0.78 Analogic Reasoning — — 0.70 — Object Memory — — 0.68 — Mazes — — — 0.47


UNIT External Validity Studies

Correlational Studies• Measures of Intelligence• Measures of Academic Achievement

Studies with Clinical and Exceptional Samples• Speech and Language Impairment (N = 57)• Learning Disabilities (N = 205)• Mental Retardation (N = 84)• Intellectually Gifted (N = 160)• Serious Emotional Disturbance (N = 23)


UNIT Standard FSIQ Correlations with Other IQs

WISC-III FSIQ across four samples (Learning Disabled, Mentally Retarded, Intellectually Gifted, and Native American).81–.84

WJ-R Tests of Cognitive Ability BCA (Est): .83 K-BIT IQ Composite: .82 Matrix Analogies Test Standard Score: .83 Raven’s Standard

Progressive Matrices T-Score: .56 TONI-2 Quotient: .63Note: All correlations are corrected to control for restriction or expansion in range.

r


Correlations Between UNIT Full Scale IQs and WJ-R Tests of Achievement

UNIT Abbreviated Battery FSIQ 133 .78 .84 .78 .86 .82 95.3 23.0

UNIT Standard Battery FSIQ 133 .81 .86 .81 .87 .85 93.8 24.4

UNIT Extended Battery FSIQ 133 .80 .87 .81 .87 .85 92.1 24.0

Mean 92.4 94.3 86.4 94.9 88.5SD 28.1 31.3 29.0 23.4 27.7

N

WJ-RBroadReading

WJ-RBroadMathematics

WJ-RSkills

WJ-RBroadWrittenLanguage

WJ-RBroadKnowledge Mean SD


Comparative Prediction of Achievement by UNIT and Other Nonverbal Measures

UNIT Abbreviated Battery FSIQ 133 .78 .84 .86 .82

UNIT Standard Battery FSIQ 133 .81 .86 .87 .85

UNIT Extended Battery FSIQ 133 .80 .87 .87 .85

Leiter-R Brief IQ* 29 .79 .80 — —

Leiter-R Full IQ* 29 .82 .82 — —

TONI-3 (Form A)* 20 .73 .76 .56 .76

TONI-3 (Form B)* 20 .71 .74 .55 .70

*Note. These data come from each test's manual.

N

WJ-RBroadReading

WJ-RBroadMathematics

WJ-RBroadKnowledge

WJ-RSkills

UNIT Validity Studies: Diagnoses and Exceptionalities

Speech and Language Impairment (N = 57) Learning Disabilities (N = 205) Mental Retardation (N = 84) Intellectually Gifted (N = 160) Serious Emotional Disturbance (N = 23)


Table 5.31

Abbreviated BatteryFSIQ Mean 94.16 98.32 4.16 0.28

SD 15.25 15.04Standard BatteryMemory Quotient Mean 94.81 99.51 4.79 0.31

SD 16.26 14.71Reasoning Quotient Mean 91.42 98.53 7.11 0.47

SD 14.97 13.69Symbolic Quotient Mean 93.67 99.89 6.22 0.41

SD 15.43 15.68Nonsymbolic Quotient Mean 92.40 98.02 5.62 0.37

SD 15.95 14.03FSIQ Mean 91.98 98.79 6.81 0.45

SD 15.42 14.48Extended BatteryMemory Quotient Mean 93.91 99.93 6.02 0.40




SD 15.82 14.29FSIQ Mean 91.33 100.05 8.72 0.58

SD 15.93 14.62

Note. Samples were matched according to age, sex, race, ethnicity, and parent education level.

UNIT Performance by a Sample With Speech and Language Impairments and a Control Sample

ClinicalSample(n = 57)

Control Sample(n = 57)

DifferenceScore

Effect Size


Table 5.32






SD 13.38 12.92FSIQ Mean 90.20 100.83 10.63 0.71





SD 14.10 13.27FSIQ Mean 89.84 100.87 11.03 0.74

SD 13.69 13.19


UNIT Performance by a Sample With Learning Disabilities and a Control Sample



DifferenceScore

Effect Size


Table 5.33






SD 14.29 12.96FSIQ Mean 62.40 96.13 33.73 2.25





SD 14.50 14.17FSIQ Mean 60.96 95.01 34.05 2.27

SD 12.72 12.91


UNIT Performance by a Sample With Mental Retardation and a Control Sample

ClinicalSample(n = 84


DifferenceScore

Effect Size


Table 5.34Control Sample(n = 160)

ExceptionalSample(n = 160)

DifferenceScore

Effect Size






SD 10.75 14.22FSIQ Mean 117.64 104.14 13.50 0.90





SD 10.83 14.35FSIQ Mean 116.48 104.28 12.20 0.81

SD 9.85 14.55


UNIT Performance by a Sample With Intellectual Giftedness and a Control Sample


Table 5.35 UNIT Performance by a Sample With Serious Emotional Disturbance and a Control Sample

Abbreviated BatteryFSIQ Mean 93.57 93.35 -0.22 0.01




SD 14.71 14.54Nonsymbolic Quotient Mean 93.43 92.96 -0.47 0.03

SD 12.91 13.70FSIQ Mean 92.65 93.09 0.44 0.03


SD 15.02 14.31Reasoning Quotient Mean 92.91 92.78 -0.13 0.01


SD 14.90 14.04Nonsymbolic Quotient Mean 93.57 93.00 -0.57 0.04

SD 12.63 14.93FSIQ Mean 93.13 93.43 0.30 0.02

SD 14.24 14.18




DifferenceScore

Effect Size


unit standardization and technical properties n standardization sample n reliability studies...

Documents