language, dialect, and register: sociolinguistics and the estimation of measurement error in the...
TRANSCRIPT
Language, Dialect, and Register: Sociolinguistics and the Estimation ofMeasurement Error in the Testing of
English-Language Learners*
Guillermo Solano-FloresAmerican Institutes for Research
NCCRESt Conference,Arizona State University in Tempe, AZ
November 18, 2004
*Research funded by the National Science Foundation, REC-0336744
A warm-up exercise
What is the animal that eats shoots and leaves?
What is the animal that eats, shoots, and leaves?
Overview
• The sociolinguistic dimension of ELL testing
• Key concepts: dialect, register, linguistic misalignment
• Heterogeneity of populations, score variation, and measurement error
• Dialect and register: as important as language in ELL testing
ELL testing: Levels of analysis
language
dialect register
The concept of dialect
Dialects used by different speech communities
Language
Dialect in testing: An example
How much money does Laura need to pay for two candies that cost 45 cents each?
How much money does Laura need to pay for two sweets that cost 45 cents each?
How much money does Laura need to pay for two 45-cent candies?
Dialect intersection
A BA B
Dialect A:Speech
Community A
Dialect B:Speech
Community B
The concept of registerregister
Registers: Uses of language in different contexts
Language
Register in testing: An example
Reproduced from TIMSS Population 1Item Pool. Copyright © 1994 by IEA, The Hague
The interaction ofdialect and register
Standard English:The length of a dinosaur was reported to have been 80 feet (rounded to the nearest 10 feet). What length other than 80 feet could have been the actual length of this dinosaur?
The interaction ofdialect and register
Standard Spanish:Se reportó que la longitud de un dinosaurio era 80 pies (redondeada a los 10 pies más cercanos). ¿Cuál longitud además de 80 pies podría haber sido la longitud real de este dinosaurio?Southern California:Se reportó que un dinosaurio midió 80 pies de largo (redondeado a la decena más próxima). Aparte de 80 pies, ¿qué pudo haber sido la medida exacta del dinosaurio?
Dialect and register
Miami:Yo di yon dinozò te mezire 80 pye de longè (Rapwoche pi pre 10 pye). Ki lòt longè pase 80 pye ki ta kapab vrè longè dinozò a?
Repons:___________
Brooklyn:Yo rapòte longè yon dinozò te 80 pye (longè a awondi nan dizèn ki pi pre a). Apa 80 pye, ki lòt vrè longè dinozò a ta ka genyen?
Repons:___________
Linguistic misalignment:Frequency and severity
How much money does Laura need to pay for two candies that cost 45 cents each?
How much money does Laura need to purchase two sweets at 45 cents a piece?
How much money does Laura need to pay for two 45-cent candies?
Linguistic misalignment
Student:• Non-standard
dialect• Partially unfamiliar
with testing register
• Content area register shaped by instructional context
T ST S
Test:• In standard dialect• Assumes full
command of testing register
• Content area register based on standards documents
Find instances of misalignment:Sam can purchase his lunch at school. Each day he wants to have juice that costs 50¢, a sandwich that costs 90¢, and fruit that costs 35¢. His mother has only $1.00 bills. What is the least number of $1.00 bills that his mother should give him so he will have enough money to buy lunch for 5 days?
NAEP: Mathematics Grade 4, 1996 Public Release
Students’ interpretations* ofHis mother has only $1.00 bills
1. His mother has only bills of one dollar
2. His mother has only one dollar
3. His mother has only dollar bills
*As reflected by students’ read-alouds
Linguistic misalignment probabilistic spaceF
req
uen
cyof
mis
alig
nm
ent
Severity of misalignment
Linguisticallychallenging items
Linguisticallysound items
Generalizability (G) theory in one slide
student (s)
item (i)
dialect (d)
si
sd
id
sid,e
Main effect
Interaction effect
Object ofmeasurement
Sources ofmeasurement error
Error due tounknown sources
Facets
Miami students tested in Standard English and Standard Haitian-Creole
student (s) 20 rater (r) 0 item (i) 6 language (l) 5 sr 0 si 11 sl 7 ri 0 rl 0 il 1 sri 0 srl 0 sil 39 ril 0 sril,e 11
Percentageof score variationSource
Miami students tested in Miami Haitian-Creole and Standard Haitian-Creole
student (s) 22 rater (r) 0 item (i) 5 dialect (d) 0 sr 0 si 23 sd 2 ri 0 rd 0 id 0 sri 0 srd 0 sid 33 rid 0 srid,e 14
Percentageof score variationSource
Approaches to ELL testing
Item response theory
• Scaling• Differential item
functioning• Between-group
designs• Reference groups• Population
homogeneity
Generalizability theory
• Measurement error
• Score dependability
• Within-group designs
• No reference groups• Population
heterogeneity
Conclusions
• Dialect and register are as important as language in the testing of ELLs
• Testing and psychometric approaches for ELLs should and can be well grounded on reasonings from sociolinguistics