the lexile framework for reading: integrating measurement and instruction copenhagen sunday, august...

The Lexile Framework for Reading: Integrating Measurement and Instruction

COPENHAGENSunday, August 14, 2011

A. Jackson StennerChairman & CEO, MetaMetrics

Research Professor, UNC-Chapel Hill

The Lexile Framework is NOT:

• A reading program

• A test or method of assessment

• Educational software

What is The Lexile Framework for Reading?

• An educational tool – a scale - that links reading material and readers under a common metric – the Lexile.

− Measures the complexity of reading material

− Measures the reading level of individual readers

The Lexile Framework for Reading

A student’s Lexile measure is her reading level.

A book’s Lexile measure is its reading difficulty.

an 1170L reader

an 1170L book

A Lexile is a measure of both reading level and text complexity, on the same scale.

Why The Lexile Framework?

• LINK reader to text under a common measure

• FORECAST levels of comprehension

• TRACK growth over time

Where are Lexile Measures Found?

• All major norm-referenced tests are linked• Over 30 million students received a Lexile measure last

school year• Over 450 publishers• Nearly 115,000 book titles to search at www.lexile.com• Over 400 million Lexiled articles

Where are Lexile Measures Found?Formative and Norm-Referenced Assessments

Where are Lexile Measures Found?Library Resources

The Lexile Scale

How high does the Lexile scale go?

How low does the Lexile scale go?

The Lexile Scale

• Lexile measures typically range from below 200L for beginning readers to above 1700L for advanced readers.

• Text below 200L represents beginning-reading material. A student’s Lexile measure of 0L or below is coded BR, signifying ‘Beginning Reading’.

Lexile Measures & Text Complexity

“Quantitative dimensions of text complexity refer to those aspects of text complexity, such as word length or frequency, sentence length, and text cohesion, that are difficult if not impossible for a human reader to evaluate efficiently, especially in long texts, and are thus today typically measured by computer software.”

-from the Common Core State Standards

Determining Text Complexity

• Sentence Length (syntactic complexity)Longer sentences are more complex and require more short-term memory to process.

Determining Text Complexity

• Vocabulary (semantic difficulty) Less familiar words impede reading fluency and affect comprehension.

Analyzing Text Readability

• Read and examine Camping is Fun. • Easy or difficult to read?

Why?

We went camping. On Monday, we put up the tent. The tent fell down! On Tuesday, we hiked. It rained andwe got wet! On Wednesday, we fished. Uncle Pete caught a boot! On Thursday, we left food on the table. The raccoons ate it! On Friday, we cookedmarshmallows. They fell in the fire! On Saturday, we heard a strangenoise. We were too scared to sleep!On Sunday, we went home. "Did you like camping?" asked Uncle Pete. "Yes!" we said. "Camping is fun!"

Camping is Fun(M. Garcia)

We went camping. On Monday, we put up the tent. The tent fell down! On Tuesday, we hiked. It rained andwe got wet! On Wednesday, we fished. Uncle Pete caught a boot! On Thursday, we left food on the table. The raccoons ate it! On Friday, wecooked marshmallows. They fell in the fire! On Saturday, we heard a strange noise. We were too scared to sleep!On Sunday, we went home. "Did you like camping?" asked Uncle Pete. "Yes!" we said. "Camping is fun!"

Camping is Fun(M. Garcia)

200L

Analyzing Text Readability

• Read and examine an excerpt of a speech from Lee Kuan Yew: The Separation of Singapore from Malaysia

• Difficult or easy to read? Why?

Speech from Lee Kuan Yew:Separation of Singapore from Malaysia

"For me, it is a moment of anguish. All my life, my whole adult life, I believed in merger and unity of the two territories. ... Now, I, Lee Kuan Yew, Prime Minister of Singapore, do hereby proclaim and declare on behalf of the people and the Government of Singapore that as from today, the ninth day of August in the year one thousand nine hundred and sixty-five, Singapore shall be forever a sovereign democratic and independent nation, founded upon the principles of liberty and justice and ever seeking the welfare and happiness of the people in a most and just equal society.“

Speech from Lee Kuan Yew:Separation of Singapore from Malaysia

"For me, it is a moment of anguish. All my life, my whole adult life, I believed in merger and unity of the two territories. ... Now, I, Lee Kuan Yew, Prime Minister of Singapore, do hereby proclaim and declare on behalf of the people and the Government of Singapore that as from today, the ninth day of August in the year one thousand nine hundred and sixty-five, Singapore shall be forever a sovereign democratic and independent nation, founded upon the principles of liberty and justice and ever seeking the welfare and happiness of the people in a most and just equal society.“

1550L

Why 75% Comprehension?

Research suggests that at 75%…• A reader can have a successful reading

experience without frustration or boredom• A reader can achieve “functional

comprehension” of the text• A reader will be sufficiently challenged (by

vocabulary and syntax) to improve

75% is the “right amount of challenge”

Managing Comprehension• Readers may experience frustration when…

−Text readability is 100L+ above their Lexile level.

• Readers may experience ease when…−Text readability is 50-100L below their Lexile

level.• Readers may experience growth when…

−Text readability is within their Lexile range.

General Reading Recommendation: Targeted text range of 100L below to 50L

above the student’s Lexile level

Note: This range may vary based on text type, reading context and purpose,

reading strategies and support, and reader motivation.

Finding targeted text on ecosystems:

Books: Lexile Measure:Inside Ecosystems and Biomes 700LExploring Ecosystems with Max Axiom, Super Scientist 750LEarth’s Ecosystems IG810LEcosystems 860LPopulations and Ecosystems 880LBiomes and Ecosystems 970LThe Eastern Forest 1010LThe Deserts of the Southwest 1080LThe Prairie 1140LEndangered Species 1160L

Time Reading and Reading Achievement

from Anderson et al., 1988, Table 3, N = 155. Reading material included books, magazines, and newspapers.

Fifth-Grade Students

Percentile Rank

Minutes of Text Reading

per Day

Estimated Number of Words

Read per Year

989070502010

67.333.316.99.22.41.0

4,733,0002,357,0001,168,000601,000134,00051,000

College and Career Readiness SkillsReading Demand of Newspapers

• USA Today (1200L)• Associated Press (1310L)• Chicago Tribune (1310L)• Wall Street Journal (1320L)• Washington Post (1350L)• NY Times (1380L)• Reuters (1440L)

College and Career Readiness SkillsReading Demand of International Newspapers

• The Egyptian Gazette (Egypt) 1440L• Oman Daily Observer (Oman) 1430L• Financial Times (Great Britain) 1430L• Straits Times (Singapore) 1410 L• Gulf Times (Qatar) 1420L• China Daily (China) 1400L• France Daily (France) 1400L• The Moscow Times (Russia) 1400L• The Australian (Australia) 1390L• German Times (Germany) 1390L• Copenhagen Post (Denmark) 1390L• Irish Times (Ireland) 1380L• Santiago Times (Chile) 1380L• Jerusalem Post (Israel) 1370L

FREE Lexile Toolswww.lexile.com

• Lexile Find a Book

• Lexile Analyzer

www.Lexile.com

Find-a-Book

Select reading interest:

Search results:

Create a reading list:

Edit reading list to personalize:

Search for books:

Search results:

Link to public library

Link to public library

Tools - Lexile Analyzer

The Lexile Analyzer

The Lexile Analyzer

Copy the article

The Lexile Analyzer

Paste the text into the Word

document

The Lexile Analyzer

Remove titles, pictures, graphs,

illustrations, captions, sub-headings, and

bylines

The Lexile Analyzer

Be sure to remove

hyper-links

The Lexile Analyzer

Be sure to save as ‘Plain

Text’

Oasis A student-centered personalized

learning platform– Does not require teacher direction

Differentiated activities Embedded assessment in the

background Web-enabled with minimal bandwidth

requirement Built on the principles of deliberate

practice…

Demo Accounts for Copenhagen

45 demo accounts have been created inside Oasis for the folks in Copenhagen.

All the usernames are: rasch# where the # is a number from 1 to 45 (eg., rasch1, rasch15, rasch42).

All the passwords are: jakob Oasis can be accessed by browsing to:

http://www.alearningoasis.com and clicking on the green “launch” button. If anyone has issues logging in, they can contact Sean Hanlon – shanlon@lexile.com

Reader Ability

Temperature

Three well researched constructs

Reader ability

Text Complexity

Comprehension

Reading is a process in which information from the text and the

knowledge possessed by the reader act together to

produce meaning.

Anderson, R.C., Hiebert, E.H., Scott, J.A., & Wilkinson, I.A.G. (1985) Becoming a nation of readers: The report of the Commission on ReadingUrbana, IL: University of Illinois

An Equation

=Reader Ability

Text ComplexityComprehension -

Conceptual

Statistical

ExpectedRawScore

=i

e (RA – TC )i

1 + e (RA – TC i)

RA = Reading Ability

TC = Text Calibrations

A causal model relating reader ability, text complexity, and comprehension.

Measures reader ability and text complexity, on a common scale—the Lexile scale.

Allows educators to forecast the level of success a reader is likely to experience with a particular text.

The Lexile Framework for Reading

Eight Features of the Causal Model Relating Text Complexity, Reader Ability, Task Difficulty and Comprehension1. The model is individual centered. The focus is on explaining

variation within person over time.

2. In this framework the measurement mechanism is well specified and can be manipulated to produce predictable changes in measurement outcomes (e.g. percent correct).

3. Item parameters are supplied by substantive theory and, thus, person parameter estimates are generated without reference to or use of any data on other persons or populations. Therefore, effects of the examinee population have been completely eliminated from consideration in the estimation of person parameters for reader ability.

4. The quantitivity hypothesis can be experimentally tested by evaluating the trade-off property for the individual case. A change in the person parameter can be off-set or traded-off for a compensating change in text complexity to hold comprehension constant. The trade-off is not just about the algebra.

5. When uncertainty in item difficulties is too large to ignore, individual item difficulties may be a poor choice to use as calibration parameters in causal models. As an alternative we recommend, when feasible, averaging over individual item difficulties to produce “ensemble” means. For example empirical text complexities can be excellent dependent variables for testing causal theories.

Eight Features of the Causal Model cont’d.

6. Causal Rasch models are individual centered and are explanatory at both within-subject and between-subject levels. The attribute on which I differ from myself a decade ago is the same attribute on which I differ from my brother today.

8. When data fit a Rasch model, differences between person measures are objective. When data fit a causal Rasch model absolute person measures (reader abilities) are objective (i.e. independent of instrument).

9. Causal Rasch models make possible the construction of generally objective growth trajectories. Each trajectory can be completely separated from the instruments used in its construction and from the performance of any other persons, whatsoever.

Eight Features of the Causal Model cont’d.

May 2016(12th Grade)

1200

1000

1400

1600

Text Demands forCollege and Career

May 2007 – April 2011

347 Encounters138,695 Words3,342 Items983 Minutes

Student 1528

7th GradeMaleHispanicPaid Lunch

Expected: 73.5%Observed: 71.7%

Theoretical versus Empirical Text Complexity for

719 Articles*

Reliability = 0.997

SEM = 12.8L

r = 0.968

r” = 0.969

R2” = 0.938

RMSE” = 89.6L

* Inclusion criteria: 50 encounters and 1,000 items

How Temperature and Pressure Relate Under Constant Volume

Temperature Volume Pressure

2000°K 20 Liters 20.0 atm

1000°K 20 Liters 10.0 atm

500°K 20 Liters 5.0 atm

250°K 20 Liters 2.5 atm

125°K 20 Liters 1.25 atm

Comprehension Rates for Readers of the Same Ability with Texts of Different Complexity or How Reader Ability and Comprehension Rate Relate Under Varying Text Complexity

ReaderAbility

TextComplexit

yText Titles

Comprehension Rates

1000L

1000L

1000L

1000L

1000L

500L

750L

1000L

1250L

1500L

The Magic School Bus, Inside the

Earth (Cole)

The Martian Chronicles (Bradbury)

The Reader’s Digest

The Call of the Wild (London)

On Equality Among Mankind

(Rousseau)

96%

90%

75%

50%

25%

Comprehension Rates for Fixed Reader Ability

Comprehension Rates for Readers of Different Ability with Texts of the Same Complexity or How Reader Ability and Comprehension Rate Relate Under Constant Text Complexity

Reader Ability Classroom Textbook Comprehension Rates

500L750L

1000L1250L1500L

1000L1000L1000L1000L1000L

25%50%75%90%96%

Comprehension Rates for Fixed Text Complexity

How Many Ways Can We Say X Causes Y? X “elicited a greater”

YX “impacts” Y

X “accounts for” Y X “has been linked to” Y

Y “is the result of” X X “didn’t diminish” Y

Y “because of” X Y “depends on” X

X “has led to” Y X “largely motivates” Y

Y “stemmed from” X X “proved critical to” Y

X “fosters” Y X “changes” Y

X “triggers” Y X “affects” Y

To causally explain a phenomenon [a measurement outcome] is to provide information about the factors [person processes and instrument mechanisms] on which it depends and to exhibit how it depends on those factors. This is exactly what the provision of counterfactual information…accomplishes: we see what factors some explanandum M [measurement outcome, raw score] depends on (and how it depends on those factors) when we have identified one or more variables such that changes in these (when produced by interventions) are associated with changes in M (Woodward, 2003, p.204).

Ensemble Calibration

The Psychometric Engine of Oasis

Oasis Items Single Instance No individual item statistics Theoretical estimates of text

complexity become empirical estimates of text complexity

Protocol for determining response illustrations and foils

Machine-generated with no human review

Item Distributions

Items are presumed to be sampled from a distribution with a mean difficulty and an imputed spread

This distribution is called an ensemble

Traditional Rasch Algorithm Iterative process Two stages of iteration

– Begin with asssumed starting values for person abilities and item difficulties (usually item difficulties)

– Estimate person abilities given item difficulties.– Fix these estimated person abilities and estimate new

item difficulties given these person abilities– Iterate until a stopping criterion is met

Ensemble Calibration Algorithm Same iterative technique as traditional

method– Fix person abilities to determine ensemble

means– Fix resulting ensemble means to determine

person abilities– Iterate until stopping criterion is met

What is Different? Individual item difficulties neither

available nor needed Start iterative procedure with person

abilities from Oasis P-delta-sigma table to account for

spread of distributions Spread of ensemble effects success rate

P-delta-sigma table A success rate (p) is determined by the difference in

the ensemble mean for a distribution and the person ability (delta) and the spread of the ensemble (sigma)

Normal distribution assumption for the ensemble of item difficulties

Table used instead of integration over the random term for programming simplicity– Dimensions: (delta -1000 to +1000 by fives,

Sigma 10 to 260 by fives)

Formula for p-delta-sigma

p-delta-sigma = uee

eu

u

u

2

2

2

21

1

Δ=delta=difference between person ability and ensemble calibration

σ=sigma=standard deviation of the ensemble

u=variable of integration=random difference of item difficulty from ensemble mean

Spread of Ensemble

Ensembles vary in mean but not in SD

SD is uniform for this implementation

With embellishment a method for

unequal variances could be derived

ClosingNo matter how it is sliced and diced, analyses of joint and conditional probability distributions yield no more than patterns of association. Nothing in the response data nor Rasch analyses of these data exposes the processes (features of the object of measurement) or mechanisms (features of the instrument) that are hypothesized to be conjointly causal on the measurement outcomes.

A. Jackson Stenner CEO, MetaMetricsjstenner@Lexile.com

Contact Info: