Florida Assessments for Instruction in Reading-Overview of the K-2 System

Barbara R. Foorman, Ph.D.

Florida Center for Reading Research

Florida State University

2

FAIR Facts Florida Center for Reading Research (FCRR) began

development in 2007 at request of FL Dept. of Ed FCRR conducted item tryouts in 2007-08 and

implementation study in 25 FL schools in 2008-09 Launched statewide in fall 2009: voluntary except for

FLKRS; over 1.6 million students assessed Administered three times a year with additional ongoing

measures available Assists Florida in its efforts to provide additional

computer-based assessment in the future Offered free of charge to Florida school districts

3

K-2 Teacher Survey Results70%+ of 150 teachers in implementation study responded• 67% completed K-2 FAIR in 5 days or less• 89% found instructions clear and EST easy to use• 90% agreed or strongly agreed that FAIR results helpful

in identifying strengths and weaknesses of students• 78% said useful in forming instructional groups• 80% would recommend FAIR to another teacher• 86% found FAIR had comprehensive approach to reading• 75% agreed results inform parents about child’s reading

4

Benefits of using FAIR Valid and reliable tool for monitoring student

progress Strong psychometric test properties form the

basis for all assessments Broad Screen tasks are directly related to

Florida standards and predict to end of year outcome measures (SESAT, SAT-10, and FCAT)

5

Benefits of using FAIR Access to the Progress Monitoring and Reporting Network

(PMRN) for recording student scores (by hand or scoring tool in K-2, automatically in 3-12)

Access to the PMRN reports that assist teachers in analyzing student data to better inform instruction

Links to instructional resources for teachers (PMRN reports, search tool, program specific professional development)

Passages equated for difficulty to allow for accurate progress monitoring and comparison of student performance

6

FAIR Administration Internet-based options

Teacher Administered in K-2 Electronic Scoring Tool – paper and pencil

scoring also available Computer-administered in grades 3-12

Adaptive assessment provides precise ability placement for reading comprehension tasks to monitor students at their instructional level

7

Reading Priorities in Florida

Six Components 4 Types of Assessment

High-Quality Initial Instruction

Immediate Intensive Intervention

Oral Language Screening Background Knowledge

Extended Time

Phonological Awareness

Progress Monitoring

Motivation Flexible Grouping

Phonics Diagnosis Explicit, Systematic Accommodations

Fluency Outcome Differentiated, Scaffolded

More frequent Progress Monitoring

Vocabulary Reading/Writing Connection

Comprehension Print-rich

8

Converging Evidence

Valid and reliable predictors of risk for reading difficulty are:

Print concepts (preK; early K) Letter name knowledge (early K) Phonological awareness and letter sounds (K-1) Rapid naming of letters (end of K to early G1) Word recognition (G1 and beyond) Vocabulary

(Fletcher et al., 2002; NELP, 2009; Scarsborough, 1998; Torgesen, 2002)

9

Comprehensive Assessment System

1. Phonemic Awareness

2. Phonics

3. Fluency

4. Vocabulary

5. Text Comprehension

6. Orthographic Skills (Spelling)

K-2 3-12

*

* embedded in text comprehension

The K-2 “Big Picture” Map

Broad Screen/Progress Monitoring Tool (BS/PMT) “All” students

• Letter Naming & Sounds• Phonemic Awareness • Word Reading

Broad Diagnostic Inventory (BDI) “All” students“Some” students for vocabulary

• Listening Comprehension• Reading Comprehension• Vocabulary • Spelling (2nd grade only)

Targeted Diagnostic Inventory(TDI) “Some” students

• K = 9 tasks• 1st = 8 tasks• 2nd = 6 tasks

Ongoing Progress Monitoring(OPM)“Some” students

•K – 2 = TDI tasks•1 – 2 = ORF

10

Predictive Validity of Broad Screen

Reliability of Broad Screen

Table 9: IRT Precision Estimates for Kindergarten and Grade 1 Broad Screen Tasks by Time of Assessment

Grade Task AP 1 AP 2 AP 3

Kindergarten Letter Names 0.51* - -

Letter Sounds 0.87 0.86 -

Phonological Awareness - 0.86 0.87

Word Reading - - 0.86

Grade 1 Word Reading 0.86 0.86 0.86

*The Letter Sounds task was more reliable than Letter Names at AP 1; however, due to the restricted range for high risk, a policy decision was made to use the Letter Name task in order to better capture the floor of the distribution. Because the Broad Screen in G2 is a timed task, precision estimates are not reported; however,

test-retest reliability was strong (.79-.84).

The K – 2 “Score” MapBS/PMT PRS = Probability of Reading Success

BDI LC = Listening Comprehension Total questions correct (implicit/explicit)

RC = Reading Comprehension Total questions correct (implicit/explicit), Fluency, Percent Accuracy Target Passage

VOC = Vocabulary Percentile Rank

SPL = Spelling Percentile Rank

TDI ME = Meets Expectations

BE = Below Expectations

OPM ORF = Adjusted Fluency

OPM TDI Tasks = ME or BE and Raw Score 13

14

Probability Score – Probability of Reading Success (PRS) Score calculated based on the BS/PMT

Probability is…the relative possibility that an event will occur or is likely to occur.

A .70 means that we expect this student to have a 70% chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10. The PMRN Reports have already converted the probability

score into a percentage so you will see .70 reported as 70%

14

A Teacher’s View of the Class Data - Class Status Report (K-2)-

©2009 Florida Center for Reading Research

Student Score Detail Box (K-2)

©2009 Florida Center for Reading Research

Excellent report to include in a student’s cumulative folder

K-2: School Status Report

Success Zones - Probability Scores

GREEN 85% or better chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10

16-84% chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10

15% or less chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10

18

YELLOW

RED

Probability of Reading Success What percentage of students are in each success zone at the

various grade levels? High Success (Green Zone) Moderate Success (Yellow Zone)

It is very important to look at the grade summary report to see the distribution within the yellow zone

Are most of the students in the yellow zone towards the 84% chance end of the zone or closer to the 16% chance

Low Success (Red Zone) What percentage of students are in each success zone within

specific teachers’ classrooms? Is there a specific class that does not seem consistent with the rest of

the grade? Click on the + sign next to the grade level summary to extend and see

the data on individual teachers at that grade level

Probability of Reading Success Look for trends within

grade levels, within teachers, and across grades.

Let’s look across grades in our sample data.

What do you notice when you compare across grades?

85% & above

(Green)

16%-84%

(Yellow)

15% & below

(Red)

K 28% 61% 11%

1st 36% 56% 8%

2nd 25% 70% 5%

Median Score The median score = the middle score

The value which 50 percent of the scores fall above and 50 percent fall below

The median score will be used throughout the PMRN on percentiles and PRS to show the middle score for your group of students. On the School Status Report the Median Percentile Score is given for

Vocabulary and Spelling (2nd only)

On bar graph form it is represented by the T and in table form it is represented as a gray shaded box.

Percentile Rank

Percentile ranks are derived scores that are used to rank one student’s performance in relation to a specific group.

On the Florida Assessments for Instruction in Reading, the specified group was a representative sample of Florida students.

Percentile rank is NOT a percentage score identifying how many items were answered correctly.

Percentile Rank is the score given on Vocabulary and Spelling (in 2nd grade only)

22

K-2: School Status Report

Target RC Passages for Grades 1 and 2 (BDI)

©2009 Florida Center for Reading Research

K-2: School Status Report

Target Passage – What does it tell me? At the School Level, the Target Passage column tells you

how many students were able to read at or above the Target Passage with 90% accuracy.

The calculation includes all students – even those who ended up in Listening Comprehension so you can answer the question… How many of my students are accurately decoding at the

level expected of this grade level at this assessment period (AP). To help remember this, write (Accuracy) in parentheses under or next

to the column header Target Passage.

Target Passage In first and second grade, this percentage can provide you

with one piece of information that will help you determine if your Core Reading Program is addressing the decoding strategies required for accurate grade-level reading.

Let’s look at the sample data 1st grade – 33.6% of my 143 first graders were able to read at or

above the Target Passage which is Passage 1 (66.4% were placed in Listening Comprehension)

2nd grade – 57.5% of my 122 second graders were able to read at or above the Target Passage which is Passage 1 (42.5% were placed in a lower 2nd grade passage, a 1st grade passage or 9 were placed into Listening Comprehension)

Total Questions Correct How many total questions correct (out of 5) did the student

answer in either a reading passage or listening comprehension. At the student level, you can get a break down of explicit (out of 3)

and implicit (out of 2) questions.

The reading comprehension (RC) graph represents all students who took a RC passage not only those who were at or above the target passage, so…what does this tell you? When they read a passage with at least 90% accuracy (their

instructional level), how many comprehension questions can they answer correctly. To help remember this, write (Comprehension) in parentheses under or

next to the column header Total Questions Correct.

K-2: School Status Report

Total Questions Correct - RC Let’s look at the First Grade Sample Data:

33.6% of 143 were at or above Target Passage (Passage 1) = 48 students 4 students = Answered 0 questions correct 7 students = Answered 1 question correct 13 students = Answered 2 questions correct 8 students = Answered 3 questions correct 12 students = Answered 4 questions correct 4 students = Answered 5 questions correct

Total Questions Correct - LC Let’s look at the First Grade Sample Data:

66.4% of 143 were below Target Passage (Passage 1) = 95 students 9 students = Answered 0 questions correct 13 students = Answered 1 question correct 26 students = Answered 2 questions correct 19 students = Answered 3 questions correct 14 students = Answered 4 questions correct 14 students = Answered 5 questions correct

Questions to Answer Using the School Status Report Where should my resources be allocated (which

grades and classes are struggling in certain areas)? Where should professional development efforts be

focused? What percentage of students is at each success zone

in each class and grade? How many students need additional support?

Are these students clustered in one grade/one teacher? What types of intervention do the students need?

Where should I focus my PD? Are there any Programs that I need to purchase?

School Grade Summary Report (K-2)

School Grade Summary Report (K-2)

Questions to answer using the School Grade Summary Report What is the distribution of scores in a particular

grade level? The School Status told us the Median Score for

Vocabulary was the 24th percentile, but now I see the full distribution and can see that the majority of my students fell within the 11th – 30th percentile range.

This tells me that overall, the First Grade group is below average in their general vocabulary knowledge

I would want to see if this is true in Kindergarten and Second Grade as well (on the School Status Report) Kindergarten Median Vocabulary = 29th percentile Second Grade Median Vocabulary = 41st

I might look to see how the Core Reading Program addresses vocabulary and if we have any additional programs addressing vocabulary (maybe we are using something different or additional at second grade?)

Questions to answer using the School Grade Summary Report What is the median score in the particular grade

level on particular score types? Median PRS in First Grade = 65% chance of success

What is the distribution of the ‘yellow’ zone (16-84%)? Of my 56% of First Graders who fall in the yellow zone,

are more of them at the high end or the low end of the zone? It is a relatively even distribution across the zone until we reach

65% and then we see that 26 students out of the 80 students in the yellow zone have a PRS of 65-74% chance of success

Resources & Conclusions

40

Web Resources FCRR (Florida Center for Reading Research), www.fcrr.org

Just Read, Florida!, www.justreadflorida.com

LEaRN (Literacy Essentials and Reading Network), www.justreadflorida.com/LEaRN

FOR-PD (Florida Online Reading Professional Development) http://forpd.ucf.edu/strategies/archive.html

Florida Response to intervention (RtI), www.florida-rti.org

Adolescent Literacy, www.adlit.org

Center on Instruction, www.centeroninstruction.org

LD Online, www.ldonline.org

Colorin’ Colorado, www.colorincolorado.org/

For questions regarding the FAIR system:

Visit the resources found on the FAIR website:

http://www.fcrr.org/fair/index.htm

Access the FAIR FAQ document found at the link below or accessed from the above website:

http://www.fldoe.org/faq/Default.asp?Dept=4&submit=GO

Contact: FAIRhelp@fcrr.org Please note the email address is not case sensitive

For more detailed information regarding the PMRN: The PMRN User’s Guides located at:

www.fcrr.org/pmrn/userguides.htm There is a section that provides an Overview of Reports and a section that

provides an Explanation of Score Types

Complete PowerPoints can be found on the PMRN website: www.fcrr.org/pmrn.htm Topics include:

K-2 Data Entry K-2 Electronic Scoring Tool 3-12 Web-based Assessment Module (WAM) School Level Users 1, 2, 3

Phone: (850) 644-0931 or 866-471-5019 Email: helpdesk@fcrr.org

Here is what you see if you click on Kindergarten, Phonemic

Awareness, Broad Screen/Progress Monitoring Tool

In summary, remember… FAIR was designed to inform instruction. FAIR data are just one part of the puzzle. Teachers

bring information from daily instruction that provides additional, important information.

As FCRR gains more experience with the data and collects data statewide, we will be able to provide guidelines on what are ‘typical’ scores at each grade level or what is a ‘good’ percentage to have in the high success zone (green zone).

We will make comparisons at the close of each assessment period to the State Median scores and percentages

47

Thank you for your time and attention!

Any Questions?