florida assessments for instruction in reading (fair) : an overview and application within a...
TRANSCRIPT
Florida Assessments for Instruction in Reading (FAIR) : An Overview and
Application within a Response to Intervention (RTI) Model
Liz Crawford, MS, CCC-SLPDirector of Interventions
November 12, 2009IDA – Orlando, FL
Agenda Overview of FAIR
K-2 3-12
Progress Monitoring & Reporting Network (PMRN) Reports
Common Profiles/Patterns in Data Connecting to Instruction Response to Instruction/Intervention (RTI) Model RTI and Special Education Identification RTI in Florida RTI and FAIR
Why New Reading Assessments?
Primary-grade assessment improved to include: 1. PreK; 2. Vocabulary; 3. Comprehension
Assesses standards in K-12; monitors progress Results predict end of year performance on standardized test
(K-2) & Florida’s Comprehensive Assessment Test (FCAT) Diagnostic inventories are linked to Florida standards and
provide information for guiding instruction Reading comprehension & oral reading fluency passages are
equated for difficulty to allow for accurate progress monitoring and comparison of student performance from one passage to another
Comprehension assessed in instructional level passages
K-2: phonemic awareness, letter knowledge, decoding, encoding, fluency, vocabulary, listening or reading comprehension
3-12: adaptive, complex & low level reading comprehension, fluency, word analysis, skill assessment
Florida Assessments for Instruction in Reading:(Teacher Administered)
Comprehensive Assessment System
1. Phonemic Awareness
2. Phonics
3. Fluency
4. Vocabulary
5. Text Comprehension
6. Orthographic Skills (Spelling)
K-2 3-12
*
* embedded in text comprehension
K-2 Assessments
The K-2 “Big Picture” Map
Broad Screen/Progress Monitoring Tool (BS/PMT) “All” students
• Letter Naming & Sounds• Phonemic Awareness • Word Reading
Broad Diagnostic Inventory (BDI) “All” students“Some” students for vocabulary
• Listening Comprehension• Reading Comprehension• Vocabulary • Spelling (2nd grade only)
Targeted Diagnostic Inventory(TDI) “Some” students
• K = 9 tasks• 1st = 8 tasks• 2nd = 6 tasks
Ongoing Progress Monitoring(OPM)“Some” students
•K – 2 = TDI tasks•1 – 2 = ORF
7
The K-2 Broad Screen/Progress Monitoring Map
K-2 Broad Diagnostic Inventory (BDI) Map
Kindergarten • AP 1 = Listening Comprehension (LC) and Vocabulary (VOC)• AP 2 = Listening Comprehension• AP 3 = Listening Comprehension or Reading Comprehension (RC) and Vocabulary
First Grade • AP 1 = Reading Comprehension or Listening Comprehension and Vocabulary• AP 2 = Reading Comprehension• AP 3 = Reading Comprehension and Vocabulary
Second Grade • AP 1 = Reading Comprehension, Vocabulary, and Spelling (SPL)• AP 2 = Reading Comprehension and Spelling• AP 3 = Reading Comprehension, Vocabulary, and Spelling
K-2 Targeted Diagnostic Inventory (TDI) MapKindergarten • Print Awareness
• Letter name and sound knowledge• Phoneme Blending• Phoneme Deletion Word Parts/Initial• Letter Sound Connection Initial• Letter Sound Connection Final• Word Building –Initial Consonants• Word Building –Final Consonants• Word Building –Medial Vowels
First Grade • Letter Sound Knowledge • Phoneme Blending• Phoneme Deletion Initial• Phoneme Deletion Final• Word Building –Consonants• Word Building –Vowels• Word Building –CVC /CVCe• Word Building –Blends
Second Grade • Phoneme Deletion Initial• Phoneme Deletion Final• Word Building –Consonants• Word Building –CVC /CVCe •Word Building –Blends & Vowels• Multisyllabic Word Reading
Targeted Diagnostic Task Example Word Building Tasks are given at each grade
level.
Grade 1 Vowels. Student moves letters to makenew words like dig, dug, pug, pig, and pit.
Practice and Task Items
u p t a e i
dog
The K – 2 “Score” MapBS/PMT PRS = Probability of Reading Success
BDI LC = Listening Comprehension Total questions correct (implicit/explicit)
RC = Reading Comprehension Total questions correct (implicit/explicit), Fluency, Percent Accuracy Target Passage
VOC = Vocabulary Percentile Rank
SPL = Spelling Percentile Rank
TDI ME = Meets Expectations
BE = Below Expectations
OPM ORF = Adjusted Fluency
OPM TDI Tasks = ME or BE and Raw Score 12
Probability Score – Probability of Reading Success (PRS) Score calculated based on the BS/PMT
Probability is…the relative possibility that an event will occur or is likely to occur.
A .70 means that we expect this student to have a 70% chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10.
13
Success Zones - Probability Scores
GREEN 85% or better chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10
16-84% chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10
15% or less chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10
14
YELLOW
RED
3-12 Assessments
Why use a computer adaptive test? What are the benefits?
A computer adaptive test (CAT) provides a more reliable assessment of student ability in a shorter amount of time than a traditional test, because it picks questions at a level of difficulty that are most informative for each student.
Computer adaptive tests provide more reliable assessments particularly for students at the extremes of ability (extremely low ability or extremely high ability).
17
The 3-12 “Big Picture” Map
Type of Assessment Name of Assessment
Broad Screen/Progress Monitoring Tool (BS/PMT) – Appropriate for ‘All’ students
• Reading Comprehension (RC)
Targeted Diagnostic Inventory(TDI) – “Some” students
• Maze• Word Analysis (WA)
Ongoing Progress Monitoring (OPM) – “Some” students
• Maze• ORF• RC
Informal Diagnostic Toolkit(Toolkit) – “Some” students
• Phonics Inventory• Academic Word Inventory• Lexile Passages• Scaffolded Discussion Templates
18
The 3-12 “Big Picture” Map
Type of Assessment Name of Assessment
Broad Screen/Progress Monitoring Tool (BS/PMT)
• 3 times a year• On the computer• Entered into PMRN
Targeted Diagnostic Inventory(TDI)
• 3 times a year • On the computer• Entered into PMRN
Ongoing Progress Monitoring (OPM)
• Maze and RC – on computer, up to 7 times a year, entered into PMRN• ORF – paper and pencil, up to 7 times a year, entered into PMRN
Informal Diagnostic Toolkit(Toolkit)
• Paper and Pencil only• Not entered into PMRN• No restrictions on when to administer
Broad Screen/Progress Monitoring Tool
Reading Comprehension Task
(3 Times a Year)
Targeted Diagnostic Inventory
Maze & Word Analysis TasksDiagnostic
Toolkit(As Needed)
OngoingProgress
Monitoring(As Needed)
If necessary
Grades 3-12 Assessments Model
Purpose of Each 3-12 Assessment RC Screen
Helps us identify students who may not be able to meet the grade level literacy standards at the end of the year as assessed by the FCAT without additional targeted literacy instruction.
Mazes Helps us determine whether a student has more
fundamental problems in the area of text reading efficiency and low level reading comprehension.
Word Analysis Helps us learn more about a student's fundamental
literacy skills--particularly those required to decode unfamiliar words and read accurately.
3-12 Targeted Diagnostic Inventory (TDI) Map
Moderate or Low Success
Probability on FCAT=
Administer TDI
Word Analysis TDI
Intervention withword recognition skills?
Intervention with reasoning skills, prior knowledge,
and/or vocabulary?
Maze TDI
Interventionwith fluency/
comprehension skills?
The Broad Screen/Progress Monitoring Tool: The RC Task 10-30 minute computer-adaptive task of reading
comprehension A student receives a minimum of one passage and a
maximum of three passages Predicts student’s FCAT success probability (FSP) Administered to all students scoring Level 1 or 2 on FCAT Administered to 3rd grade students identified with a reading
deficiency by their districts Other students may take the assessment at the discretion of
their districts
TDI: The Maze Task Two 3-minute (grade-level) passages
Administered 3 times a year
Student responds to embedded cloze items within text
Assesses text reading efficiency (i.e., reading accuracy and speed, and gist - level comprehension)
TDI: The Word Analysis Task
A 5-15 minute computer-adaptive spelling task that assesses a student’s understanding of letter/sound correspondence
A student receives a minimum of 5 words and a maximum of 30 words
Results indicate whether probability of success in reading is hampered by difficulties with word-level skills.
33
The 3-12 “Score” Map
Reading Comprehension - BS/PMT
FCAT Success Probability (FSP) Color- coded
Percentile Standard Score Lexile Ability Score and Ability Range FCAT Cluster Area Scores
Maze - TDI Percentile Standard Score Adjusted Maze Score
Word Analysis - TDI Percentile Standard Score Ability Score (WAAS)
OPM RC – Ability Score, Ability Range, Cluster Scores Maze – Adjusted Maze Score ORF (3rd – 5th) Adjusted Fluency Score
Ongoing Progress Monitoring (OPM)
Ongoing Progress Monitoring (OPM)K-2 All of the TDI tasks are available as OPM probes ORF in grades 1 and 23-12 ORF at grades 3-5 MAZE at grades 3-12 RC at grades 3-12 Informal toolkit with:
Scaffolded Discussion Templates Lexile® Passages Phonics Inventory Academic Word Inventory
The purpose of frequent progress monitoring
Progress monitoring during the year is used to determine whether or not students are responding adequately to their current instructional environment.
If their growth within their current environment is determined to be inadequate, they should experience an instructional change designed to increase their rate of progress.
There are different types and purposes for progress monitoring, which call for a range in the frequency of administration (e.g., weekly, monthly).
Issues involved in deciding on the frequency of progress monitoring
Setting a reasonable interval for progress monitoring actually depends on three pieces of information about the reading assessment being used:
1. How much growth on a given outcome (i.e., oral reading fluency) can be reasonably expected during a given interval of time?
2. How reliably does the assessment measure the outcome of interest?
3. How closely are the different probes of the ability equated for difficulty?
General issues about the frequency and utility of different kinds of progress monitoring assessments
Measures like oral reading fluency are referred to as “general outcome” curriculum based measures because they assess student proficiency on a general outcome of interest.
They have the advantage of being able to determine growth rates for individual students that can be compared against a desirable standard of growth over time.
Thus, they can indicate whether or not an intervention is sufficiently powerful to “close the gap” on the outcome measure over a reasonable period of time.
Although they provide information about the necessity for a change in a student’s instructional environment, they do not necessarily provide information about how the program should be changed in order to increase student performance.
General issues about the frequency and utility of different kinds of progress monitoring assessments
Information that can be used to suggest ways that instruction should be changed is more likely to be provided by assessments that are very closely tied to the specific curriculum that is being used to guide instruction.
This type of assessment, frequently referred to as “classroom-based formative assessment”, should be given very frequently to assess whether the student is learning what is being taught.
Although this type of assessment has immediate value for making day-to-day adjustments to instruction, it has less value as a measure of general progress in learning to read, since it represents such a small part of the overall reading process.
Examples of “general outcome” curriculum based measures Oral Reading Fluency (ORF)
OPM Maze Task
OPM Reading Comprehension
OPM TDI Tasks – OPM*
*These tasks form a developmental continuum where 80% mastery is expected. Thus, these are similar to what many teachers might use for classroom-based formative assessment.
Examples of “classroom-based formative assessment” Unit Tests
Weekly Quizzes
Writing Samples
Informal Inventories
In summary… General outcome measures should not be given more
frequently than is necessary to establish a reliable estimate of growth.
Giving them more frequently than that takes time away from instruction unnecessarily.
In contrast, teachers should be constantly (hourly, daily, weekly) acquiring formative assessment data about their student’s progress in mastering their specific lesson objectives.
References for OPM Jenkins, J. R., Graff, J. J., & Miglioretti, D. L. (2009).
Estimating reading growth using intermittent CBM progress monitoring. Exceptional Children, 75, 151-163.
Use of ongoing progress monitoring to improve reading instruction. (2009). A guidance document prepared by the Florida Center for Reading Research (FCRR).
Torgesen, J. K., & Miller, D. H. (2009). Assessments to guide adolescent literacy instruction. Portsmouth, NH: RMC Research Corporation, Center on Instruction.
PMRN Reports
K-2 and 3-12
What is an AHM?The Active Header Menu (AHM) is found at the top of most reports. It allows the User to navigate between reports and filter the data that is displayed.
In the report below, it is AP3 and the teacher is filtering her second grade class so that the report only displays those students who have been in her class all year. It also provides links to legends that help explain the report and a link for “printer friendly” version of the report.
School Status Report (K-2)
Target Passages for Grades 1 and 2 – BDI - Reading Comprehension
©2009 Florida Center for Reading Research
School Status Report (3-12)
School Progress Report (K-2 example)
Bar Graph View of Data
School Progress Report (K-2 example)
Tabular View of Data
School Grade Summary Report (K-2)
School Grade Summary Report (K-2), - continued
School Grade Summary Report (K-2)
School Grade Summary Report (3-12)
School Grade Summary Report (3-12)- continued
School Grade Summary Report (3-12)
School Demographic Report (K-2 example)
School PRS Distribution Graph by Ethnicity (K-2 example)
District and School Data File The data file allows all of the data to be downloaded so that Users can
complete their own analyses. If the PMRN does not have a report comparing the groups of interest, then the
data file allows a school or district to download the data for analysis using other systems such as Excel or SPSS.
The file can be downloaded at the district or school level. The file is a text file.
Each school has its own file and it can be downloaded by the SL1, SL2, SL3, and SL4 Users at the school (up to 7 people at the school have access).
The process for accessing the file is described at http://www.fcrr.org/pmrn/download2access_excel.htm.
Student Ongoing Progress Monitoring (OPM) ORF Report
Grades 1-5
©2009 Florida Center for Reading Research
62
Instructional Changes-Pop up
Class Status Reports and Student Score Detail Boxes
K – 2 and 3 – 12
Class Status Report (K-2)-
©2009 Florida Center for Reading Research
Student Score Detail Box (K-2)-
©2009 Florida Center for Reading Research
Class Status Report- (3-12)
Student Score Detail Box-
(3-12)
Some Common Profiles or Patterns in Data
K-2 Class Status Report Worksheet – data analysis questions
How is comprehension impacted?
Assessment/Curriculum Decision Tree for Reading Improvement for the Florida Assessments for Instruction in Reading
72
Common Profiles/Patterns of Performance
73
Common Profiles/Patterns of Performance
Box 1
Box 2 + 4
Box 2 + 5
Box 3 + 4
Box 3 + 5
74
Steps to the Activity
Sort your students according to the profiles presented and write their names down on the worksheet under the appropriate profile.
After your students are sorted into profiles, answer the following questions: What profiles most frequently occurred in your student
data? For which profiles do you have resources currently in
place? In which areas do you need more professional development/support/resources to meet student needs?
Based upon the frequency of various profiles of student needs, will you address these through adjustment of Tier I, Tier II, or Tier III?
75
Assessment/Curriculum Decision Tree for Reading Improvement Grades 3-12 for the Florida Assessments for Instruction in Reading Worksheet - Handout 8
©2009 Florida Department of Education
Connecting Assessment to Instruction
Links to Instructional Resources
77
Linking to Resources through PMRN
Teachers will have the option of clicking on column headers for: K-2
RC, LC, Vocabulary, Spelling, and TDI Tasks 3-12
RC, Maze, WA These links will take them to a search page which will in turn bring
them to a list of resources designed to assist instruction in the chosen area.
Resources include K-2: Empowering Teacher Routines and Student Center
Activities 3-5: Empowering Teacher Routines and Student Center
Activities K-12: LEaRN videos and other LEaRN resources
You can also get to some of these same resources through a search tool located on FCRR home page (www.fcrr.org)
Search Tool for FCRR resources
For Student Center Activities (SCAs) for K-5 and for Empowering Teachers (ET) Routines K-3
There are two search tools: one links to FAIR and one links to DIBELS Searchable by grade level and then
5 components and then skills SSS and then enter standard number DIBELS subtest or FAIR The skills on the class status report worksheet match up
with the skills on the search tools under each component of reading instruction!
78
Parent Letters
You may access the Parent Letters in the PMRN in two different ways: From the Class Status Report click on the link at
the bottom of the page that says Print all Parent Letters – this will print letters for the whole class
Users can get to the Parent Letters under the Student Reports tab – this will allow the teacher to print only letters for specific students.
79
Response to Instruction/Intervention
What is RtI? (NRCLD)National Research Center on Learning Disabilities
“ RTI is an assessment and intervention process for systematically monitoring student progress and making decisions about the need for instructional modifications or increasingly intensified services using progress monitoring data.” pg. i.2
What is RtI? (NASDSE)National Association of State Directors of Special Education
RtI is the practice of (1) providing high-quality
instruction/intervention matched to student needs (2) using learning rate over time and level of
performance to (3) make important educational decisions.
Essential Components Implementation (NASDSE)
Multi-tier model of instruction Problem solving method An integrated data collection/assessment
system
Components 1 & 2: Tiered Model of School Supports & the Problem-Solving Process
ACADEMIC and BEHAVIOR SYSTEMS
Tier 3: Intensive, Individualized, Interventions.
Individual or small group intervention.
Tier 2: Targeted, Supplemental Interventions & Supports.
More targeted interventions and supplemental support in addition to the core curriculum and school-wide
positive behavior program.
Tier 1: Core, Universal Instruction & Supports.
General instruction and support provided to all students in all
settings.
Essential Component 3:Integrated Assessment Systems (NASDSE)
• Directly assess specific skills in standards• Assess “marker variables” [demonstrated to lead to
the ultimate instructional target, (e.g., reading comprehension)]
• Sensitive to small amounts of growth• Brief• Repeatable• Easy to use• Direct relationship to instructional decision- making
RtI and Special Education Identification
Sec. 614 (b) (6)
Specific Learning Disabilities (A) IN GENERAL
“…a local educational agency shall not be required to take into consideration whether a child has a severe discrepancy…”
Sec. 614 (6) Specific Learning Disabilities
(B) ADDITIONAL AUTHORITY
“…a local educational agency may use a process that determines if the child responds to scientific, research-based intervention as a part of the evaluation procedure…”
Process refers to “Problem Solving Process” Responds refers to “Response to Intervention”
Problems with the traditional system “Wait-to-fail model” (Vaughn and Fuchs 2003; NASDSE)
Late identification for students in need Imprecise screening through teacher observation False negatives (unidentified students) who don’t
get the services they need Identification measures that are not linked to
instruction Separation of general and special education Over representation of some minority groups Undocumented benefits
Special Education Eligibility Comparison of historical system to RtI system
See NASDSE documents See Vaughn and Fuchs article, 2003
It is not about getting them in, it is about getting them the help they need.(“If I could just get my child into special education everything would be fine.” – a parent)
Specific Learning Disabilities (SLD) Rule in Florida
Rule Implementation Synopsis Status
Adopted by SBE on January 21, 2009 Effective as of March 23, 2009
Implementation SP&P identifies applicable process school-by-school
A. RtI Process within Comprehensive Evaluation OR B. RtI Process and relevant pattern of strengths and
weaknesses within Comprehensive Evaluation Effective July 1, 2010, schools can no longer require
pattern of strengths and weaknesses as allowed in option B
96
Florida SLD Criteria for Eligibility(until July 1, 2010)
Underachievement in:Oral expression
Listening comprehensionWritten expressionBasic reading skills
Reading fluency skillsReading comprehensionMathematics Calculation
Mathematics problem-solving
RtI & Pattern of Strengths/Weaknesses:
RtI & pattern of strengths and
weaknesses relevant to identifying SLD
Conditions 1 and 2 not primarily the result of:
Visual, hearing or motor disability
Intellectual disabilityEmotional/Behavioral
disabilityCultural factors
Irregular attendanceEnvironmental or
economic disadvantageClassroom behavior
Limited English proficiency
Condition 1 Condition 2 Condition 3
+ +
RtI:Insufficient response to
scientific, research-based intervention
or
Florida SLD Criteria for Eligibility(after July 10, 2010)
Underachievement in:Oral expression
Listening comprehensionWritten expressionBasic reading skills
Reading fluency skillsReading comprehensionMathematics Calculation
Mathematics problem-solving
RTI:Insufficient response to
scientific, research-based intervention
Conditions 1 and 2 not primarily the result of:
Visual, hearing or motor disability
Intellectual disabilityEmotional/Behavioral
disabilityCultural factors
Irregular attendanceEnvironmental or
economic disadvantageClassroom behavior
Limited English proficiency
Condition 1 Condition 2 Condition 3
+ +
Collection of Current Technical Assistance
99
NEW SLD Web link on the BEESS Web site featuring related rules, technical assistance,
documents, and Web sites:
http://www.fldoe.org/ese/sldr.asp
What does RtI look like in Florida?
www.florida-rti.org
FAIR and RTI
109
RtI and K-2 Florida Assessments for Instruction in Reading (FAIR)
Broad Screen/Progress Monitoring Tool
(BS/PMT)
“All” students
Tier 1
Broad Diagnostic Inventory
(BDI)
“All” students
“Some” students for vocabulary
Tier 1 and 2 Sometimes Tier 3
Targeted Diagnostic Inventory
(TDI)
“Some” students
Tier 2 and 3
Ongoing Progress Monitoring
(OPM)
“Some” students
Tier 2 and 3
109
Guiding Questions for Determining Target Skill(s) How well is core instruction meeting the
needs of our students in the areas of: Phonemic awareness? Phonics/Word Analysis? Word Reading? Reading connected text accurately? Vocabulary? Comprehension?
School Status Report (K-2)
112
RtI and 3-12 Florida Assessments for Instruction in Reading (FAIR)
Broad Screen/Progress Monitoring Tool (BS/PMT) – Appropriate for ‘All’ students
Tier 1
Targeted Diagnostic Inventory
(TDI) – “Some” students
Tier 2 and 3
Ongoing Progress Monitoring
(OPM) - “Some” students
Tier 1 – RC OPM Tier 2 and 3
Maze and ORF OPM
Informal Diagnostic/Progress Assessment Toolkit
“Some” students
Tier 2 and 3
School Status Report (3-12)
School Grade Summary Report (3-12)
School Grade Summary Report (3-12)- continued
116
Common Profiles/Patterns of Performance
Questions??