assessment: the shared data experience
DESCRIPTION
TRANSCRIPT
The Shared Data Experience How We Learn to Love Our Data!
Akron Ready Steps Assessment Presentation
AKRON, OHIP30 November 2009
Karen Burstein, PhD, MPHProject Evaluator
SWI 2
ALL SUCCESSFUL PROJECTS BEGIN AND END WITH THOUGHTFUL, ACCURATE ASSESSMENT.
3
Data Drives Our Lives
SWI
SWI 4
Teachers as Scientists
SWI 5
Rate Yourself- What data (or not) prompts you to choose a curriculum?
SWI 6
Evaluation as a Key Component
SWI 7
Let Your Application be Your Guide
SWI 8
Caveat 1
SWI 9
Caveat 2
SWI 10
SWI 11
So, Why Assess Children?
SWI 12
SWI 13
So, Why Assess Teachers?
SWI 14
SWI 15
Assessment drives the planning process;
which drives our teaching & instructional activities;
which are assessed for efficacy based on children’s performance.
AND THE CYCLE CONTINUES
SWI 16
Implications for Assessment
SWI 17
SWI 18
Reliability
SWI 19
Validity
SWI 20
Bias
SWI 21
Planning Annual ERF Assessments-
SWI 22
Set Up a System
SWI 23
Setup the Assessment Setting
A table for each assessment/ examiner
A place for children in between assessments
Limit visual and auditory distraction
Make sure teachers have comfortable chairs
Make sure that children’s feet are flat on the floor
SWI 25
Prepare a Dataset
SWI 26
Moving Children Through Assessments
Use the dataset to create name badges
A method by which to track the assessments given
A daily check sheet of assessments given
PROJECT RECOVERY DAILT ASSESSMENR LOG
NAME (pre-printed from dataset)
PPVT EOWPVT GRTR PALS TOPEL CAP
Amanda Smith KB/12-15-08 Stella Gales KB/12-15-08 Alisa Burroughs KB/12-15-08 Mary Smith KB/12-15-08
SWI 27
ARS Assessments
SWI 28
Kindergarten Screener
SWI 29
Kindergarten Screening Tool
• A screening tool is a short assessment of the presence of behaviors that is a reliable and valid indicator of a skill. A screening tool should be used to decide whether or not to take specific actions such as seeking referrals, further testing, and instructional activities. A test is a more rigorous and complex assessment of knowledge and skill with more extensive reliability and validity properties.
SWI 30
Why Screen?• Research shows that learning to read is a process that begins long
before children enter kindergarten. During the pre-kindergarten years, children develop the early literacy skills that help them to learn how to read during the first few years of elementary school. Early screening and intervention are the keys to overcoming reading difficulties and avoiding the problems that go along with them. Early literacy screening: Helps parents and early childhood professionals understand each child's progress towards developing literacy concepts and acquiring skills.
• Helps to identify developmentally appropriate experiences and teaching that can be used to support early literacy learning.
• Alerts early childhood professionals and parents to seek additional advice if a child does not appear to be making appropriate progress.
SWI 31
GRTR Screener
• Use this 20-question research-based screening tool with your 4-year-olds. The score will show if a child's pre-reading skills are weak, strong, or somewhere in between. And activities and resources to improve those skills will be provided.
• This tool is designed to screen a child twice during the year before kindergarten. Use the tool first in the fall one year before the child enters kindergarten, and again the next fall before kindergarten begins, to measure the child's progress. Don't use the tool more than three times in a year. It's not designed to measure small changes, and children develop new skills gradually.
SWI 33
SWI 34
Peabody Picture Vocabulary Test-IV
Measure of Receptive Language
Peabody Picture Vocabulary Test, Fourth Edition
By Lloyd M. Dunn, PhD and Douglas M. Dunn, PhD
SWI 36
SWI 38
TRAINING ITEMS Form B ages 4+
SWI 41
Assessment of Language and Literacy
Measure of Expressive Language, Phonological Awareness, Alphabet
Knowledge, Concepts of Print
SWI 42
SWI 43
• The Assessment of Literacy and Language (ALL™) aids in early detection of language disorders that could lead to reading difficulties. ALL assesses spoken language and written language skills, including:
• Listening comprehension• Language comprehension• Semantics• Syntax• Phonological awareness• Alphabetic principles/phonics• Concepts about print
With ALL, you can identify language disorders, language and emergent literacy deficits, emergent literacy deficits, and weak language and emergent literacy. View case studies to see how ALL can impact a child you know.
SWI 44
• Details the nature and severity of the problem• Aids in making recommendations for intervention• Provides information about where a child should be in his
or her reading and language skills based on grade level. Strategies for improving a child’s language skills in each of the areas assessed are included in the Examiner’s manual.The Parent Questionnaire provides additional information about a child’s language and literacy history and current skills.
• Norm-referenced scores• Criterion-referenced scores
SWI 46
GET It, Got It, Go!
Measure of Progress Monitoring
SWI 47
GGG (IGDI)
• Three subtests– Rapid Object Naming (1 minute)– Rhyming (2 minutes)– Alliteration (2 minutes)
SWI 48
Standardized Administration DOES Matter
• (These assessments are formally called Individual Growth and Development Indicators, or IGDIs.)
• Standardization of administration procedures, which includes: timing, appropriate prompting, administering sample items—is critical so that you get accurate results.
• By standardizing, we mean that IGDIs are administered in exactly the same way, for each child, across different assessment periods. This assures that IGDIs are administered accurately and reliably.
• Standardized administrations will minimize any systematic or unnecessary error. By standardizing, we are increasing the accuracy or reliability of our scores from one administrator to the next, from one child to the next, and for an individual child's scores from time a to b to c to d.
SWI 49
Each IGDI comes with a set of administration instructions.
• On the administration cards, bold print indicates exactly what the examiner should say.
SWI 50
Always Begin with Sample Items
• If the child meets the criteria indicated in the sample items, move to administration. Discontinue criteria are provided for the picture naming, rhyming, and alliteration measures. These criteria provide an indication of whether a child a). understands the task at hand and b). has sufficient skills for continuing with the administration.
• For example, the discontinue criteria for picture naming is as follows: The child is administered four sample cards (baby, bear, apple, cat). The administrator may stop administration if the child does not name all four of these pictures correctly.
SWI 51
Timing
• Each IGDI should be administered using a stopwatch according to the exact time specified.
SWI 52
Prompting
• Be Careful Not To Provide Corrected Feedback During Administration
• During IGDI "Administration" (not referring to sample card administrations) it is important not to prompt or provide feedback for correct or incorrect responses. Reinforcing statements such as "great," "good job," etc. are encouraged. However, feedback should not correspond to correct or incorrect responses made during administration (e.g., "that was right, " "that was wrong, etc)."
SWI 53
What Happens When Administration Is Not Standardized
"Melissa's" teacher administered the rhyming IGDI to Melissa on Oct. 14th and got a score of 7. She then administered rhyming again on Nov. 14th and got a score of 14. It appears that Melissa grew considerably in rhyming skill during this month. However, Melissa's teacher indicated that she didn't use a stopwatch. Instead, she used a clock with no second-hand during both administrations. She simply estimated 2-minutes time.
Here, we have a situation where we can not be sure whether Melissa actually grew in rhyming skill, since it may be that there was simply more time allotted to her during her second administration. Furthermore, lets say Melissa's teacher administered rhyming to all her students without timing exactly 2 minutes. In this case, we can not make accurate comparisons across students since maybe some student may have done better than others, simply because they were allotted more time.
SWI 54
GGG (IGDI)
• Picture Naming (Ages 3-5)• The format of this indicator presents a child with images of
objects commonly found in preschoolers' natural environments (i.e., home, classroom, community), one at a time, asking a child to name the pictures as fast as possible. Categories of objects used in this format include animals, food, people, household objects, games and sports materials, vehicles, tools, and clothing. Each photograph and line drawing is printed on an 8" x 5" index card. After providing a set of sample items, the examiner asks the child to look at each card and name it as quickly as possible. After exactly one minute, the examiner stops the activity and counts the total number of pictures named correctly.
SWI 55
Alliteration (Ages 3-5)
• We identified a set of words commonly known by preschoolers and obtained images of these words. Stimulus cards include one image at the top of each card (e.g., rain) and a set of three images in a row at the bottom of each card (e.g., house, rake, pig), one of which starts with the same sound as the target picture. After providing a set of sample items, the examiner asks the child to look at each card and point to one of the three pictures at the bottom of the card with the same initial sound as the fourth, target picture. The task continues for a total of two minutes. The score generated from this format is the number of pictures the child correctly identifies within two minutes.
SWI 56
Rhyming (Ages 3-5)
Stimulus cards include a target photo or line drawing at the top of each card (e.g., bees) and a set of three photos/drawings in a row at the bottom of each card (e.g., pants, gate, cheese), one of which rhymes with the target picture. After providing a set of sample items, the examiner asks the child to look at each card and point to one of the three pictures at the bottom of the card that sounds the same as (or rhymes with) the fourth, target picture. The task continues for a total of two minutes. The score generated from this format is the number of pictures the child correctly identifies within two minutes.
SWI 57
Useful Data
SWI 58
Bar Graph of Performance by Class
Class87654321
100
80
60
40
20
0
82.479.6
76
96.8
87
80.5
89.3
78.777.2
91.995
8584
RETURNING TO PRESCHOOL
GOING TO KINDERGARTEN
KINDER BOUND
Year 1 Baseline PPVT Standard Scores by Teacher by Kinder Status
SWI 59
Line Graph of Class Performance
Bailey
Blakely
Burton
CzattDarr
in Fike
Harris
Hughley
Jackso
n
Kitchen
Lurry
Mantin
McCray
Medina
Mettler
Nicol
Patters
on
Prosp
ero Ruiz
Seag
ers
24 2427
2017
2724
14
36
23
30
22
12
30
2025
22
31
22 20
26
15
24
1824
26 28
23
33
3632
23
19
30
26
29 34
32
29
21
GGG Picture NamingGGGPNCOR1 GGGPNCOR2
SWI 60
Sample Weekly Data
A Sample Graph with Aim and Performance Lines
0102030405060708090
100
w1 w2 w3 w4 w5 w6 w7 w8 w9
Week
Per
cen
tag
e o
f C
orr
ect
Res
po
nse
s Exceptionally Well
Typical (Aim Line)
At-Risk
Special Ed
SWI 61
Class 1 Class 2 Class 3Site
0.00
5.00
10.00
15.00
20.00GRTR Pretest
GRTR 2nd Test
. Get It Got It Go (N = 46, participating in both pretest and 2nd test)
SWI 62
Teacher ReportACE3 Fall 2006 Standardized Assessments
Agency: Teacher: School: Head Start
Last Name First Name PPVT PALS Letters Identified IPT
GGGPicture
Naming (#/95) GGG
Allit (#/45)GGG
Rhyme (#/52)Rapid Object
NamingGRTR
#/20
Dray 40 6 A,B,D,L,N,Y NE 4 0 0 6 0
Corine 74 0 LE 20 0 0 12 10
Alyssa 69 5 A,C,D,S,Y NE 10 0 0 3 0
Ulysses 58 2 L,U NE 8 0 0 3 0
Joey 86 6 A,B,D,E,G,Y LE 23 0 0 28 6
Brandon 69 2 A, B NE 6 0 0 7 0
Timothy 70 4 A,B,S,T LE 24 0 0 0 4
Anthony 72 5 A,C,G,E,M NE 8 1 0 8 0
Miguel 73 5 A,B,D,M,S LE 14 0 0 4 6
Kristofferson 75 1 X LE 21 0 0 12 10
Dale 77 9 A,B,D,G,I,L,M,O.R FE 13 0 0 38 13
Déjà 82 4 A,D,E,J LE 23 0 0 44 11
Jaden 71 5 A,R,D,N,T LE 15 2 0 6 0
Kai 80 2 B,G LE 21 0 0 31 7
Satori 74 2 X,S FE 29 0 0 42 11
Keanu 77 21 A,B,D,O,E,X,S,P,G,L,K,R,T,F,Z,U,H,M,Y,V,J, LE 29 0 1 38 15
Angel 81 12 A,B,D,O,E,X,S,C,G,R,U,H, LE 25 0 2 31 17
Isabelle 73 6 NE 7 0 0 6 0
Nina 65 NE8 9 0 0 8 0
Tyrus 89 7 B,X,N,T,H,J,Q FE 0 0 0 20 11
PPVT:PPVT Standardized Score
HSLN:Head Start Lower Case Letter Naming
IPT:Individual Proficiency Test of English Language(NE=Non-English Speaking; LE=Limited English Speaking; FE=Fluent English Speaking)
GGG: Get it Got it Go
GRTR:Get Ready to Read
SWI 63
Plan, Plan, Plan
SWI 64
Review, Review, Review
SWI 65
Celebrate Your Success (based on your data)
For more information please contact:• Karen Burstein, PhD, Evaluator 480-222-8800 x 1