© caroline gooden, kentucky early childhood data system, 2007 kentucky gseg general supervision...

© Caroline Gooden, Kentucky Early Childhood Data System, 2007

Kentucky GSEGGeneral Supervision Enhancement Grant


The GSEG Outcomes and Process Overview of Continuous Assessment Guide

and implications for First Steps How KEDS is working for Preschool Behind the Scenes – Ensuring valid and

reliable data


Two Major Goals

Pilot the Standards and Continuous Assessment Guide to Determine….

– how well CBA address the standards and outcomes– how well children with disabilities fit into the framework– supports needed to implement in programs state-wide

Pilot the 3 National Early Childhood Outcomes for Part C and 619 (Preschool) of IDEA


Continuous Assessment Guide

Diagnostic section designed to support Evaluation activities as outlined in IDEA with respect to eligibility– Norm-referenced standardized instruments (908

KAR 2:130 & 635 of IDEA)



Classroom/Instructional section designed to support Assessment activities as outlined 908 KAR 2:130 with progress monitoring and program planning

– Criterion referenced or curriculum based tools– “an ongoing process of observing a child's current

competencies (including knowledge, skills, dispositions and attitudes) and using the information to help the child develop further in the context of family and caregiving and learning environments”



Purpose of Classroom/Instructional Assessments

1. Identify the individualized needs of children to inform intervention and instructional planning,

2. Develop individualized plans (e.g., IFSPs)

3. Inform families and other team members of child developmental status.


Why CR/CB over Norm-Referenced

Gather information for all the major developmental domains or dimensions of development.

Tools were reviewed for their ability to:– Be developmentally and culturally valid; – Yield comprehensive and useful information – Allow for multiple methods from multiple sources without jeopardizing

reliability or validity – Connect to important learning represented in the Kentucky EC Standards,

Provide outcomes that match family goals and cultural preferences.

It is widely accepted that the tools included in the screening and diagnostic assessment section of CAG do not provide

information sufficient for the purposes of classroom/instructional program planning and this is supported

in 908 KAR 2:130.


Objectives of GSEG

1. Determine current use of the standards and assessment guide

2. Help programs develop assessment systems for data on outcomes

3. Help programs use assessment data for child and program improvement

4. Document assessment process for replication


GSEG Pilot Sites

Anderson Co. Preschool– Rural public preschool center– ~ 300 children ages 3 and 4

Growing Together – Urban private early childhood center– ~180 children, ages 0-5 years

2 Rural/urban early intervention providers– ~20 children, ages 0-3

All serve children with & without disabilities


Kentucky’s Approved Assessment Tools

Preschool Child Observation Record (COR)

Child Observation Record for Infants & Toddlers (COR I-T)

Creative Curriculum OUNCE Scale Work Sampling System (WSS) Brigance Inventory of Early

Development-II (IED-II) Assessment, Evaluation, &

Programming System (AEPS) Early Learning Accomplishment

Profile (E- LAP)

Learning Accomplishment Profile–3 (LAP–3)

Transdisciplinary Play Based Assessment (TPBA)

Hawaii Early Learning Profile (HELP)

Carolina Curriculum for Preschoolers with Special Needs ( CCPSN)

Carolina Curriculum for Infants and Toddlers with Special Needs (CCITSN)


Kentucky’s Approved Assessment Tools

Preschool Child Observation Record (COR)

Child Observation Record for Infants & Toddlers (COR I-T)

Creative Curriculum OUNCE Scale Work Sampling System (WSS) Brigance Inventory of Early

Development-II (IED-II) Assessment, Evaluation, &

Programming System (AEPS) Early Learning

Accomplishment Profile (E- LAP)

Learning Accomplishment Profile–3 (LAP–3)

Transdisciplinary Play Based Assessment (TPBA)

Hawaii Early Learning Profile (HELP)

Carolina Curriculum for Preschoolers with Special Needs ( CCPSN)

Carolina Curriculum for Infants and Toddlers with Special Needs (CCITSN)


GSEG Activities

Year 1 (2004-05) – Training on standards and assessment guide – Selection of assessment tools– Training on assessment tools by publishers

Year 2 (2005-06) – Collection of data on all children served by pilots – Training on assessment tools– Developing data platform for correlations to standards and

to OSEP child outcomes


What is KEDS?

Kentucky Early Childhood Data System

Provides a consistent platform for collection of data from:– Diverse programs and providers across the state– Diverse numbers of approved assessment tools


Why KEDS?

Supports improved instruction through ongoing, continuous assessment– Gathering information in the context of everyday

routines and activities to obtain a representative picture of children’s abilities and progress

Improves outcomes for children Documents child outcomes for Office of

Special Education


How Does Continuous Assessment Work for Early Intervention Providers

Embedded with RBI Supported by the

Consultative Model Requires ongoing

communication among team members



Providers continue to record observations, field notes, gather “evidence” following a home visit

Providers continue to communicate with other members of the child’s team and family members on progress (IFSP goals and outcomes)

Providers record mastery of skills as appropriate on assessment protocol.


Example with Carolina Curriculum



Providers record mastery of skills as appropriate on assessment protocol.


How KEDS was Designed for Preschool

Ongoing meetings with OSEP, ECO and KDE staff (accountability and early childhood) to determine: – how the system should integrate with state data

system– who should enter data and at what level– what formats should be used by those in the field– how training and support will be provided for

assessments and data transmission


How KEDS works for Preschool

Log In Accounts created at two levels

Preschool Coordinators have access to all children in their program

Teachers have access to children on their caseload



Child level data is imported into the system from KDE

STI – child tracking system



Preschool Coordinators establish the teacher accounts and assign children to the teacher


How KEDS Works for Preschool

Teachers conduct quality assessments on each student:

Teachers tally data twice a year– November– May

Coordinators monitor teacher assessments and data entry for accuracy


How KEDS Works for Preschool

Teachers verify imported data and complete additional demographic data on each child


Verification of Data

Child Unique Identifier Number Date of Birth Name Gender Ethnicity IEP Enrollment Date Exit Date District School


Additional Data

Does child have an IEP – Developmental Delay, Speech/Language, or Severe

Child Status– at-risk eligible (up to 150 % of poverty)– over income– LEP

Assessment used with child and format of the assessment (on-line, diskette, or paper-and-pencil)


Assessment Recording Formats

Online systems - direct administrative access is provided to UK

Diskette or CD systems - electronic submission of information via Keds Online

Paper-and-pencil versions– Publisher permission to establish data entry

system– Transmit data via Excel via Keds Online


Sample Paper – Pencil Entry System


What Happens Behind the Scenes - Methodology

Align individual assessment items to KY standards and benchmarks then to OSEP Outcomes

Activity– Conceptual framework– 4 phase validity process


KY’s Conceptual Framework


Benefits of the System

Allows flexibility for providers to choose a tool that best meets the needs of individual children (with state guidance)

Puts the focus on implementing high quality assessments that provide immediate data to support intervention

Immediately responsive to changes in required outcome reporting at the federal level

Allows for reporting across multiple programs


How Does KEDS Work?


4-Phase Validation Process

Expert Panel – General cross walk of all CB assessment tools to benchmarks and

outcomes– Detailed cross walks of all assessment items to benchmarks for

preschool tools Statistical analysis of child data on benchmarks, standards and

OSEP outcomes (Correlational and Factor Analysis) for three pilot assessments

Concurrent validity studies of 2 assessments using BDI-II Teacher/Provider ratings and rankings of child progress on

standards, benchmarks and OSEP outcomes


Sample CrosswalkArts & Humanities Standard 1: Participates and shows interest in a variety of visual art, dance, music and drama experiencesBenchmark 1.1: Develops skills in and appreciation of visual arts

Developmental Continuum

Example Behaviors COR CC WSS LAP-3

AEPS Brigance

CCPSN

Help

1)Uses a variety of media and materials for exploration (e.g. paint, glue, three-dimensional materials, technology, etc.).2)Uses a variety of art forms, elements and materials for representing people, places, and things in the environment.3)Observes and responds to artwork produced by other individuals and/or cultures.

1. Kenny chooses to paint at the easel three days in a row. He paints with one color, covering the entire page.2. Sally uses finger paint to make swirls on black paper. 3. Shavon uses glue and ribbon on paper.4. The teacher puts Q-tips at the art table and Monica dips them in paint to make spots on paper. 5. Ben uses a computer program to create a picture then glues on a tree-twigs picture frame after printing the picture.1. Olivia uses markers to draw eyes, nose, and a mouth on a paper plate.2. Jarrad paints a picture of his dog at the easel.3. Maggie sticks leaves she has gathered outside to a tree trunk shaped from play-dough.4. Trina builds her house with Popsicle sticks and glue.5. Kyle draws a picture of his brother, including facial features, hair, arms/hands, legs/toes, and a belly button.1. Katie watches a classmate making a snake out of clay and then makes one herself.2. Brian looks intently at the picture of the farm. He says, "The horses are running and the sheep are eating the grass."3. Michelle comments that some of the trees in the photograph are green and some are orange.4. After the teacher reads the story about Native American mask making, Mark paints a Native American mask using watercolors.

I.1J.1, 2l.2-5J.3-5X.2None

None37. F through IIINone

NOTE:FOR ALL WSSITEMS, ALL ITEMS REFER TO 3’S AND 4’S UNLESS OTHERWISE NOTEDVI. A.3NoneVI. B.1

FM 4, 9, 10, 16, 19, 25, 30, 32, 35, 39;PW 1, 2, 6, 8, 13, 16, 17FM 34PW 14, 29None

NoneFM B 2.1None

NoneC-3.6, 3.7None


Statistical Validation of Items

Data reduction– Based on crosswalk analysis, single items pulled together

to create scales, a confirmatory factor analysis conducted to ensure items are measuring the same dimension

Establishing reliability– Are the scales measuring the content of the benchmark with

repeated use? Ensuring internal consistency

– Do the scales from the crosswalk “hang” together? Predictive validity

– Regression analysis



Conduct sample tests*– Measures of central tendency– Outlier analysis

Test individual assessment tools– Measures of central tendency– Outliers within tools– Investigate age bands - If no age bands are available

for individual assessments, investigate continuum* No group norms are available for B-3 tool selection, therefore in depth analysis of the

sample for B-3 will be the main focus



Convert individual assessment scores to common metric

– Allows for comparison between assessments that was not possible before conversion

Conversion formula

– Conversion based solely on the individual assessment tool,

and once converted, can be compared to any other z score– Investigate the relationship of items and scales to KY

benchmarks

x - µZ = σ



Align KY benchmarks to OSEP outcomes Report entry data on Outcomes based on

OSEP’s five category reporting system


Final Conversations and Questions

Have the funding to support the system design for First Steps and other CHFS early childhood programs

Need to complete the validation work with 0 – 3 assessment tools

Need to identify specific program issues and considerations for implementation, specifically….


How KEDS can be designed to support Early Intervention

Continued meetings and discussions with OSEP, ECO and EI staff to determine:

1. How the system should integrate with state data system – specifically CBIS

2. Who should enter data and at what level

3. What formats should be used by providers

4. How training and support will be provided – have trainers available for most assessments

5. Data transmission processes – have basic systems in place