first 5 · 2012. 10. 5. · page 4 task 6 data collection management system, outcomes collection,...

The meeting is voice recorded in its entirety. A CD will be available for checkout from the First 5 Sacramento Commission offices at 2750 Gateway Oaks Dr., Suite 330, Sacramento, the day after the meeting.

The on-line version of the agenda and associated materials are posted for your convenience at

http://www.first5sacramento.net/default.htm. Some documents may not have been posted on-line because of their size and/or format. As they become available, hard copies of all documents are available from the Clerk of the

Commission at the First 5 Sacramento Commission offices. of 1

FIRST 5 Sacramento Commission

EVALUATION COMMITTEE 2750 Gateway Oaks Drive, Suite 330

Sacramento, CA 95833

AGENDA

Monday September 20, 2010 1:00 PM Members: Robert Bonar, Scott Moak, Marilyn Ratkay Advisory Committee Members: Terry Jones, Betsy Uda Staff: Toni Moore, Carmen Garcia-Gomez Consultants: Fred Molitor, Lisa Branton, Gary Resnick Clerk: Cheryl Johnston _______________________________________________________________________

1. Approve August 16, 2010 Draft Action Summary (5 minutes)

2. Commission Staff Update (5 minutes)

3. WRMA Monthly Progress Report (5 minutes)

4. Discussion: Evaluation of New Dental Contractors (5 minutes)

5. School Readiness Evaluation Report for the Fiscal Years 2008 – 2010 (30 minutes)

6. CBI Evaluation Results for Fiscal Year 2009 - 2010 (10 minutes)

7. Update: Smile Keepers Data Collection (10 minutes)

8. Committee Member Comments (5 minutes) a. Miscellaneous b. Future Agenda Items

9. Public Comments on Non-Agenda Matters (5 minutes)

Walter R. McDonald & Associates, Inc.

Contract #06/07-ADM-031

First 5 Sacramento Evaluation

Monthly Progress Report for:

August 2010

September 8, 2010

Prepared for:

First 5 Sacramento Commission

Prepared by:

Walter R. McDonald & Associates, Inc.

2720 Gateway Oaks Dr., Suite 250


[email protected]

916-239-4020, ext. 246

mailto:[email protected]

First 5 Sacramento Evaluation Contract

SECTION I: INTRODUCTION, PURPOSE AND SCOPE

The Sacramento County First 5 Commission has contracted (Contract #06/07-ADM-031) with

the team of Walter R. McDonald & Associates, Inc. and Harder+Company Community Research

for evaluation services during the 2007-2010 period. For this deliverables-based contract, our

team submits monthly progress reports as a key communication tool for all concerned

stakeholders. In the monthly progress report we identify the status of major deliverables in the

contract. We also preview deliverables that are expected to be submitted in the subsequent month

and identify key questions or challenges that need to be resolved.

For this work period, our primary activities related to Tasks 5 and 6.

SECTION II: WORK PLAN

The work under this contract is organized according to the following tasks.

Task 1 Strategic Planning Activities

Task 2 Evaluation Design, Scope and Report

Task 3 Funding Processes

Task 4 Contract Negotiations

Task 5 Evaluation Support, Planning, Collecting and Reporting

Task 6 Data Collection Management System, Outcomes Collection, Analysis and

Reporting

Task 7 First 5 Sacramento Commission Evaluation Annual Report

Task 8 State Evaluation and First 5 California Annual Report

SECTION III: PROGRESS Task 1 Strategic Planning Activities

Scheduled Period of Activity: Phase 2/Year 3 & 4

Deliverables: Monthly Progress Reports

Status: Completed

Task 2 Evaluation Design, Scope and Report

Scheduled Period of Activity: Phase 1/Year 1 & Phase 2/Year 2 & 3

Deliverable 1: Draft Evaluation Design Report

Deliverable 2: Final Evaluation Design Report

Deliverable 3: Evaluation Design Report (Year II)

Deliverable 4: Evaluation Design Report (Year III)

Status: Completed

Task 3 Funding Processes

Scheduled Period of Activity: Phase 2/Year 2

Deliverables: Monthly Project Reports

Status: Completed

Task 4 Contract Negotiations

Scheduled Period of Activity: Phase 1/Year 1

Deliverable: Completed


Scheduled Period of Activity: Phase 1/Year 1 & Phase 2/Year 2, 3 & 4

Deliverable: Monthly Progress Reports

On August 11 WRMA and First 5 Sacramento staff met to discuss the 2009/10 annual reporting

requirements for First 5 California, including the transfer of data from Persimmony to First 5

California and the required analyses of Persimmony and Parent Interview data.

The August 19 School Readiness Coordinators’ meeting included a presentation on the findings

from the Preschool Bridging Model from the report “Evaluation of School Readiness Services:

2008 to 2010.” The meeting also included a lengthy discussion on the burden to school staff

related to recruitment for Smile Keepers services.

The Data Manager participated in the Sacramento Native American Health Center, Inc. contract

negotiation meeting on August 19, and the contract negotiation meetings with The Effort on

August 19 and 31.

On August 19, the Project Director and Data Manager participated in a conference call with

Linda Fong-Somera, Jennifer Auld, and Martha Haas of CPS Hearts for Kids program. The focus

of the discussion was on the number of children who have been brought into CPS upon exit from

the hospital, and therefore did not need a Hearts for Kids medical clearance exam. Martha Haas

confirmed that the required demographic information for these children will still be entered into

Persimmony along with services other than the medical clearance exams.

We conducted the first wave of sampling for the 2010 Parent Interview in August. A total of 150

mothers were sampled from the Nutrition Result Area. Given the relatively few families from

Health Access entered into Persimmony every month, we will select all available families until

100 interviews are completed. As such, in August 170 families were sampled from Persimmony

(Table 1).

Table 1. Status of Families in Persimmony and Sampling for the 2010 Parent Interview

Result Area Entered into

Persimmony

Sampled on

8/18/10

Available on

8/23/10

Effective Parenting 36 0 36

Health Access 20 20 0

Nutrition – Breastfeeding 276 150 126

Nutrition – Childhood Obesity 0 0 0

School Readiness 58 0 58

Total 390 170 220

Recruitment packets (recruitment letter, consent form, return postage-paid envelope, and

immunization card) were mailed to all 170 parents on August 23.

On August 20 we had a meeting with Carmen Garcia-Gomez, Verronda Moore, and Gina

Roberson to request that we receive the data export for Family Support Collaborative in

September (for clients recruited in July and August) rather than October. This would allow us to

sample one month earlier than we would be able under the previous quarterly schedule. Gina

agreed to make the related request of LPC staff, who agreed to provide us with these data in

September. On August 23 the Data Manager sent an email to all contractors reminding them of

the need to enter clients in Persimmony as soon as possible so that we can sample and interview

parents as soon as possible from the date of consent. As such, we are undertaking a number of

strategies to interview parents as soon as possible.

Our preparation for the 2010 Parent Interview to begin September 7, 2010 also included updating

all field interviewer training materials, advertising for new field interviewers, screening and

interviewing applicants, and hiring new bilingual Spanish and Russian/Ukrainian interviewers.

Contract negotiations have been completed for all contractors except The Effort and Sacramento

Native American Health Center Inc. We continue to work on developing milestones, conducting

consent and intake trainings, and drafting Contractor Evaluation Plans. Table 2 outlines the

status of these efforts through the end of August.

Table 2. Status of Developing Milestones, Conducing Trainings, and Contractor Evaluation

Plans (CEPs) for New Contract Cycle

Contractor

Milestones Trainings CEPs

Drafted Review Finalized

Consent/ Intake

Training Data Entry

Training Drafted Review Signed/

Received

Early Care

Child Action x x x x

x x x

Dental

CPS x x x n/a x x x x

Smile Keepers x x

x

x x

The Effort

Sacramento Native American Health Center Inc.

Health Access

Cover the Kids x x

x x x x x

HKHF x x x x x x x x

Effective Parenting

Family Support Collaborative x x

x x x x x

DHHS CPS - Effective Parenting

x x

n/a x x x x

Sacramento Children's Home - Crisis Nurseries

x x x x x x x x

Nurse Family Partnership x x

x x x x x

Nutrition- Obesity

Health Education Council x x

x x x x x

Nutrition- Breastfeeding

DHHS WIC x x x x x x x x

Task 6 Data Collection Management System, Outcomes Collection, Analysis and

Reporting

Scheduled Period of Activity: Phase 1/Year 1 & Phase 2/Year 2, 3 & 4

Deliverables: Monthly and Quarterly Progress Reports

On August 18 the Data Manager met with Carmen Garcia-Gomez, Cathy Spivey, Preschool

Bridging Model (PBM) staff (Ginger Swigart and Natalie Woods Andrews), and Child Action,

Inc. staff (Jaci White, Lina Harris, and Anthony Garcia) to discuss how environmental

assessment data will be collected during the current contract cycle. At this meeting all parties

agreed that the best method for obtaining environmental assessment data from PBM site will be

for PBM staff to continue reporting to WRMA the environmental assessment scores on all sites.

PBM staff will now include the names of the providers of each classroom assessed; in other

words, the center name and each of the teachers for that classroom will be provided with the

classroom score and assessment date. PBM staff also agreed to provide Child Action, Inc. with a

list of the provider names to enable them to indicate that the provider is a PBM participant when

Child Action, Inc. staff enter the relevant data into CARES through Persimmony.

The Data Manager provided on-site technical assistance to the Nurse Family Partnership on

August 26 and installed Citrix on computers (used to view Persimmony) for new data entry staff

and answered questions related to consent and intake.

The following trainings were provided by the Data Manager in August:

August 20 – consent/intake training, Smile Keepers program – SETA.

August 20 – consent/intake training, Smile Keepers program – Twin Rivers USD.

August 21 – consent/intake training, River Delta USD.

August 26 – client-level and milestone data entry training, San Juan USD.

August 30 – client-level data entry training, CPS.

Task 7 Annual First 5 Sacramento Evaluation Report

Scheduled Period of Activity: Phase 2/Year 3 & 4

Deliverables 1: Annual First 5 Sacramento Evaluation Report (Year 3)

Status: Completed


Status: Completed

Task 8 State Annual Report

Scheduled Period of Activity: Phase 2/Year 2, 3 & 4




Status: Completed

SECTION V: WORK TO BE PERFORMED IN THE NEXT REPORTING PERIOD


Key Tasks:

- Continue to work on the 2009/10 annual report data.

- September Evaluation Newsletter.

Retrieve consent forms.

Task 6 Data Collection Management System, Outcomes Collection, Analysis

and Reporting

Key Tasks:

- Provide additional Persimmony training to contractor

SECTION VI: KEY CHALLENGES OR QUESTIONS TO BE RESOLVED

- None to report.

EVALUATION NEWSLETTER August 2010

THE EVALUATION TEAM:

The evaluators of First 5 Sacramento represent a team from Walter R. McDonald & Associates, Inc. (Sacramento)

and Harder+Company Community Research (Davis). The evaluation team is dedicated to assessing the impact of

First 5 Sacramento services on families throughout Sacramento, while minimizing the amount of time that

contractors have to spend on evaluation activities. Please call us if you have any questions about the evaluation of

First 5 Sacramento services: WRMA 916.239.4020; Harder+Company 530.757.8420.

74%

63%58%

80%

64%

92% 93%

82%

96%

84%

Writing Center

Math Area Science Area

Art Area Place for Child to

Play Alone

Before PBM Services After PBM Services

School Readiness Evaluation 2010

The evaluation of the second year of the current

cycle of School Readiness services is completed.

It included looking at School Readiness services

to children, parents, and families in six school

districts and services to teachers/providers

involved in the Sacramento County Office of

Education Preschool Bridging Model (PBM).

We have recently completed the report which

explores changes in child outcomes between the

2008/09 and 2009/10 school years, changes in

parent outcomes between fall and spring, overall

outcomes in relation to services, district

teacher/classroom outcomes, and changes in

PBM teacher/classroom outcomes before and

after receiving services.

New PBM Evaluation in 2010

The pre-post PBM evaluation is new to this

year’s report. Last year, PBM teachers/providers

piloted the Teacher/Provider Survey, the

instrument used in the current PBM evaluation.

Ginger Swigart, the PBM School Readiness

Coordinator, provided feedback and helped

refine the questionnaire which included adding

questions regarding the knowledge and

participation of CARES and follow-up questions

about teachers’/providers’ enrollment in

professional development trainings and higher

education coursework. The Teacher/Provider

Survey was used to collect information regarding

teacher/provider beliefs, classroom practices,

and professional development.

PBM Pre-Post Evaluation: PBM Early

Childhood Education Specialists work with

privately-owned early childhood educators from

preschools, child care centers, and family homes

to improve the quality of education and care

provided to children. Ninety teachers/providers

completed the Teacher/Provider Survey both

before and after receiving PBM services.

PBM Outcomes: Positive changes were reported

by teachers/providers receiving PBM services.

There was a significant increase in teacher

related training enrollment and in the number of

teachers’/providers’ knowledge of CARES

(Figure 1).

Figure 1: Changes in PBM Teacher/Provider Education and Professional Growth, Fall 2009 to Spring 2010

Teachers/Providers also reported a significant

increase in the types of learning centers they

provide to children in their classroom (Figure 2),

suggesting PBM services increased the quality

of education and care provided to children. Figure 2: Significant Changes in Reported Types of PBM Site Learning Centers, Fall 2009 to Spring 2010

These and other PBM findings are included in

the upcoming Evaluation of School Readiness

Services 2008 to 2010 report.

24%

67%71%

93%

Before PBM Services

After PBM Services

Enrolled in Teacher Related Training

Heard of CARES

FIRST 5 SACRAMENTO COMMISSION

To: Evaluation Committee Members

From: Fred Molitor Walter R. McDonald & Associates, Inc. Date: September 8, 2010

Subject: Evaluation of New Dental Contractors

The May 17 Evaluation Committee meeting included presentation and approval of the specific items in the revised Parent Interview instrument to address ongoing and new Result Area and Cross-Cutting indicators. As discussed at this meeting, responses from the Parent Interview will continue to be the main source of data to address most Result Area indicators, including, “Percent of children who have seen a dentist by one year of age” among families recruited through the Health Access Result Area. Excluded from our memo and discussion on May 17 were the new contractors funded under the Dental Result Area who will provide direct services to children. The purpose of this memo is to revisit the Parent Interview as a possible means of collecting data to address the Dental indicator, now that client-level data will be available for families receiving services directly through this Result Area. Sacramento County Child Protective Services. The rationale for assessing the Dental indicator for this contractor is tenuous given the services they provide and the ability to obtain signed consents. The Heart for Kids project rarely involves contact with parents; in the few cases where CPS staff do interact with parents, it is unlikely that they would be willing to sign the consent. The current Contractor Evaluation Plan (CEP) calls for CPS staff to collect and enter into Persimmony demographic and service information for the child only, not the parent. These data will be entered into Persimmony at the client-level, but will not include names or other identifying information. The Effort and Sacramento Native American Health Center, Inc. Unlike the previous funding cycle, we now have contractors funded under the Dental Result Area who are providing dental services. With client-level data available from these contractors, we could take a sample of families and recruit them for the Parent Interview. However, the services these contractors provide are directly related to the indicator. As such, it does not make sense to ask a sample of parents to recall when and which types of services their children received when we will have more accurate information from all children receiving these services. Our recommendation is to exclude these families from the Parent Interview. Smile Keepers. All children receiving Smile Keepers services will be older than one year of age, and thus it does not make sense to sample Smile Keepers families to collect data for the Dental Result Area indicator. We could include Smile Keepers families in the Parent Interview to address the related Cross-Cutting indicator, “Percent of children who have seen a dentist in the past year.” However, this would in effect amount to evaluating one contractor rather than services provided within a Result Area,

of 2 C:\Documents and Settings\fmolitor\Local Settings\Temporary Internet Files\Content.Outlook\BHKRN8S0\Memo to Evaluation Committee RE Eval of Dental Contractors 05Sept10FMmr.doc

plus we would have to re-allocate a proportion of our Parent Interview sample size from the other Result Areas. The related benefit does not outweigh the costs of reducing statistical power for the analyses of Result Area indicators for the Health Access, Nutrition, Effective Parenting, and School Readiness Result Area.

Prepared for: First 5 Sacramento Commission

2750 Gateway Oaks Drive, Suite 330 Sacramento, CA 95833

Prepared by: Walter R. McDonald & Associates, Inc. 2720 Gateway Oaks Drive, Suite 250


Evaluation of School Readiness Services 2008 to 2010

DRAFT Not for distribution or citation

First 5 Sacramento DRAFT—Not for distribution or citation

ii


Table of Contents Executive Summary………………………………..………….……...1 Chapter 1: Introduction……………………………………….……………...5 Chapter 2: School Readiness Services at School Districts………………....……....11

Chapter 3: Changes in Children Outcomes……………………………….…….17 Chapter 4: Changes in Parent Outcomes……………………………….….…...21 Chapter 5: Services Related to Outcomes…………………………….....……..23 Chapter 6: Teacher/Classroom Outcomes: School Districts……………..…..…….27 Chapter 7: Teacher/Classroom Outcomes: Preschool Bridging Model…….....….......33

Conclusions…………….....………………………………………..…..….39 Appendix…………………..………………………………………..…..….43

iii


iv


Executive Summary

1 In fiscal year 2008/09, 36.8% of First 5 Sacramento program expenditures supported School Readiness services, followed by Effective Parenting services

(28.0%) and lactation support services and childhood obesity prevention interventions through the Nutrition program area (12.0%). The remaining program areas are Health Access, Early Care, Community Building, and Dental.

1

The First 5 Sacramento Commission places a high priority on School

Readiness; services to prepare children to enter kindergarten “ready to

learn” receive the largest proportion of funds available to Sacramento

County through the California Children and Families Act (the 1998

Proposition 10 tobacco tax) of the seven funded program areas. 1

School Readiness programs are currently implemented at nine

Sacramento County school districts with services offered to parents and

teachers as well as to children. These services range from multi-session

parent workshops, to preschool, to providing families with information

and referrals on topics such as immunizations and health insurance.

During the last two years, 6,677 children and 4,056 adults, or a total of

5,928 families, received School Readiness services through the funded

school districts.

School Readiness services are also provided to staff at privately-owned

early childhood programs through the Sacramento County Office of

Education, which implements services using the Preschool Bridging

Model (PBM). PBM services include on-site mentoring and coaching

support as well as instructional materials to teachers and providers with

the goal of enhancing the quality of child care to better prepare children

for elementary school. During 2008/09 and 2009/10, the PBM provided

services at 200 child care sites across Sacramento, reaching over 2,500

children.

The variation and complexity of First 5 Sacramento School Readiness

necessitated different evaluation methodologies and outcome data to

assess those services delivered in area school districts versus those

available through early childhood programs.


School Readiness Services Delivered through Sacramento Schools The evaluation of School Readiness services provided through school districts over the last two years included

collecting data on parent and child outcomes using three different instruments. The Parent Survey included items to

assess changes over time in areas such as parents’ involvement with their children and levels of stress. The Teacher

Child Report assessed social, emotional, and motor development among a sample of pre-kindergarten-aged children.

These same children also participated in the Child Assessment, a standardized assessment of language and cognitive

skills administered by trained school staff.

Analyses of the data from these instruments included examining:

Child outcomes between the 2008/09 and 2009/10 school years;

Changes in parent outcomes within the school years; and

Services in relation to child and parent outcomes.

Aggressive, hyperactive,

and withdrawn behaviors

significantly declined

Early math skills and language understanding significantly

increased among children

assessed in Spanish

Motor development and levels of

attention/persistence

significantly increased

Key Findings Child Outcomes Between 2008/09 and 2009/10 school years:

Parent Outcomes

From fall to spring during the school years:

Parents reported a significant increase in

the number of activities with children, such as

talking about what happened in school or helping the child learn

letters or numbers

Parents reported significantly

lower levels of stress

2


Services Related to Outcomes

Parent Discussions & Engagement services

bring together groups of parents for facilitated

discussions on topics such as child

development, the importance of family liter-

acy, and activities to support cognitive

development.

Literacy Programs include events for fami-

lies that emphasize the importance of reading

at home; time is provided for parents and chil-

dren to read together. Services include English

as Second Language (ESL) workshops.

Transition Activities include family visits to

the kindergarten; workshops to familiarize

children with the classroom, teacher, and

kindergarten activities; and school registra-

tions and orientations for parents on school

readiness and expectations for kindergarten.

Letter recognition and,

Language understanding.

Higher literacy scores among children, and

Lower levels of stress

and greater levels of emotional support among parents.

Parent Discussions & Engagement services were

linked to:

Literacy Programs and Transition Activities were

related to improved:

A fourth evaluation instrument called the Teacher/Provider Survey was used to assess teacher/classroom outcomes at

selected schools providing education to children whose families enrolled in School Readiness services. For this com-

ponent of the evaluation, data were collected to assess the quality of education and care offered to children receiving

School Readiness services.

Teacher/Classroom Outcomes

High quality services are being delivered to children and parents. Parent involvement appears relatively high; both in terms of the types of contact that teachers report having with parents as well as the different ways in which parents are involved in the classrooms. Teachers use a variety of instructional modalities in developmentally-appropriate durations with children. Children are frequently exposed to learning activities aimed toward language development.

Teachers provide developmentally-appropriate classroom activities and materials to children. Teachers reported developmentally-appropriate beliefs and attitudes about best practices with young children.

Among Spanish-speaking children:

3


School Readiness Services Delivered through Child Care Sites

The Teacher/Provider Survey was completed by the same 90 staff at child care centers and family home providers at

the beginning and after receiving PBM services. In this case the instrument was used to assess changes over time in

teacher/classroom outcomes as a result of First 5 Sacramento-funded services.

Our evaluation revealed that PBM services were related to:

4

An increase in the overall quality of the

early childhood programs as

derived from a number of

standardized measures

An increase in teacher/provider

enrollment in professional development

trainings

An increase in parent

involvement in the classroom

An increase in the time spent on

activities shown in studies to be related

to children’s cognitive and social-

emotional development

Increases in the percent of sites with

areas specifically designated for learning about writing, math,

science, and art


Chapter 1 Introduction

In accordance with the California Children and Families Act (approved

by California voters in 1998 under Proposition 10) First 5 programs are

funded through the statewide, 50 cent tax on all tobacco products. State

and county First 5 Commissions have a history of prioritizing these

funds to support School Readiness programs. School Readiness services

focus on preparing children academically, socially, and emotionally for

kindergarten.

In recent years an increased emphasis has been placed on children’s

social and emotional development in School Readiness programs.

Educators agree that it is best to address problematic behaviors such as

aggression and hyperactivity as soon as possible (ideally before

kindergarten entry); otherwise these behaviors persist and become

extremely disruptive to the learning environment in kindergarten and

beyond.

School Readiness programs also often include services to identify and

address health issues that could interfere with children’s success in

kindergarten. School Readiness services are delivered at schools, but

also through child care centers and family child care providers. Finally,

successful School Readiness programs provide services to parents as

well as to children ages 0 to 5 years.

5


Fiscal year 2009/10 represented the second year of the

current cycle of School Readiness services. During the

year, services were available within nine Sacramento

County school districts (Table 1.1).

Elk Grove Unified School District Folsom Cordova Unified School District Galt Joint Union Elementary School District Natomas Unified School District River Delta Unified School District Robla Elementary School District Sacramento City Unified School District San Juan Unified School District Twin Rivers Unified School District

For the last two years the Sacramento County Office

of Education has implemented School Readiness

services designed to enhance the quality of child care.

This program, titled the Preschool Bridging Model

(PBM), supports a team of Early Childhood

Education Specialists to work directly with teachers

and providers at preschools, child care centers, and

family child care providers to provide consultation,

instructional resources, and environmental quality

improvements.

The nine school districts and the PBM each have a

“School Readiness Coordinator” who oversees the

programs and functions as a liaison between First 5

Sacramento and the staff providing the services.

School Readiness Coordinators’ responsibilities

include managing all aspects of program

implementation. They provide budget oversight and

monitor outcomes to ensure that services are of

sufficient quality and having the desired impacts.

They also work directly with the evaluation team

from Walter R. McDonald & Associates (WRMA)

and Harder+Company Community Research to

implement the evaluation of First 5 Sacramento

School Readiness services.

Workshops

Literacy Programs

Screenings

Information, activities, and

materials such as backpacks and school supplies

Pre-kindergarten Summer Camps

Increase parent knowledge and encourage positive

parenting practices

School Readiness programs in Sacramento County

represent the First 5 Sacramento Commission’s largest

investment and are supported, in part, by state-match

funds.

Sacramento School Readiness services include:

Promote reading, writing, and

language development

Identify speech, development, vision, oral,

and general health problems

Prepare children and parents for kindergarten

entry

Expose children with little or no preschool experience to a classroom environment

and school activities

Table 1.1: First 5 Sacramento School Readiness School Districts, 2009/10

6


recognize (a) all the letters of the alphabet, (b) most of

them, (c) some of them, (d) none of them” and “Can

this child write (his/her) first name even if some of the

letters are backward?”

The Child Assessment documented children’s language

and cognitive skills through a standardized assessment

administered by school staff other than the selected

children’s primary teachers.

The Parent Survey asked parents about various

activities conducted with the child, and included items

to measure levels of social support and perceived

stress.

Finally, the Teacher/Provider Survey assessed teacher

and classroom outcomes in nine areas. Table 1.2 lists

the specific outcomes pertinent to the child, parent, and

teacher/classroom.

Evaluation Methodologies and Instruments

The evaluation of First 5 Sacramento School

Readiness services during fiscal years 2008/09 and

2009/10 was quite complex. It involved four different

instruments to collect data on children, adults, and

teacher/provider staff from school districts and early

childhood programs.

Data were also collected at two points in time from

the same groups of parents and teacher/providers at

child care sites to assess changes over time in

important outcomes. Finally, school staff both

recruited parents to participate in the evaluation and

recorded the services that each family received.

These data allowed us to conduct client-level

analyses examining specific services in relation to

child and parent outcomes. The details of the overall

evaluation are best appreciated by examining the

methodology used to assess outcomes from services

provided at funded school districts separately from

the methods used to evaluate PBM services.

Evaluation of Services Delivered Through Schools

Evaluation data came from parents, children, and

teachers at the participating school districts2 using

four separate instruments.

The Teacher Child Report recorded children’s social

and emotional status with a series of questions from

which overall composite scores were developed in

the areas of Aggression (based on four items),

Hyperactivity (three items), Withdrawn behaviors

(seven items), Motor Development (three items), and

Attention/Persistence (nine items). A series of five

questions were also used to develop an Overall

Literacy score, and included items such as “Can child

2 Three school districts were not funded to begin services until 2009/10 and thus were not included in this evaluation: Galt Joint Union Elementary School

District, Natomas Unified School District, and River Delta Unified School District.

7


Table 1.2: Child, Parent, and Teacher/Classroom Outcomes by Evaluation Instrument for First 5 Sacramento School Readiness Services Delivered Through Schools, 2008 to 2010

Data for the Teacher Child Report and Child

Assessment were collected in the spring of each

school year.

Changes in child outcomes between the two cohorts

of children (2008/09 versus 2009/10 school year)

were analyzed and are presented in Chapter 3 of this

report.

During each school year the parents of randomly

selected families completed the Parent Survey twice,

once in the fall and then again in the spring. Overall

changes in parent outcomes between these two points

in time during which families received School

Readiness services are presented in Chapter 4.

The evaluation findings presented in Chapters 3 and

4 do not take into account the specific services each

School Readiness family received. Of all services

provided at school sites over the two-year period of

this report (as presented in Chapter 2), those services

considered to potentially impact child and parent

outcomes in the short term were examined and are

presented in Chapter 5.

Chapter 6 presents teacher/classroom outcomes

related to a series of quality indicators such as

teacher/providers’ levels of training and continuing

education in early childhood education, and parent

involvement in the classroom. This and other

information collected on the Teacher/Provider

Survey were used to assess the degree to which

children in First 5 Sacramento School Readiness

programs have received high quality services

representative of those considered to be “best

practices.”

The findings presented in Chapters 3 through 6 of

this report came from individuals selected through

probability sampling techniques. Random samples of

families in the fall of 2008/09 and 2009/10 formed

8

Instrument

Child Outcomes

Teacher Child Report

Aggression

Hyperactivity

Withdrawn

Motor development

Attention/Persistence

Overall literacy

Child Assessment

Early math skills

Letter naming

Language understanding

Receptive vocabulary

Parent Outcomes

Parent Survey

Minutes spent reading to child in past week

Activities with child in the past week

Activities with child in the past month

Emotional support

Parental stress


Teacher education and professional growth

Teacher involvement with parent

Parent involvement in classroom

Minutes of instructional modality

Frequency of learning activities

Type of learning centers in classroom

Absenteeism

Teacher’s beliefs and attitudes about best practices Process quality in classroom

Teacher/Provider

Survey


Figure 1.1: Criteria for Selection for Participation in Evaluation of First 5 Sacramento School Readiness Services, 2008/09 and 2009/10

the foundation for subsequent data collection

activities in each year, as depicted in Figure 1.1.

Parents from these families were asked to participate

in the Parent Survey in the fall and spring. Pre-

kindergarten children (ages 4 or 5 years) from these

families were selected for the Child Assessment; and

their teachers recorded their behaviors using the

Teacher Child Report. Finally, the teachers/providers

of children whose parents completed the fall Parent

Survey participated in the Teacher/Provider Survey.

Evaluation of Services Delivered Through Child Care Sites

The Early Childhood Education Specialists, providing

PBM services in 12 Sacramento County school

districts, asked the teachers and providers to complete

the Teacher/Provider Survey at two points in time.

Teacher / Provider Survey Completed by Teachers

Families Randomly Sampled

Child Assessment Administered to 4 and 5 Year Old Children

Teacher Child Report Completed by Teachers of 4 and 5 Year Old

Children

4 and 5 Year Old Children from Parents who

Completed the Parent Survey

Pre Parent Survey Completed by

Parents Post Parent Survey Completed by

Parents

Teachers of Children 0 to 5 Years Old Whose Parent Completed the

Parent Survey

SPRING FALL

9



Teacher/Provider Survey

Teacher education and professional growth

Teacher involvement with parent Parent involvement in classroom

Minutes of instructional modality

Frequency of learning activities

Type of learning centers in classroom

Process quality in classroom

The teacher/classroom outcomes for the evaluation of

PBM, as presented in Table 1.3, represent most of those

appearing in Table 1.2.

The difference in the evaluation methodologies has to

do with how comparisons were made. Outcomes

assessed on the Teacher/Provider Survey completed by

school staff were compared with data representative of

best practices for early childhood education instruction.

Alternatively, the teacher/provider outcomes presented

in Chapter 7 are compared before and after PBM

services were provided to assess the potential impact of

these services.

Table 1.3: Teacher/Classroom Outcomes by Evaluation Instrument for First 5 Sacramento School Readiness Services Delivered Through Child Care Sites, 2008 to 2010.

10


Chapter 2 School Readiness Services at

School Districts During 2009/10, a total of 3,515 children and 2,268 parents received

School Readiness services, equating to 3,034 School Readiness families

for the year. These counts represent increases in the number of children

(3,162), parents (1,788) and families (2,894) documented as School

Readiness clients during 2008/09.3 These increases are due in part to the

three new school districts funded to begin services on July 1, 2009, but

mostly to the original six school districts providing services to more

families.

This chapter presents information on the various types of School

Readiness services delivered to children, parents, and families during

the year. When reviewing the three tables that present this information

it should be kept in mind that not all services were available in all

school districts, and some services were available only to children

within certain ages. For example, Preschool services were available in

three of the six school districts and Playgroups are designed for children

younger than 4 years.

3 Annual counts of children and adults are duplicative; numbers represent children and parents receiving services within a given year and some clients received

services in both fiscal years.

11


Services to Children School Readiness services for children consist of either

multi-session, structured learning opportunities offered

in classroom settings or screenings to identify and then

address health and developmental problems. During the

2008/09 and 2009/10 school years, roughly one-in-10

School Readiness families had a child in Preschool and

just under one-fifth of families received Summer Camp

services (Table 2.1). Health Screenings were provided

to children in just over one-fourth of School Readiness

families; the proportion of families receiving Speech/

Language & Development Screenings and Oral Health

Services, which include assessments and treatment, are

also presented in Table 2.1.

Table 2.1: First 5 Sacramento School Readiness Services for Children, 2008/09 and 2009/10

Service Name Description of Service

Families Receiving Service

2008/09 n=2,894

2009/10 n=3,034

Health Screenings*

Comprehensive health screenings for children addressing: medical, developmental, vision, hearing, oral health, and immunizations. Some individual hearing, vision, and/or immunization screenings for children. Oral health screenings include dental varnishes, and all screenings result in referrals for follow-up when needed.

26.8% 29.0%

Oral Health Services* Oral health screenings and/or cleanings and varnishes for children. 5.3% 10.9%

Preschool Preschool for children ineligible for other preschool programs due to family income. Services include social, emotional, physical, language development, and learning activities.

9.4% 9.8%

Speech/Language & Development Screenings

Speech/language or developmental screenings, or behavioral and developmental assessments including diagnosis for children. 16.3% 17.2%

Summer Camp

Pre-kindergarten camps offered for 4-6 weeks over the summer, provide exposure to a classroom setting and kindergarten concepts such as numeracy, literacy, and social interaction; intended for children with little or no preschool experience.

15.8% 19.7%

Services to Families Literacy Programs deal specifically with reading,

writing, and language development, emphasizing the

importance of “parents as their child’s best first

teacher,” and reading to their child, as well as

providing a location, the time, and materials for parents

to read to their children. This service was provided to

the largest proportion of families of all services

delivered to children and parents at the same time

(Table 2.2). Moreover, during the 2009/10 school year,

nearly half (47.3%) of all School Readiness families

were exposed to Literacy Programs. Transition

Activities were received by just under one-third of

families during both school years. Transition Activities

optimize the potential for kindergarten success by

12

*Dental varnishes provided with the oral health screening listed in Health Screenings cannot be separated from the other health screenings provided based upon how the data were tracked and recorded. Oral Health Services are not included in Health Screenings.


Table 2.2: First 5 Sacramento School Readiness Services for Families, 2008/09 and 2009/10



2008/09 n=2,894

2009/10 n=3,034

Health Insurance Referrals or Information

Screenings and referrals for health insurance, or receipt of health insurance information. 3.7% 4.4%

Home Visits

Education on positive parenting practices, child development, and/or the importance of family literacy; a needs assessment to provide case management and connect families with additional services, such as for special needs children; and/or supplying literacy materials/kits.

5.3% 5.7%

Literacy Programs

Literacy programs include family night events to discuss family literacy and the importance of reading at home; parents and children to read and talk together; library time for parents and their children; English as Second Language (ESL) workshops for parents to affect family literacy; and/or a supplement to preschool for Hmong speaking children/parents. Another program is curriculum based including two components: workshops for parents discussing the importance of literacy and take-home book bags for children to read with their parents.

40.6% 47.3%

Playgroups

Playgroups for children and their parents teaching and encouraging developmentally-appropriate parent-child interactions; some provide age-specific Learning Toolkits with tools and materials for parents and child.

12.3% 12.8%

Speech/Language & Development Interventions

Intervention services include speech/language consults; direct support to parents for developmental and/or speech/language needs of their child; developmental play therapy for the child; and/or follow-up linkage to special needs services.

13.3% 5.0%

Transition Activities

Activities include visits to the kindergarten for children; workshops to familiarize children with the classroom, teacher, and kindergarten activities; school registrations and orientations for parents on school readiness and expectations for kindergarten; parent exposure to the kindergarten classroom, and the school district/system; and/or parent-teacher meetings and opportunities to meet the principal.

30.9% 30.2%

Transition Materials

Backpacks with school readiness information, supplies, and activities; and/or kits with information on child development, parenting, nutrition, dental care, and community resources for families with children transitioning into kindergarten.

16.7% 42.6%

13


introducing families to school processes, expectations,

and staff. Both Literacy Programs and Transition

Activities are directly in line with the overall goal of

School Readiness: to prepare children and parents for,

and smooth the transition to, kindergarten.

The proportion of families receiving each type of

School Readiness services remained fairly stable

between the two years, except for Speech/Language &

Development Interventions and Transition Materials

services. Transition Materials provide pre-kindergarten

children with backpacks containing supplies they need

for school success, and include important information

for their parents on child development, nutrition, and

community resources. There was a substantial increase

in the proportion of families receiving this service in

2009/10.

Home Visits involved a number of services for the

family, including education on positive parenting

practices and child development, developmentally-

appropriate activities that can be done with the child,

and case management for referrals for additional

services needed. Home Visits were provided to roughly

five percent of School Readiness families, and Health

Insurance Referrals and Information to about four

percent of families.

Services to Parents

Certain types of School Readiness services directed at

parents require quite a degree of commitment among

those who agree to participate. These include Parent

Discussions & Engagement services, which involve

facilitated discussions with groups of parents, and multi

-session Parenting Instructional Workshops, which rely

on structured curricula of instruction (Table 2.3). In

2008/09, 637 families participated in one or both of

these services; in 2009/10, one or both parents from

579 families received these services.

Less intensive School Readiness services but still of

great importance include the distribution of Health

Information on topics such as immunizations and

mental health (and making referrals when needed).

Very few Lactation Support Referrals for expecting or

new mothers were provided in 2008/09 and none

occurred in 2009/10. Classes with a focus on Education

on Child Development, Education on Children’s Early

Academic Skills, and Education on Children’s

Nutrition and Health began in the 2009/10 school year

through two of the three 2009 expansion school

districts – the Galt Joint Union Elementary School

District and the Natomas Unified School District.

14


Table 2.3: First 5 Sacramento School Readiness Services for Parents, 2008/09 and 2009/10



2008/09 n=2,894

2009/10 n=3,034

Education on Child Development

Classes for parents focused on the brain, motor, social/emotional, and language development of a child. -- 1.1%

Education on Children’s Early Academic Skills

Classes for parents focused on early math and literacy skills of children, including examples of what to work on with their child to prepare for kindergarten.

-- 0.4%

Education on Children’s Nutrition and Health

Classes for parents focused on children’s nutritional and overall wellness needs. -- 1.5%

Health Information Information for parents on mental health, oral health, and/or immunizations, and some referrals for children’s mental or oral health follow-up.

9.9% 6.3%

Lactation Support Referrals

Through home visitation, referrals for lactation support to pregnant mothers or mothers with newborns. 0.4% --

Parent Discussions & Engagement

Topical discussions with groups of parents and facilitator including: child development, behavior, attachment, parenting, ‘male involvement,’ mental/emotional health, physical health, kindergarten readiness, importance of family literacy, "parents as their child's best first teacher," and activities to support cognitive development. Parent engagement in governance groups such as advisory committees or school site councils to participate in school-based decision making.

13.1% 8.9%

Parenting Instructional Workshops

Workshops for parents utilizing curriculum to increase dialogue and positive interaction between parent and child, and/or learn how to manage children's emotional responses.

8.9% 10.2%

15


16


Chapter 3 Changes in Children Outcomes

17 17

Data to assess children outcomes came from two sources: the Teacher

Child Report and the Child Assessment.

The Teacher Child Report collected information about children’s

problem behaviors, as observed by their teachers. Aggression was

assessed with items such as “disobeys rules or requests” and “hits or

fights with others.” Statements to measure Withdrawn behavior

included “hard to understand what he or she is saying,” “keeps to

himself or herself,” and “lacks confidence in learning new things or

trying new activities.” Items to quantify Hyperactivity included “is

very restless,” “fidgets all the time,” and “can’t sit still.”

The Teacher Child Report also included an Attention/Persistence

dimension based on nine statements, and a three-item Motor

Development dimension. Attention/Persistence is related to children’s

ability to sustain attention and their diligence toward accomplishing

challenging tasks (e.g., “child sticks to an activity for as long as can

be expected for a child of this age”). Motor Development items

included “Can this child button (his/her) clothes?” and “Does this

child hold a pencil properly?”

The Child Assessment was administered by school staff not directly

involved with the selected children’s care (i.e., not their primary

teachers). The one-on-one assessment was comprised of short

versions of well-known standardized, validated items to measure

Early Math Skills (counting and simple arithmetic), Letter Naming

(asking children to identify as many of 26 letters as they could),

Language Understanding, and Receptive Vocabulary (a short form of

the Peabody Picture Vocabulary Test).


Children’s Problematic Behaviors, Motor Development, and Levels of Attention/Persistence

Social and emotional factors were assessed by the

teachers of 4 and 5 year old School Readiness children

at the end of the 2008/09 and 2009/10 school years.

Multiple items on the Teacher Child Report were used

to develop composite scores for Aggression,

Hyperactivity, Withdrawn behaviors, Attention/

Persistence, and Motor Development.

Our first set of analyses consisted of comparing scores

on these social and emotional dimensions between the

two school years. We found significant decreases in

scores for the three problematic behaviors (Figure 3.1),

and significant increases in Motor Development and

Attention/Persistence scores (Figure 3.2) from 2008/09

to 2009/10.

Figure 3.1: Changes in Levels of Problematic Behaviors Among School Readiness Children, 2008/09 versus 2009/10

Figure 3.2: Changes in Levels of Motor Development and Attention/Persistence Among School Readiness Children, 2008/09 versus 2009/10

We presented the findings appearing in Figures 3.1 and

3.2 to the School Readiness Coordinators in June 2010

and asked them to help us identify possible reasons for

the observed changes between the two school years.

Given that there were no substantial changes in School

Readiness services from 2008/09 to 2009/10, we

suspected that these across-the-board improvements

were likely due to influences other than First 5

Sacramento-funded services. Feedback from the School

Readiness Coordinators suggest other factors in

addition to First 5 Sacramento services may have

influenced outcomes.

The School Readiness Coordinators indicated that the

2009/10 school year saw an increased awareness

among preschool teachers on the importance of social

and emotional development for school readiness. For

example, during the year teaching summits and

professional training opportunities focused on

addressing techniques and strategies that foster social-

emotional and minimize children’s problematic

behaviors. In addition, those early intervention

15.1

8.5

17.1

8.6

12.7

6.0

0.0

10.0

20.0

2009 2010Aggression HyperactivityWithdrawn

79.2

86.6

91.9

97.0

75.0

80.0

85.0

90.0

95.0

100.0

2009 2010

Attention/Persistence

Motor development

18


programs addressing problematic behavior among

children in kindergarten through 12th grade became

available to preschool children during the 2009/10

school year.

In fact, School Readiness Coordinators reported that

referrals to psychologists and social workers had

increased throughout the school districts. These system

-level changes likely explain most of the improvements

in children’s social and emotional behaviors, but some

program-specific factors identified by the School

Readiness Coordinators may have also played a part in

the observed changes.

During the 2009/10 school year, for example, the

Sacramento City Unified School District included

books designed to educate children on inappropriate

social behaviors, such as “Teeth are not for biting” and

“Feet are not for kicking,” into its “Raising a Reader”

program.

There was also greater emphasis on the First 5

Sacramento-funded “Incredible Years” intervention for

teachers, which focuses on “strengthening teacher

classroom management strategies, promoting children's

prosocial behavior and school readiness (reading

skills), and reducing classroom aggression and

noncooperation with peers and teachers.”

Finally, our evaluation may have influenced the

findings through a “testing effect”: Teachers may have

placed an increased importance on social and emotional

behaviors during the 2009/10 school year as a result of

seeing the emphasis on these measures on the Teacher

Child Report they administered in 2008/09.

Language and Cognitive Skills Among Spanish Speaking Children Randomly selected 4 and 5 year old children were also

assessed on their cognitive performance and

proficiency in language. An initial screening section of

the standardized Child Assessment, administered by

trained school staff, determined whether a child should

participate in the English and/or Spanish versions of the

full assessment.

Our analyses revealed no differences in scores for

2008/09 versus 2009/10 for children assessed in

English. However, among children assessed in Spanish,

we found evidence of improvements in all four topic

areas (Figure 3.3), with children assessed in the

2009/10 school year significantly outperforming those

assessed in the previous year in the areas of Early Math

Skills and Language Understanding. Figure 3.3: Changes in Cognitive and Language Measures Among Spanish-Speaking School Readiness Children, 2008/09 versus 2009/10

The School Readiness Coordinators confirmed that

there has been an increasing awareness that

encouraging Spanish-speaking children to

communicate in their primary language (or at least not

10.7

12.8

11.3

14.7

10.4

12.9

14.7

15.3

9.0

11.0

13.0

15.0

17.0

2009 2010

Early math skillsLetter namingLanguage understandingReceptive vocabulary

19


dissuading them from doing so) facilitates learning in

all subject areas, including math.

School Readiness Coordinators also stated that such

messages are communicated to parents participating in

Parent Discussion & Engagement services (discussed in

Chapter 2).

Elk Grove Unified School District’s Literacy Programs

also emphasizes to parents the importance of teaching

their children in their primary language. As such,

Spanish-speaking parents may have been more willing

in 2009/10 to help their children with their homework

compared to the previous year.

Two School Readiness Coordinators also indicated that

their districts have either implemented a new math

curriculum or placed a greater emphasis on math

instruction during the 2009/10 school year. New or

increased math lessons may have benefited most those

children with the weakest math skills.

In fact, the increase in Early Math Skills scores from

10.7 to 12.8 calculated for children assessed in Spanish

began to approach the scores of 16.1 for children

assessed in English in 2008/09 and 2009/10.

20


Chapter 4

Randomly-selected First 5 Sacramento parents were approached by

school staff at two points in time during the 2008/09 and 2009/10

school years, in the fall and spring, and asked to complete the Parent

Survey. This instrument allowed us to assess changes over time on

outcomes applicable to parents, such as the amount of time they spend

interacting with their children and levels of stress as related

specifically to parenting.

A series of 14 questions asked parents how often “you or someone in

your family” participated in certain activities with the child, including

storytelling; helping the child learn letters, words, or numbers;

watching a children’s movie together; and attending a religious activity

or religious school.

Changes in Parent Outcomes

21


One set of questions asked parents how often they

conducted such activities during the previous week;

the other set of questions was concerned with the past

month.

Two of the five items used to measure Parental Stress

were “I find myself giving up more of my life to meet

my children’s needs than I ever expected” and “Since

having a child, I have been unable to do new and

different things.”

Overall composite scores were developed based on

responses to these items with a theoretical range of 0

to 100, with higher numbers representative of greater

levels of involvement with the child and elevated

levels of stress.

From fall to spring, levels of parent-reported

Activities with Child During Previous Week and

Activities with Child During Previous Month

significantly increased. Figure 4.1 also shows that

parents’ levels of stress significantly decreased over

this time.

Figure 4.1: Fall to Spring Changes in Measures of Activities with Children and Parental Stress, 2008/09 and 2009/10

The degree to which family members interact with

young children, for recreation or learning purposes,

and levels of parental stress, are important to

ascertain in a comprehensive evaluation of School

Readiness services because these measures can be

predictive of important child outcomes.

In fact, our analyses of these measures and scores

computed from the Teacher Child Report and Child

Assessment found greater levels of activities with

children significantly associated with increases in

scores for Language Understanding, Receptive

Vocabulary, Early Math Skills, and Overall Literacy.

Likewise, higher levels of parental stress were

significantly related to lower scores on Language

Understanding, Receptive Vocabulary, Early Math

Skills, and Motor Development. Thus, the changes

observed for Activities with Child During Previous

Week, Activities with Child During Previous Month,

and Parental Stress from fall to spring – during which

time families received School Readiness

services – may be responsible for positive outcomes

in language, cognitive, and motor development.

Since the instruments used to measure these child

outcomes, the Teacher Child Report and Child

Assessment, were only administered one time during

the school years, we cannot examine whether

changes found from the Parent Survey correspond in

a similar manner to changes in child outcomes. We

can, however, examine specific School Readiness

services in relation to child outcomes, and these

findings from these analyses are presented in the next

chapter.

89.8

61.7

31.7

92.1

64.4

28.3

20.0

40.0

60.0

80.0

100.0

Activities with Child During

Previous Week

Activities with Child During

Previous Month

Parental Stress

Fall Spring

22


Chapter 5 Services Related to Outcomes

In Chapter 2 we saw that 19 different types of services were provided

to children, families, and parents by First 5 Sacramento School

Readiness-funded school districts during the past two years. Each of

these services had to meet the following criteria before it was

examined in relationship to the child and parent outcomes discussed

in Chapters 3 and 4.

First, it had to be plausible that the service could influence the

outcomes assessed in this evaluation, at least in the short term. For

example, the potential benefits of Health Screenings are numerous

and include identifying and addressing medical or developmental

needs, or preventing diseases through immunizations. However, the

scope of our evaluation limited us from collecting data on these types

of outcomes.

23


relation to child and parent outcomes. Playgroups

were examined in association with parent outcomes

only; Teacher Child Report and Child Assessment

data were collected on 4 and 5 year old children and

very few families with children of these ages

participated in Playgroups, because these services are

directed at children under 4 years of age.

Next, those parent socio-demographic characteristics

shown in other studies to be related to the child and

parent outcomes assessed in this evaluation were

identified. These variables were then included in our

statistical analyses as “control” variables.

In the absence of a true control group from which to

make comparisons – a group of First 5 Sacramento

families who did not receive School Readiness

services – the best way to isolate the influence of

School Readiness services is by discounting the

influence of important control variables through

Table 5.1: Services Meeting the Criteria for Analyses with Child and Parent Outcomes

Indirect Effect on Outcomes

Undetectable in the Short Term

Too Few Services for Analyses

Examined in Relation

to Outcomes

Health Screenings X Speech/Language & Development Screenings X Oral Health Services X Preschool X

Summer Camps X Services for Families Literacy Programs X

Transition Materials X

Transition Activities X

Speech/Language & Development Interventions X Playgroups Parent Outcomes Only

Home Visits X Health Insurance Referrals or Information X X Services for Parents Parent Discussions & Engagement X

Parenting Instructional Workshops X

Health Information X X Lactation Support Referrals X X

Services for Children

As such, Health Screenings was not examined in

relation to the outcomes assessed on the Teacher Child

Report, Child Assessment, or Parent Survey. In all,

seven types of services were excluded from these

analyses based on this rationale (Table 5.1).

Next, we excluded a service if fewer than five percent

of families were reported as having received the

service. That is, too few families received Summer

Camps or Home Visits to justify inclusion in the

analyses. In statistical terms there was not sufficient

power to be able to detect differences based on so few

families having received these services.

As seen in Table 5.1, Health Insurance Referrals or

Information, Health Information, and Lactation Support

services did not meet either of the two inclusion

criteria.

Six services met the criteria and were examined in

24


advanced statistical analyses.

Specifically, the following three variables were

included in the linear regression models examining the

relationship between each service and each outcome:

(1) primary language (English versus all other

languages); (2) employment (full-time versus part-time

employment or unemployed); and (3) education (high

school graduate or higher versus less than a high school

education).

These statistical analyses found School Readiness

services linked to increased scores on Letter Naming,

Language Understanding, and Overall Literacy (Table

5.2).

In fact, children screened as eligible for the Spanish

version of the Child Assessment benefited most from

School Readiness services. Spanish-speaking children

from families receiving Literacy Programs, which

include English as Second Language (ESL) workshops

for parents, scored higher on Letter Naming and

Language Understanding than children from

Spanish-language families not receiving these services.

Spanish-speaking families participating in Transition

Activities which are designed to familiarize the child

and parents with kindergarten staff, procedures, and

expectations, had children who demonstrated greater

knowledge of Letter Naming than children whose

families did not receive these services.

Parent Discussions & Engagement services were

related to improved outcomes in the Letter Naming

component of both the English and Spanish versions of

the Child Assessment. These services were also

significantly related to Overall Literacy scores, which

represents the most comprehensive measure of

cognitive development in our evaluation.

Overall Literacy scores were based on five items on the

Teacher Child Report: (1) recognition of most or all

letters; (2) counting to 20; (3) ability to write or draw

rather than scribble; (4) demonstrated capacity to write

one’s own name; and (5) identification of all primary

colors.

Parent Discussions & Engagement were found to be the

only School Readiness services related to changes in

parent outcomes. We saw in Chapter 4 that Parental

Table 5.2: First 5 Sacramento School Readiness Services Related to Language and Cognitive Child Outcomes

Service Outcome Did Not

Receive Service Received Service Difference

Literacy Programs Letter Naming (Spanish) 11.0 14.8 +3.8*

Language Understanding (Spanish) 10.9 12.4 +1.5**

Transition Activities Letter Naming (Spanish) 12.2 16.0 +3.8*

Letter Naming (Spanish) 12.4 18.9 +6.5**

Letter Naming (English) 16.3 21.4 +5.1**

Overall Literacy 3.9 4.6 +0.7**

Parent Discussions & Engagement

* Service related to difference in outcome by P ≤ 0.10 when controlling for parent language, education, and employment. ** Service related to difference in outcome by P ≤ 0.05 when controlling for parent language, education, and employment.

25


-7.7

7.5

-2.7

0.9

-10.0

-8.0

-6.0

-4.0

-2.0

0.0

2.0

4.0

6.0

8.0

10.0

Parent Discussions & Engagement Services

No Parent Discussions & Engagement Services

Stress decreased from fall to spring. However, levels of

Parental Stress declined to a significantly greater extent

among parents participating in Parent Discussions &

Engagement services (Figure 5.1).

Levels of Emotional Support were not found to have

changes over time, except for parents receiving Parent

Discussions & Engagement services (Figure 5.1).

Items on the Parent Survey used to assess Emotional

Support are related to the frequency to which parents

can “look to others for companionship, assistance, or

other types of support” when needed. These services

may increase emotional support by connecting parents

at the facilitated meetings, or by linking participants to

others by promoting involvement in school advisory

committees or parent/teacher groups.

Figure 5.1: Parent Discussion & Engagement Services Related to Parent Outcomes

Emotional Support

Parent Stress

26


Chapter 6 Teacher/Classroom Outcomes:

School Districts Children’s academic performance and emotional and social behaviors

have been shown to be related to their teachers’ level of education,

classroom practices, and beliefs.4 These and other measures were

obtained from teachers from the six participating School Readiness

school districts using the Teacher/Provider Survey.

The majority of the 80 teachers participating in the survey in 2009/10

were early care educators for preschool-aged children. About one-third

of teachers taught preschool (32.5%), followed by 28.6% from Head

Start classrooms, and finally 20.8% of teachers taught

pre-kindergarten programs. The remaining teachers worked in

kindergarten classrooms (15.6%) or playgroup programs (2.6%).

In this chapter the term “teacher” refers to the primary adult who

directly provided care and education to each child, including child care

providers and playgroup instructors.

4 Howes, C., Burchinal, M., Pianta, R., Bryant, D., Early, D., Clifford, R., et al. (2008). Ready to learn? Children's pre-academic achievement in pre-

Kindergarten programs. Early Childhood Research Quarterly, 23(1), 27-50.

27


Assistant1.7 Associate

Teacher8.3

Teacher8.3

Master Teacher

11.7Site

Supervisor63.3

Progam Director

6.7

In addition to Child Development Permits, other

professional growth opportunities that have been

linked to classroom quality include membership to a

professional Early Childhood Education association

and enrollment in professional development trainings.

School Readiness teachers’ responses to related items

on the Teacher/Provider Survey showed that 42.5%

were a member of a professional association and

82.5% were currently enrolled in a related training,

such as coursework specialized in speech and

language development, special education, or

classroom quality.

Based on the high percentage of teachers with formal

education and continuous professional development,

we conclude that children from families enrolled in

First 5 Sacramento services receive education and care

from highly qualified teachers.

Teacher Education and Professional Growth Teachers’ education, particularly to the level of a

Bachelor of Arts (BA), is an important factor in quality

teaching for early childhood programs.4 The majority

(80.0%) of School Readiness teachers were found to

have a BA degree or higher. Among teachers without a

BA, 56.3% were currently pursuing a higher degree.

Lastly, over half of teachers with a BA had a degree in

early childhood education.

Teachers’ professional development has also been

associated with greater quality of teacher-child

interactions, specifically more positive and emotionally

sensitive interaction with children.5 The California

Commission on Teacher Credentialing adopted the

Child Development Permit as a system to credential

individual teachers as they expand their child

development qualifications. Teachers can obtain a

Child Development Permit on six levels, depending on

their combination of college education, professional

training, and hours of experience.

Three-fourths (n = 60) of School Readiness teachers

had a Child Development Permit, with 81.7% of

credentialed teachers at the three highest levels (Master

Teacher, Site Supervisor, or Program Director) (Figure

6.1).

Figure 6.1: Percentage of Teachers at Each Level of the Child Development Matrix (n = 60), Fiscal Year 2009/10

5 Fuligni, A., Howes, C., Lara-Cinisomo, S., & Karoly, L. (2009). Diverse pathways in early childhood professional development: An exploration of early

educators in public preschools, private preschools, and family child care homes. Early Education and Development, 20(3), 507-526.

28


Figure 6.2: Teacher Involvement with Parent and Parent Involvement in Classroom Scores, Fiscal Year 2009/10

Minutes of Instructional Modality Early childhood programs can have positive long-term

effects on children in the areas of academic

achievement and social adjustment.7 Specifically,

classroom quality has been identified as an important

program factor associated with children’s cognitive and

social-emotional development. 8

One important factor in classroom quality is the type

and duration of learning activities. The National

Association for the Education of Young Children

(NAEYC) recommends a portion of the children’s

instruction be in small groups and include

“child-choice activity periods.”9 Child-directed pretend

play has shown to benefit children’s cognitive behavior

(e.g., increased memory and attention).10

School Readiness teachers were asked to estimate the

amount of time they allot for children to spend in each

Teacher Involvement with Parent Parents’ involvement in their children’s education is

important. Children benefit from their parents’

interaction at home and in the classroom (“classroom”

in this report refers to any facility where children re-

ceive care and education ). In fact, children’s

preliteracy skills have been linked to their parent’s

involvement in their early childhood program.6

Participating School Readiness teachers identified on

the Teacher/Provider Survey the different methods they

employ to keep in contact with parents. Two of the six

items were “regular parent-teacher conferences” and

“responding to parents’ notes or telephone calls within

two days of receiving them.”

Figure 6.2 shows that, out of a total possible score of 6,

teachers’ average score was 5.1; indicating that School

Readiness teachers initiate contact with parents by

using five out of the six strategies.

Parent Involvement in Classroom Teachers were asked a series of questions about the

ways parents are directly involved in the classroom

(e.g., volunteering or helping in the classroom, or

attending parent education meetings or workshops on

topics such as job skills or childrearing).

The average score for parent involvement was 4.5 out

of 6 (Figure 6.2). This indicates that teachers receive

parent participation in over four of the six participation

activities.

6 Arnold, D., Zeljo, A., Doctoroff, G., & Ortiz, C. (2008). Parent involvement in preschool: Predictors and the relation of involvement to preliteracy

development. School Psychology Review, 37(1), 74-90. 7 Barnett, W. (1995). Long-term effects of early childhood programs on cognitive and school outcomes. The Future of Children, 5(3), 25-50. 8

Lambert, R., Abbott-Shim, M., & Sibley, A. (2006). Evaluating the Quality of Early Childhood Educational Settings. Handbook of research on the education of young children (2nd ed.) (pp. 457-475). Mahwah, NJ US: Lawrence Erlbaum Associates Publishers. 9

National Association for the Education of Young Children. (2009). Developmentally Appropriate Practice in Early Childhood Programs Serving Children from Birth through Age 8. Position statement. Online: http://www.naeyc.org/files/naeyc/file/positions/position%20statement%20Web.pdf 10 Gmitrová, V., & Gmitrov, J. (2004). The primacy of child-directed pretend play on cognitive competence in a mixed-age environment: Possible interpretations. Early Child Development and Care, 174(3), 267-279.

5.1

4.5

0.0

1.0

2.0

3.0

4.0

5.0

6.0

Teacher involvement with parent

Parent involvement in classroom

29

http://www.naeyc.org/files/naeyc/file/positions/position%20statement%20Web.pdf�


Frequency of Learning Activities Children’s early literacy skills, including letter

recognition and knowledge of vocabulary, have been

linked to subsequent reading comprehension.11 To

promote early language development, the NAEYC

recommends that teachers of early childhood

programs incorporate language activities, such as

“reading to (children) in small groups and talking with

them about the stories,” throughout their curriculum. 12

School Readiness teachers were asked how often they

engage in literacy activities that promote language

Average Score

Letter Recognition (learning names of the letters)

9.0

Writing Activities (practice writing the letters of the alphabet, write own name)

8.1

Story and Print Conventions (listen to you read stories where they see the print, learn about conventions of print)

7.8

Phonics and Rhyming (learn about rhyming words and word families)

7.8

Vocabulary Development (learn new words or prepositions)

6.9

Expressive Vocabulary (dictate or retell stories)

5.1

Listen to you read stories but they don't see the print

3.7

Minutes (Average)

Teacher-directed whole class activities 68.3

Teacher-directed small group activities 40.1

Teacher-directed individual activities 35.6

Child-selected activities 75.2

Total Time Spent in Learning Activities 219.5

development. Scoring for this outcome ranged from 0

to 10.

Table 6.2 shows Letter Recognition (learning the

names of the letters) was on average the most frequent

language learning activity, followed by Writing

Activities (practicing writing the letters of the alphabet

or writing their own name). Story and Print

Conventions activities (e.g., children listen to stories

and see the print) and Phonics and Rhyming activities

were also used frequently with the children.

Teachers indicated they less often engage in Expressive

Vocabulary activities (having children dictate or retell

stories) and activities where children listen to stories

without seeing the print as language promoting

activities.

Table 6.2: Average Frequency of Learning Activities in the Classroom, Fiscal Year 2009/10

of the following instructional modality on a typical

day: teacher-directed whole class activities, teacher-

directed small group activities, teacher-directed

individual activities, and child-selected activities.

Table 6.1 shows that on a typical day, School

Readiness children spend about one-third of the time

in child-selected activities, and overall, a larger

proportion of their time in small group or individual

activities than in teacher-directed whole-class

activities.

Table 6.1: Average Minutes per Day Children Spend in Each Type of Instructional Modality, Fiscal Year 2009/10

11Dickinson, D., & McCabe, A. (2001). Bringing it all together: The multiple origins, skills, and environmental supports of early literacy. Learning

Disabilities Research & Practice, 16(4), 186-202. 12

National Association for the Education of Young Children. (2009). Developmentally Appropriate Practice in Early Childhood Programs Serving Children from Birth through Age 8. Position statement. Online: http://www.naeyc.org/files/naeyc/file/positions/position%20statement%20Web.pdf

30



Types of Learning Centers in Classroom The NAEYC suggests that as a part of developmentally

-appropriate teaching practices, teachers in early

childhood programs should provide a variety of

materials for children to use in exploration and play

(e.g., books, building blocks, dramatic play material). 12

To obtain an element of developmentally-appropriate

practices, School Readiness teachers were asked to

report the types of learning centers (areas in the

classroom designated for certain activities) they had in

their classroom. Out of a total of nine learning centers,

86.3% of teachers reported having at least seven

different centers. Reading Area with Books, Writing

Centers, Math Area, Art Area, and Dramatic Play

Area were reported by 90.0% or more of teachers as

centers available in the classroom for children (Table

6.3).

Table 6.3: Percentage of Teachers Reporting Each Type of Learning Center in the Classroom, Fiscal Year 2009/10 %

Reading area with books 100.0

Writing center 98.8

Math area 98.8

Art area 91.3

Dramatic play area 90.0

Science or nature area 88.8

Computer area 72.5

Private area for one or two children to be alone 71.3

Listening center 58.8

Absenteeism Chronic absenteeism has been associated with lower

reading and math achievement.13 Quality classroom

environment and practices, specifically sanitary health

practices, has been shown to reduce the number of

school days young children miss.14

Participating School Readiness teachers reported

relatively few children on average were absent from

class. When asked about the class attendance as a

whole, 81.3% of teachers stated that two or fewer

children were absent from class on an average day.

Teachers were also asked about individual children

who repeatedly miss class; 91.3% of teachers had two

or fewer students consistently absent from class.

Teacher’s Beliefs and Attitudes about Best Practices Children’s levels of cognitive and social skills have

been related to developmentally sensitive instruction

(i.e., teachers utilizing developmentally-appropriate

practices).15

Participating School Readiness teachers were asked to

respond to items to assess their knowledge of and

beliefs about developmentally-appropriate practices for

young children. They rated both positive statements

such as “classroom activities should be responsive to

individual differences in development,” and negative

statements such as “students should work silently and

alone on seatwork”.

Responses to these items were combined to develop an

overall Teacher Development Attitude Score. Teachers’

average score was 6.9 out of 10, with 65.1% of teachers

scoring 7 or higher.

13 Chang, H. N., & Romero, M. (2008). Present, engaged, and accounted for: The critical importance of addressing chronic absence in the early grades. New

York: National Center for Children in Poverty. Online: http://nccp.org/publications/pdf/text_837.pdf 14

Sandora, T. J., Shih, M., & Goldmann, D. A. (2008). Reducing absenteeism from gastrointestinal and respiratory illness in elementary school students: A randomized, controlled trial of an infection-control intervention. Pediatrics. 121, 1555-1562. 15

Burchinal, M.R., & Cryer, D. (2003). Diversity, child care quality, and developmental outcomes. Early Childhood Research Quarterly, 18, 401-426.

31

http://nccp.org/publications/pdf/text_837.pdf�


Teachers also rated statements pertaining to their

beliefs in children’s initiation of learning activities

(e.g., “Children should be allowed to cut their own

shapes, perform their own steps in an experiment, and

plan their own creative drama, art and writing

activities”).

Teachers’ average Teacher Child Initiated Beliefs score

was 4.4 out of 5 indicating high beliefs in child

initiated learning.

Process Quality in Classroom Process quality was measured using items related to

curriculum accessibility to parents, such as “parents can

visit the center/home unannounced,” and children’s

activities, such as “time spent playing outdoors.”

School Readiness teachers on average scored 5.3 out of

6, with 51.3% of teachers obtaining the highest score,

indicating high process quality.

32


Chapter 7 Teacher/Classroom Outcomes:

Preschool Bridging Model The Sacramento County Office of Education implements the Preschool

Bridging Model (PBM) as an on-site mentorship and support program

for the privately-owned early childhood programs. Early Childhood

Education Specialists provide education and materials to early

childhood program instructors at over 100 sites each year. Ninety

teachers/providers from family child care homes, preschools, child care

facilities, and pre-kindergarten classrooms completed the Teacher/

Provider Survey in 2009/10 before and after receiving PBM services.

Our evaluation of PBM services included examining the fall 2009 to

spring 2010 changes in teachers/providers’ professional development,

beliefs, and classroom practices.

33


Figure 7.1: Changes in PBM Teacher/Provider Education and Professional Growth, Fall 2009 to Spring 2010

Teacher Education and Professional Growth Teachers’ professional development has been shown to

improve classroom quality and children’s

development.16 Our evaluation found that Early

Childhood Education Specialists’ mentorship and

support services were significantly related to an

increase in teacher/provider development. The percent

of teachers/providers enrolled in professional

development trainings increased from 24.4% to 67.4%

after PBM services were introduced (Figure 7.1).

Early Childhood Education Specialists strive to educate

teachers/providers on available professional growth

opportunities. The Comprehensive Approaches to

Raising Educational Standards (CARES) is a

professional development incentive program to

encourage early childhood educators to continue their

education.

Over nine-in-10 (93.3%) teachers/providers knew about

CARES by spring, which represents a significant

increase from the 71.1% of teachers/providers

reporting that they heard of CARES in the fall (Figure

7.1). Teachers’/providers’ participation in the CARES

program also increased over this time from 12.2% to

20.2%, however this increase was not found to be

statistically significant.

24.4%

67.4%71.1%

93.3%

20.0%

40.0%

60.0%

80.0%

100.0%


Enrolled in Teacher Related Training

Heard of CARES

Teacher Involvement with Parent

Teachers/providers reported an increase over time in

parent involvement in their classrooms. From fall to

spring, teachers’/providers’ scores for initiating parent

involvement increased significantly from 2.3 to 3.2,

indicating that by spring, teachers/providers used at

least three of the six methods of communication with

parents (Figure 7.2).

16 Fuligni, A., Howes, C., Lara-Cinisomo, S., & Karoly, L. (2009). Diverse pathways in early childhood professional development: An exploration of early educators in public preschools, private preschools, and family child care homes. Early Education and Development, 20(3), 507-526.

34


Figure 7.2: Changes in PBM Site Parent Involvement, Fall 2009 to Spring 2010

Parent Involvement in Classroom

Parent involvement in the classroom scores also

increased significantly during this time, suggesting that

by spring teachers/providers included and received

parent participation in at least three activities on

average (Figure 7.2).

Minutes of Instructional Modality

The Teacher/Provider Survey included items related to

the amount of time that children spent in child-selected

activities. This is important because children’s ability

to select and direct their own activities is related to

fostering their cognitive and social-emotional

development.17 We found a 19 minute increase in the

average number of reported minutes per day that are

devoted to child-selected activities (Figure 7.3). This

suggests teachers/providers encourage children to

choose activities throughout the day, as

recommended by the NAEYC. 18

Figure 7.3: Changes in Minutes of Child-Selected Activities at PBM Sites, Fall 2009 to Spring 2010

Frequency of Learning Activities

Teachers/providers were asked how often they engage

in reading and language activities with children, such

as learning letters of the alphabet, writing, and phonics

and rhyming. Teachers’/providers’ fall to spring

responses showed a significant increase in Letter

Recognition (learning the names of letters), from 7.7

to 8.4, indicating that teachers/providers taught letter

names more frequently after receiving PBM services.

The remaining six activities were not significant,

including Writing Activities and Vocabulary

Development.

17 Dickinson, D., & McCabe, A. (2001). Bringing it all together: The multiple origins, skills, and environmental supports of early literacy. Learning Disabilities

Research & Practice, 16(4), 186-202. 18

National Association for the Education of Young Children. (2009). Developmentally Appropriate Practice in Early Childhood Programs Serving Children from Birth through Age 8. Position statement. Online: http://www.naeyc.org/files/naeyc/file/positions/position%20statement%20Web.pdf

35

113

132

100

110

120

130

140


2.3

3.2

2.6

3.1

1.0

1.5

2.0

2.5

3.0

3.5

Fall 09 Spring 10

Teacher Initiated Parent Invovlement

Parent Involvement in Class



Process Quality in Classroom The process quality of the early childhood programs

were scored using quality indicators such as “minutes

of television per day,” “time children spent playing

outdoors,” and whether the “teacher meets the needs of

specific children.” Teachers’/providers’ Quality

Indicator Score significantly increased, from 4.5 in fall

to 5.1 in spring (Figure 7.5). This suggests the process

quality in the classroom improved after teachers/

providers participated in the PBM.

Type of Learning Centers in Classroom

PBM services include working with teachers/providers

to help construct a variety of activity centers for

children to explore, such as reading areas, dramatic

play corners, and art centers. The percent of teachers/

providers who responded “yes” when asked if they

provide a specific learning center increased for all nine

types of learning centers listed on the Teacher/Provider

Survey. Reports of classrooms with a Writing Center,

Math Area, Science Area, Art Area, and a Place for

Child to Play Alone significantly increased in spring

(Figure 7.4).

Overall, the average number of learning centers

increased significantly from 5.9 to 7.1. This suggests,

on average, teachers/providers offered a greater

number of interest areas to children after working with

the Early Childhood Education Specialists; in fall the

average classroom had at least five different learning

centers and in spring children on average had available

to them at least seven different play areas.

Figure 7.4: Significant Changes in Reported Types of PBM Site Learning Centers, Fall 2009 to Spring 2010

36

74%

63%58%

80%

64%

92% 93%

82%

96%

84%

50%

60%

70%

80%

90%

100%

Writing Center Math Area Science Area Art Area Place for Child to Play Alone

Before PMB Services After PBM Services


Figure 7.5: Changes in PBM Sites' Process Quality, Fall 2009 to Spring 2010

37

4.5

5.1

4.2

4.4

4.6

4.8

5.0

5.2


Quality Indicator Score


38


Conclusions

39

Nearly 6,700 children and over 4,000 parents received First 5

Sacramento School Readiness services during the 2008/09 and 2009/10

school years. This report includes an extensive list of findings that

provide evidence in support of the expectations that these children were

better prepared for kindergarten, and that services affect parents in a

number of ways that benefit the family overall.


Language and Cognitive Outcomes To many, the benchmark for evaluating the success of

School Readiness is whether services influence basic

academic skills (i.e., reading, writing, and arithmetic).

Perhaps the most impressive finding in this area is that

School Readiness services delivered to parents (i.e.,

Parent Discussions & Engagement services) were

related to improvements in children’s overall literacy,

which included measures of reading, writing, and

arithmetic applicable to 4 and 5 year old children.

We also found that math skills and language

understanding significantly improved between the

2008/09 and 2009/10 school years. Improvements in

math may have been due to a greater emphasis placed

on math and a new math curriculum introduced into

School Readiness schools in the 2009/10 school year.

Changes in how Literacy Programs were delivered may

have been responsible for the increase in language

understanding scores. In fact, when we examined

specific services in relation to outcomes, we found

Literacy Programs empirically linked to improvements

in language understanding as well as to letter naming

scores.

Higher letter naming scores were also associated with

having received First 5 Sacramento-funded Parent

Discussions & Engagement and Transition Activities

services. However, all of these findings apply only to

bilingual or monolingual Spanish-speaking children,

and not to English-only speaking children (with one

exception: Parent Discussions and Engagement ser-

vices were related to letter naming among English-only

speaking children).

First 5 Sacramento School Readiness services appear to

have the greatest impact on those most in need. That is,

services did not impact monolingual English children

who had higher scores on all four outcomes derived

19 Our new evaluation of First 5 Sacramento services, beginning September 1, 2010, includes the ability to make comparisons on pertinent outcomes between children who have and have not received School Readiness services up to the third grade.

from the Child Assessment, our standardized

assessment by trained school staff, and improved scores

among Spanish-speaking children began to approach

those levels obtained by English-speaking children. In

other words, School Readiness services allowed

Spanish-speaking children, who enter School

Readiness services with lower academic skills, to

improve to levels comparable to their English-only

speaking classmates. It might be the case that School

Readiness services do eventually positively influence

English-only speaking children’s academic

performance, but our current evaluation design does not

allow for assessing such longer-term outcomes. 19

Social, Emotional, and Motor Development Outcomes To teachers and staff in elementary schools,

benchmarks for evaluating School Readiness services

must include children’s social and emotional behaviors.

During the last two school years, related behaviors

improved in all the outcomes that we assessed in these

areas. Specifically, levels of aggressiveness, withdrawn

behaviors, and hyperactivity decreased, while levels of

attention/persistence increased.

The instrument used to assess these behaviors also

included items to measure children’s levels of motor

development, which were also found to improve among

School Readiness children between the 2008/09 and

2009/10 school years. Certain School Readiness

services, such as Preschool and Parenting Instructional

Workshops, are designed specifically to address

children’s behaviors. However, having received these

services was not found to be related to behavioral

outcomes. This appears to be due to the fact that a

number of new services and strategies, separate from

School Readiness services, are being employed in

Sacramento schools to address children’s social and

emotional behaviors. As such, the increased,

multi-pronged approach to dealing with children’s

40


problematic behaviors explains the improvements for

the most recent school year, and likely overshadowed

any measureable effects that one particular type of

School Readiness services potentially had.

Parent Outcomes Our evaluation included obtaining answers to questions

from parents about behaviors that likely impact school

readiness, such as helping “your child ... learn letters,

words, or numbers.” We also took measures of parents’

emotional state, such as stress and social support. These

conditions are certainly conducive to a home

environment that promotes children’s learning, but also

to optimal parent-child relationships and other factors,

such as propensity for domestic violence. These

measures were taken from parents at two points in time,

before (or at the beginning of) and after the family

received School Readiness services.

Our analyses revealed that measures of parents’ levels

of activities with their children, derived from 14

different items, increased; while levels of stress, based

on seven items, decreased. The supposition here is that

School Readiness services lead to these improved

outcomes. This supposition was tested by examining

specific services in relation to these outcomes. The

result of these analyses was that parents who

participated in Parent Discussion & Engagement

services reported lower levels of stress and higher

emotional support.

Strengths and Weaknesses of the Evaluation of Services Through School Districts

It is important to consider the strengths and weaknesses

of our evaluation when interpreting the findings for

School Readiness services delivered in Sacramento

area school districts.

One strength of our study is the sampling methodology.

School Readiness families were selected using

probability sampling techniques, which means that our

findings can be generalized to the larger School

Readiness population of children and parents. This

approach also prohibited school staff from selectively

identifying for the evaluation those children and

parents which might have benefited most from services.

In terms of measurement, a major strength of our

evaluation is that key outcomes came from a

standardized, one-on-one assessment with proven

reliability and validity administered by trained staff

other than the assessed children’s primary teacher.

Thus, findings reported for math skills, letter naming,

and language understanding were based on information

with a high degree of validity and low degree of bias.

Our analyses of these child outcomes, as well as the

parent outcomes, were based on statistical tests that

controlled for the influence of parent language,

education, and employment; thus discounting (but not

excluding) the possibility that our reported findings are

related to factors other than School Readiness services.

One final notable strength of our study was that parent

outcomes were based on within-subject comparisons.

That is, we measured changes over time within same

group of parents and not different groups of parents

selected in the fall and spring of the school year.

Tracking families through the school year required

additional work by the School Readiness Coordinators,

but the payoff is that it increases our confidence that

the findings are based on services rather than other

factors.

As with most evaluation studies, limitations of our

design include the fact that information from the Parent

Survey and Teacher Child Report came from self

report. There is always the chance that some parents

may have purposely given themselves better scores in

the spring, or that teachers purposely rated certain

41


42

students as better behaved, just to make School

Readiness look good.

Other weaknesses include that we were not able to

examine the potential impact of certain services, such

as Home Visits, on certain outcomes because of our

limited sample sizes, or services in general on the

longer term outcomes.

Evaluation of the Preschool Bridging Model Turning to the evaluation of services provided by the

Preschool Bridging Model, we found a host of positive

outcomes related to the services provided by Early

Childhood Education Specialists at 100 privately-

owned early childhood programs. These included

increases in the number of teachers/providers enrolled

in professional development trainings, parent

involvement in the classroom, five types of learning

centers, and overall process quality.

These findings are impressive and certainly suggest

that these First 5 Sacramento-funded School Readiness

services are leading to children who are better prepared

for kindergarten. Data from the PBM evaluation,

however, are subject to potential biases. Teachers and

providers are asked to complete the Teacher/Provider

Survey by those providing the services. By the time the

follow-up survey was due, the relationship developed

with the Early Childhood Education Specialists may

consciously or unconsciously cloud teachers’/

providers’ judgment toward providing fully objective

information.


Data Collection Parent Survey

Families from each of the six school districts were

randomly sampled and asked to complete the Parent

Survey in the fall and spring of each school year. A

translated version of the instrument was available for

parents who identified Spanish as their primary

language. School Readiness Coordinators recruited the

sampled parents and distributed the instrument, and in

some cases administered it over the telephone, at the

school site, or the parents’ homes. They also met with

an evaluation team member on an ongoing basis to

review the status of recruitment. The reason behind

each unsuccessful recruitment attempt was

documented, and included whether the parent had

moved or refused to participate. The overall completion

rates (both fall and spring) were 75.2% and 80.7% for

the Parent Survey (Table A.1).

Child Assessment

Four and 5 year old children of families who completed

the fall Parent Survey were selected for the Child

Assessment. The assessment was administered by

school staff not directly involved with these children’s

care. Prior to data collection, the evaluation team

trained the school staff at one of two trainings on the

proper procedures for administering the Child

Assessment. These training sessions covered the

importance of refraining from coaching, providing

neutral praise, and procedures for administrating and

scoring each page. The training also included an

item-by-item review with particular attention paid to

the initial screening section to determine whether a

child should receive the English, Spanish, or both

Appendix versions of the assessment. Finally, the trainees

practiced administering the assessment to each other

while evaluation team members observed and provided

guidance. Each trainee received a Child Assessment

instrument, instructional materials, and child specific

scoring sheets pre-labeled with the children’s name.

Assessments were completed on 153 children in the

spring of 2009 and 187 children in the spring of 2010

(Table A.1).

Teacher Child Report

The Teacher Child Report was completed by the

teachers of the 4 and 5 year old children sampled for

the Child Assessment. The School Readiness

Coordinators recruited the teachers for the survey, and

monitored data collection by meeting weekly with

evaluation team staff who reviewed and collected

completed instruments. In 2009, 90.8% of the 153

Teacher Child Reports were completed; in 2010, 88.8%

of the 187 were completed (Table A.1).

Teacher/Provider Survey: School Districts

School Readiness Coordinators identified the

educational instructors who worked closest with each

child from the fall Parent Survey sample to complete

the Teacher/Provider Survey. In some cases these were

preschool or playgroup teachers, child care providers,

speech therapists, or case management staff. School

Readiness Coordinators distributed and collected the

Teacher/Provider Survey. Evaluation team staff

reviewed surveys for completion when meeting with

individual School Readiness Coordinators each week.

All Teacher/Provider Surveys were completed in 2009,

and 97.6% were completed in 2010.

43


Teacher/Provider Survey: Preschool Bridging Model

Early Childhood Education Specialists from the

Sacramento County Office of Education recruited 101

teachers/providers from 100 privately funded early

childhood education programs to participate in a fall

and spring Teacher/Provider Survey. In spring, 89.1%

of teachers/providers completed both the pre and post

Teacher/Provider Survey.

Table A.1: Sample Size and Completion Rates by Instrument, Evaluation of First 5 Sacramento School Readiness Services, 2008/09 and 2009/10 2008/09 School Year 2009/10 School Year n sampled % completed n sampled % completed

Parent Survey 318 75.2 424 80.7

Child Assessment 153 84.3 187 87.2

Teacher Child Report 153 90.8 187 88.8

Teacher/Provider Survey administered in school districts

55 100 80 97.6

Teacher/Provider Survey administered at Preschool Bridging Model sites

-- -- 101 89.1

44


First 5 Sacramento Commission 2750 Gateway Oaks Drive, Suite 330


Phone: (916) 876-5865 Fax: (916) 876-5877

Email: [email protected]

www.First5Sacramento.net

FIRST 5 SACRAMENTO COMMISSION

To: Evaluation Committee Members

From: Fred Molitor Walter R. McDonald & Associates, Inc. Date: September 8, 2010

Subject: Results from CBI Analyses; Suggestion for Dissemination of Findings Beyond Annual Report

In last year’s annual report we presented outcomes related to CBI attendance from the 2008 and the 2009 Parent Interview separately. Recall that the highest scores for the various measures of social capital were found for those attending a First 5 Sacramento CBI event, and the consistencies between the findings for 2008 and 2009 were very striking. We have finished compiling the master list of CBI events occurring in 2009/10, and compared parent responses on the Parent Interview with the master list to identify those who likely attended a CBI event during the year. Our analyses for the forthcoming 2009/10 annual report included data from all three years. We examined each outcome across the three groups and, unlike previous years, controlled for important demographics – parent race/ethnicity, parent language, and family education. As seen in Table 1, linear relationships were found for all social capital outcomes; each of these analyses yielded statistical significance. With the Evaluation Committee’s approval, we propose to develop a Research Brief based on these findings for submission to the American Journal of Public Health. Research Briefs require fewer than 800 words and one table or figure, which we believe is an excellent format for disseminating these results. Of course we will ask that First 5 Sacramento staff review our draft article before submission to the journal. The Evaluation Committee, First 5 Sacramento staff, and WRMA should also try to identify other avenues for communicating these findings to the research community and general public.

of 2

Table 1. Social Capital Outcomes in Relation to Levels of Participation in Community Events

Community event attendance during previous 6 months

None

Non-CBI Event

CBI Event

Community Connectedness

No. people in neighborhood know by name

4.7 6.3 8.0

No. people who visit each other’s homes

1.5 2.0 3.2

No. friends/relatives who live in neighborhood

1.4 2.1 2.5

Neighborhood Cohesion (range = 2.5, 9.5)

6.3 6.5 6.8

No. of community resources (range = 0, 7)

1.5 1.7 1.9

Efficacy (range = 1.7, 4.0)

2.9 3.1 3.2