chapter 1: introduction - drfiene.files.wordpress.com · web viewthe purpose of this literature...
TRANSCRIPT
Dissertation Proposal
Submitted to Northcentral University
Graduate Faculty of the School of Educationin Partial Fulfillment of the
Requirements for the Degree of
DOCTOR OF EDUCATION
by
Sonya Stevens
Prescott Valley, ArizonaAugust, 2016
Table of Contents
Chapter 1: Introduction........................................................................................................1
Background....................................................................................................................3Statement of the Problem...............................................................................................4Purpose of the Study......................................................................................................5Research Questions........................................................................................................5Hypotheses.....................................................................................................................6Nature of the Study........................................................................................................6Significance of the Study...............................................................................................7Definition of Key Terms................................................................................................9Summary......................................................................................................................10
Chapter 2: Literature Review.............................................................................................10
Documentation.............................................................................................................10The Social Context of Early Childcare Programs........................................................11Effects of Program Quality on Child Growth and Development................................17Oversight and Evaluation of Childcare Programs.......................................................19Summary......................................................................................................................33
Chapter 3: Research Method.............................................................................................33
Research Method(s) and Design(s)..............................................................................35Population....................................................................................................................38Sample.........................................................................................................................40Materials/Instruments..................................................................................................41Data Collection, Processing and Analysis...................................................................44Summary......................................................................................................................47
References..........................................................................................................................49
Appendices........................................................................................................................55
Dissertation Proposal Components – 1_Sonya Stevens 3
Chapter 1: Introduction
According to the U.S Bureau of Labor Statistics, 72 % of children were cared for
in non-parental situations beginning in the first year of life in 2013 (Huston, Bobbitt &
Bentley, 2015). As a result, childcare has become a primary environment where children
can be taught a range of physical and academic skills as well as guided in the areas of
positive behavior, social interactions and conflict resolution (Crowley, Sangchoon &
Rosenthal, 2013). Quality care provides children with many benefits including, but not
limited to, fewer injuries, early referral to health, developmental and behavior services,
and care for children with special needs (Crowley et al., 2013). Quality in early
childhood is directly related to children being safe, healthy and ready to learn (Alkon,
Rose, Wolff, Kotch & Aronson, 2015; Crowley et al., 2013; Huston et al., 2015).
Although the direct responsibility for ensuring the health and safety of children
lies with families and local childcare providers, most states have recognized the
importance of providing oversite and support to childcare providers through formal
inspections (Kayira, 2016; Lapp Payne & Testa, 2014). Agencies across the United
States are working to ensure provider meet overall health and safety standards as well as
improved quality through effective and reliable monitoring to ensure compliance with
regulations (Alkon et al., 2015; DEL.wa.gov, 2015). However, the reliability and content
and social validity (Wolf, 1978) of the monitoring tool to assess the quality of care for
children has yet to be considered. Without a reliable assessment tool, as well as
stakeholders’ (e.g., licensors, childcare program directors and staff, business owners and
parents) understanding of the inspections relevance to achieving program goals,
information from annual monitoring cannot effectively increase program outcomes for
Dissertation Proposal Components – 1_Sonya Stevens 4
children (Ledford, Hall, Conder & Lane, 2016; Strain, Barton & Dunlap; 2012; Wolf,
1978).
No matter how valid and reliable an assessment tool is, the results are potentially
meaningless unless the program stakeholders consider the inspection to be valuable
beyond receiving a passing grade on the inspection (Ledford et al., 2016; Strain et al.,
2012). Licensing staff and providers must build a partnership with all program personnel
centered on shared values of the inspection’s intended purpose. One of the largest
challenges licensing agencies face is establishing good working relationships with
providers and obtaining confidence in the monitoring process (Kayira, 2016). It is
important stakeholders understand that the purpose of the inspection process is to
facilitate quality programming necessary to effectively guide child growth during the
early years (Ledford et al., 2016).
In order to establish these relationships, agencies must be fair, objective and
consistent with all licensed providers across all regions; licensors must be objective, work
independent of external influences, and be consistent (Fiene, 2014; Kayira, 2016). Yet,
the performance of onsite inspections can vary greatly along a continuum of objectivity;
licensor objectivity is often centered on the frequency and ease of measures, potentially
undermining the validity of objective standards (Amirkhanyan, Kim, & Lambright,
2013). Licensing tools such as procedures and checklists must be aligned to ensure
objectivity and consistency between licensors, but it is also important to determine the
social validity of monitoring systems to reliably demonstrate the effects of childcare
programming on the well-being of young children.
Dissertation Proposal Components – 1_Sonya Stevens 5
Background
According to the Childcare in America: 2016 State Fact Sheet Report (2016),
Childcare providers across America care for approximately 11 million children five-
years-old and under. Because so many children are being cared for outside of the home,
it has become a focus nationwide to ensure children are being cared for in safe
environments. The Childcare Development Block Grant (CCDBG), a federal program
that provides states the financial means to award low income families subsidies to help
pay for the high cost of Childcare, was reauthorized in 2014 and required participating
states to impose stronger health, safety and quality regulations (Raikes, 2012). This is
due to fact children learn best when they have a foundation of safety and health (Cote-
Lussier & Fitzpatrick, 2016).
To meet these new requirements states are working to assess and put into place
new policies, procedures and systems. For example, Washington State, in an effort to
improve regulatory methods, assessed the use of monitoring procedures in a 2014
analysis of licensors compliance with procedure 10.1.8 (Appendix A). Licensing analysts
found that from a sample of 1401 monitoring visits conducted in 2014, approximately
11% of the total licensing visits did not switch from an abbreviated checklist to a
comprehensive checklist when required (DEL Data Analysis Report, 2015). This
discovery prompted the creation of an Ad Hoc committee to explore the systems and
procedures in order to improve licensor compliance.
Recommendations were made to alter the checklist system to allow licensors time
to focus on areas of critical non-compliance by creating a focused system; the ability to
focus on areas where corrections need to be made rather than in areas providers are
Dissertation Proposal Components – 1_Sonya Stevens 6
currently in compliance (Ad Hoc meeting notes, 2015). Recommended changes did not
include elements of an adequate evaluation of the licensing system, nor any methods to
check for reliability and validity. This study will not only use stakeholder input to create
and evaluate a focused monitoring system but will also ensure the relevance of a focused
checklist system will help meet licensor and provider needs and lesson the unintended
effects of higher standards (Moloney; 2016; Strain et al., 2012).
Statement of the Problem
With so many children in licensed care, it is critical for state monitoring systems
provide an accurate baseline of protection through a reliable monitoring system and clear
methodology (Alkon et al., 2015; Kayira, 2016). Licensor consistency is critical to the
licensing system; without it, the effectiveness and the quality of early childhood
programs can be adversely affected. Without transparent demonstration of consistency
and fairness, providers and the public may become distrustful of the licensing system and
challenge the usefulness of the monitoring system (Kayira, 2016). As the childcare
licensing industry works to improve the health, safety, and quality of licensed childcare
for the millions of children in out-of-home care, minimal attention has been directed
toward determining the reliability, validity and social validity of monitoring tools
(NCCCQI, 2014).
Many states use licensing tools intended to measure performance; however, how
the checklist systems are designed often exposes the monitoring visits to a subjective-
objective dichotomy due to the tool allowing for individual judgement by the licensors
(Amirkhanyan et al., 2013). Kayira (2016) explained the consistency and objectivity of
monitoring systems is directly related to the equality of monitoring regulations between
Dissertation Proposal Components – 1_Sonya Stevens 7
childcare providers; yet, few have examined the reliability and validity of monitoring
tools with a focus on social validity (Alkon et al., 2015). Therefore, it is not clear how
the tools used to conduct monitoring should be designed to ensure consistent inter-rater
reliability and content validity in addition to social validity as perceived by all
stakeholders. Without evidence of these factors, the effects of annual monitoring on the
licensed childcare programs cannot be effectively evaluated.
Purpose of the Study
The purpose of this mixed method evaluation study is to determine the reliability
and content and social validity of the focused monitoring tool used to monitor the
foundational health and safety of childcare programs in Washington State. Four licensed
childcare centers, four to eight licensors, and other advisory stakeholders in Washington
State will participate in the study. Data will be collected through a series of focus groups,
and 14 strategically designed focused monitoring visits that will include interviews with a
representative sample of the program stakeholders for each program.
Research Questions
The focus of the present study will be to evaluate the reliability as well as the
content and social validity of the annual monitoring tool used to provide focused
oversight to licensed childcare centers. A mixed method program evaluation research
design will be used to ensure multiple inputs from all stakeholders in order to understand
childcare monitoring practice in action (Frye & Hemmer, 2012). Data will be collected
using interviews and focus groups and analyzed using qualitative methodology to
evaluate the social validity of focused monitoring. Additional data will be collected
using the focused monitoring tool and analyzed using descriptive and inferential statistics
Dissertation Proposal Components – 1_Sonya Stevens 8
to evaluate the reliability and content validity of the tool used. Data will be collected and
analyzed to answer the following research questions:
Q1. How do stakeholders describe the value, usefulness, and effects of state
administrated focused monitoring?
Q2. Is the focused monitoring observation tool a valid and reliable measure of
the foundational health and safety concerns that must be met by state
licensed early childhood programs?
Hypotheses
Nature of the Study
The nature of the study is a mixed method program evaluation with the intent to
evaluate the reliability and validity of a focused monitoring tool used during annual
licensed childcare monitoring inspections in Washington State. This study will
concentrate on evaluating the tools’ value and usefulness in maintaining health and safety
quality and effective programming in licensed center childcare programs. The program
evaluation research design will be guided by the Context, Input, Process, and Product
Evaluation (CIPP) (Stufflebeam & Shinkfield, 2007). CIPP provides a framework for
integrating program evaluation with program management and development. Using this
framework, three broad tasks are performed for each type of evaluation: (1) delineating
the information needed for decision making; (2) obtaining the information; and (3)
synthesizing the information to make programmatic decisions.
CIPP (Stufflebeam & Shinkfield, 2007) will guide the evaluation by allowing
multiple inputs from various stakeholders in order to understand Childcare monitoring
consistency between licensors and the social validity of the tool (Fry & Hammer, 2012).
Dissertation Proposal Components – 1_Sonya Stevens 9
Participants will include licensing leadership, licensing staff and childcare providers in
order to develop, implement and evaluate the reliability and validity of a focused
monitoring system. Overall, descriptive statistics and input analysis will be conducted to
the target sample population in the form of focus groups and interviews.
Additional data will be gathered from a pilot implementation of the focused
checklist by licensors during unannounced monitoring visits of volunteer provider
childcare center programs. This will gather quantitative information regarding needed
changes to the monitoring tool as well as the relationships between monitoring
procedures and licensor inter-rater reliability. Finally, pre and post implementation
interviews of licensors and providers will gather qualitative data for the purpose of
assessing the validity of the tool. Using the four steps outlined in the CIPP process will
allow the input from a variety of stakeholders to effectively describe the value, usefulness
and effects of focused monitoring as well as ensuring its validity and reliability when
assessing foundational health and safety concerns that must be met by state licensed
childcare programs.
Significance of the Study
The majority of the health and safety concerns addressed in research has
concentrated on the licensing standards used for annual inspections and the frequency to
which licensing visits are conducted; there is very little research on the inspection tool
itself (Alkon, Rose, Wolff, Kotch & Aronson, 2015). Differential monitoring was
developed with the intent to save licensors time, thereby saving licensing agency’s
money by concentrating on sites with significant compliance issues with more in-depth
Dissertation Proposal Components – 1_Sonya Stevens 10
monitoring checklists and rewarding those programs with a history of high compliance
with the use of an abbreviated checklist (Fiene, 2014).
This research study is designed to refine the differential monitoring methodology
to include focused monitoring as a means for licensors to concentrate more heavily on
areas of non-compliance while limiting time spent on areas where providers need
minimal or no licensor support. An annual monitoring tools that is reliable and valid
could potential increase provider compliance demonstrating statewide childcare increases
in the areas of quality, safety and health in which children learn best. Additionally, by
concentrating on the monitoring tools’ reliability and validity, licensing agencies are
better informed on how to provide licensing consistency which could prove to increase
the agency’s liability protection (Kayira, 2016).
As more and more children are entering into the licensed Childcare systems, it is
critical to re-assess and maintain systems that provide appropriate oversight, ensuring
each child’s health and safety. While this study’s main audiences are policy makers and
licensing professional and practitioners within the State of Washington, the substantive
significance of this tool can be applied nationwide as it can be adapted to be inclusive of
any licensed childcare regulations. Furthermore, as this study concentrates solely on the
validity and reliability of a focused monitoring tool, it lays a foundation for future study
regarding its effectiveness in the area of increasing overall compliance: It is possible
providers receiving focused assistance in only the areas in which they struggle to be
complaint will, overtime, increase programming and environmental quality so children
can receive higher quality of care in which to learn (Alkon et al., 2015; Crowley et al.,
2013; Huston et al., 2015).
Dissertation Proposal Components – 1_Sonya Stevens 11
Definition of Key Terms
Annual monitoring. A checklist data system licensors use once a year to
evaluate the level to which licensed childcare facilities are in compliance to the state
regulations (Crowley et al., 2013). These are completed with either a full checklist where
all the rules are inspected or an abbreviated checklist where only a selected set of rules
are inspected (NCCCQI, 2014).
Checklist. A simple measurement tool licensors use to measures compliance
with state rules in a yes/no nominal format. Either the facility is in compliance with rules
or not in compliance. (Alkon et al., 2015).
Differential monitoring: A method to regulate child care facilities based on the
frequency and depth of monitoring based on the facility’s history of compliance
(NCCCQI, 2014). The use of differential monitoring/inferential inspections by licensing
agencies was developed as a time saving technique and a technique to focus regulatory
efforts on facilities that required additional inspections or technical assistance (Fiene,
2014).
Key Indicator. A licensing measurement system utilizing a shortened or
abbreviated version of a comprehensive checklist measuring compliance with rules
through a statistical methodology. Only key predictor rules are included on an indicator
checklist (Fiene & Carle, 2010; NCCCQI, 2014).
Rating scale. A more complex measurement tool in which a Likert type of rating
is employed going from more to less or high to low. A rating scale is always used in the
development of weighting/risk assessment systems. It is not used in measuring
compliance with rules or at least it hasn’t been used in the past.
Dissertation Proposal Components – 1_Sonya Stevens 12
Revised Code of Washington. The Revised Code of Washington (RCW) are
permanent laws now in force in the state of Washington. RCW are a collection of
Session Laws passed and ratified by the Legislature signed by the Governor or passed
through the initiative process (Washington State Legislature, 2016).
Risk Assessment/Weighting System. A Likert type of measurement that utilizes
a modified Delphi technique to determine the relative risk to children if there are
violations with specific rules. Risk assessment/weighting systems are developed by
sending a survey to a selected sample of persons/stakeholders in order for them to rank
the relative risk of violation with specific rules (Fiene & Carle, 2010).
Washington Administrative Code (WAC). WAC are regulations of executive
branch agencies for Washington State. These regulations are a source of primary law in
Washington State and are specific to a subject or agency (Washington State Legislature,
2016).
Summary
Licensing oversight agencies monitor childcare centers across the nation in order
to ensure children are cared for in safe and healthy environments. They are responsible
for the development and implementation of oversight tools such as checklists, policies
and procedures used by licensors to monitor and provide technical assistance. Through
this process, licensing is able to assure foundational guidelines and standards are met
providing a baseline for quality early learning programming (Alkon et al., 2015). This
mixed method CIPP research study is designed to evaluate and refine the licensing
methodology used by Washington State to include focused monitoring as a means for
licensors to concentrate more heavily on areas where providers need more support rather
Dissertation Proposal Components – 1_Sonya Stevens 13
than on areas of profiency. An annual monitoring checklist tool that is reliable, valid and
is perceived to be useful to both providers and licensors could potential increase provider
compliance (Fiene & Nixon, 1985; Fiene, 2014; Raikes, 2012).
Chapter 2: Literature Review
Millions of children spend a great deal of their week in formal childcare settings
for a variety of reasons (Houston, 2015). Early childhood centers are required by state
licensing agencies to ensure that young children are safe, healthy and ready to learn
(Crowley et al., 2013). Ensuring the practice of health and safety practices in early
childhood centers requires formal oversight via a monitoring system. The same
monitoring system also has the potential to increase program quality as well as increase
the value of childcare within the community (Li, Farkas, Duncan, Burchinal & Vandell,
2015). The purpose of this literature review is to describe the context and purpose of
childcare and early education, a brief review of research on the relationship between
quality programming and early childhood development, and the process used to evaluate
and monitor childcare programs.
Documentation
A preliminary literature search was conducted to identify studies that provided
information pertaining to childcare licensing in the United States specific to regulations,
monitoring/evaluation systems and the reliability and validity of monitoring tools. Given
the limited nature of current research in the specific area of childcare licensing, the search
was expanded to quality improvement measures. Documentation was gathered through a
variety of empirical and peer reviewed articles found in the Northcentral University
Library with specific search engines including EBSCOhost, Sciencedirect, Sage Journals,
Dissertation Proposal Components – 1_Sonya Stevens 14
and ProQuest. Additional background literature was found through the library of
Pennsylvania State University, the Department of Early Learning, the Administration for
Children and Families, and Researchconnectrions.org. All studies that reported on state
licensing systems and other relevant information such as information linking quality
improvement in childcare to licensing or specific elements of health and safety were
integrated into the literature review. Additional studies were identified by reviewing the
reference list of the studies previously identified.
The Social Context of Early Childcare Programs
In 2013, 72 % of American children were cared for by someone other than a
parent, many of them cared for in one of the over 107,286 childcare centers throughout
the United Stated (Alkon et. al, 2015; Huston, Bobbitt & Bentley, 2015). According to
the U.S. Bureau of Labor Statistics, this was due, in part, to the 63.9% of women with
young children being employed (Houston, 2015). Childcare has become such a
foundational part of a working parents’ life that in 2008, the U.S. Congress financed
approximately $5 billion in childcare subsidies to support parental employment for low-
income families and childcare quality improvement efforts with the Childcare
Development Fund (CCDF) (Raikes, 2012). The need for childcare is defined by the
social context in which it is used and or needed. This could have to do with increased
families with two income parents, the raise in increased single parents, the increase body
of evidence demonstrating the benefits of early education; especially for dual language
learners and children from low income demographics, among other reasons (Tucker-
Drob, 2012).
Dissertation Proposal Components – 1_Sonya Stevens 15
Families choose childcare based on a variety of reasons such as cost,
convenience, programming (Raikes, 2012; Sandstrom & Chaudry, 2012), developing.
positive relationships, and becoming autonomous learners in a positively focused social
environment, but parents’ greatest concern is for their child’s safety within the
environment and the quality of the educational programming (Scopelliti & Msatti, 2013).
Unfortunately, low income families are often forced into low-quality childcare options
lacking the desired safety qualities and high quality programming because of unstable or
unpredictable work as well as non-traditional work hours (Palley, 2012). Regardless of
income, the main parental preference for childcare is that children are safe and cared for
by trustworthy caregivers along with basic elements of quality programming. Parents
must rely on licensing systems to ensure these foundational levels of care, regardless of
their income status (Crowley, 2013; Sandstrom & Chaudry, 2012).
The purpose of childcare has evolved far beyond a parental need for a child to be
“watched over” while he or she goes to work. Global economic competition dictates the
United States has future generations be equipped with strong early learning foundations
and school success (Winterbottom & Piasta, 2015). Participation in preschool and early
education has demonstrated marked improvements in children’s learning. For example,
Tucker-Drob (2012) found preschool attendance was associated with higher reading and
math scores, especially for children from lower socio-economic families and racial
minorities. Results such as this have legislatures and policy makers, as well as parents,
look to early education as a means to help better prepare children for kindergarten and
beyond while lessoning the achievement gaps (Tucker-Drob, 2012; Lipscomb, 2013;
Yazejian, Bryant, Freel, & Burchinal 2015).
Dissertation Proposal Components – 1_Sonya Stevens 16
Early childhood education is characterized by early learning and development
theories and methodologies including foundational health and safety practices (Crowley
et al., 2013). While empirical evidence justifies the need for increased early education,
there is renewed interest by the federal and state government to ensure children are cared
for in safe and heathy environments as well as are provided quality educational
programming. In fact, global attention has recently been placed on national health and
safety standards including increased health and safety standards within licensed childcare
as well as unlicensed providers. In the United States, the interest is to not only increase
health and safety standards but to also keep in mind the social context as well as the
purpose of childcare (Crowley et al., 2013; Tucker-Drob, 2012).
At the foundation, children in licensed childcare should be cared for in
environments that are healthy and safe. Developing appropriate environments will set
children on a path to ongoing physical, emotional, social and academic development.
This is particularly important in early childhood because children’s brains are developing
rapidly through ongoing experiences laying the foundation for more complex learning
later in life (D’Souza & Gurin, 2016; Villanueva et al., 2016). Maslow (1973) described
this need through the theory of hierarchy of needs in which children need to be have their
basic needs met and a feeling of safety before cognitive learning and self-actualization
can be met (D’Souza & Gurin, 2016). Safe, healthy and developmentally appropriate
early learning environments provide exposure and opportunities for participation in these
essential experiences (Villanueva et al., 2016).
The theoretical concept of positive human growth beginning with a foundation of
trust and safety was introduced by Erik Erikson. A clinician and cultural anthropologist,
Dissertation Proposal Components – 1_Sonya Stevens 17
Erikson created a theoretical framework outlining connections between the different
stages of human growth with a particular emphasis on the relationship between societal
influences and the individual. For example, an infant learning to walk must deal with the
fears of being safe versus the desire to be autonomous. The surroundings in which the
child is learning a new skill may predict the chances of success (Batra, 2013): In a safe
environment, growth and development are nurtured when the child is given appropriate
encouragement and help as the child explores the environment. If the environment is
unsafe, the child may fear danger and revert to crawling instead of walking to avoid
injury, thereby reducing the desire and trust in the caregiver to try again.
It is essential educational facilities promote children’s ability to engage in
learning through assured health and well-being. According to Côté-Lussier and
Fitzpatrick (2016), children who do not feel safe at school have been linked to lower
academic outcome and lower levels of engagement. It was also found that children in
unsafe environments tend to engage in risky behavior and are more aggressive which
creates behavior issue that could potentially adversely affect the entire classroom (Cote-
Lussier & Fitzpatrick, 2016). The benefits of providing safe and healthy environments
are numerous and coincide directly with the need to provide quality early learning
environments for children, a fact that has not evaded national attention.
Because there is renewed interest in giving children a firm foundation of health
and safety in which to develop, the Childcare Development Block Grant (CCDBG, 2014)
was reauthorized with additional stipulations for health and safety accountability
measures and quality improvement (Administration for Children and Families, 2016;
Lapp-Payne & Testa, 2014;). With these new authorizations in place, states desiring to
Dissertation Proposal Components – 1_Sonya Stevens 18
use the CCDF funds to subsidize childcare for low income families were required to
address specific health and safety issues as well as training to license-exempt providers
such as Friends, Family and Neighbors (FFN) (Raikes, 2012). Nevertheless, Lipscomb
(2013) found that the financial assistance provided through the subsidy system is still not
enough to afford low-income children to enroll, and maintain enrollment, in quality
programs justifying the need to increase subsidy payments to income families and/or
require the increase of quality in all childcare programs.
Researchers have found that unlicensed care exposes children to particular risks
and hazards. Miller (2013) described some of those hazards as contaminated food,
unprotected pools, steep stairs, unclean pets, improper handwashing and so on. It has
also been argued environmental hazards of the childcare sites, such as high levels of lead
or mercury, can adversely affect children’s health and learning (Hudson, Miller & Seikel,
2013; Somer, Harvey & Rusnak, 2011). Unfortunately, systems throughout the United
Stated and Canada continue to struggle to find one system to provide consistent
monitoring and enforcement of unlicensed childcare (Miller, 2013).
When children are cared for in environments that are safe, healthy and
developmentally appropriate they experience fewer illnesses and injuries, are more likely
to have better health care, accurate referrals to developmental services when needed, and
are simply more prepared and ready to learn. When childcare lacks the foundational
elements of quality, children are put in greater risk for illness, injuries and other risk
factors limiting learning opportunities (Crowley et al., 2013). It is for these reasons there
have been multiple state and federal initiatives to improve health and safety for children
in childcare settings (Crowley et al., 2013; Meloy, Lipscomb & Baron, 2015).
Dissertation Proposal Components – 1_Sonya Stevens 19
Perhaps the largest initiative to increase health and safety in childcare comes from
standards outlined in the Caring for Our Children (CFOC) (Crowley et al., 2013; Hudson
et al., 2014). Crowley et al. described CFOC as an inclusive set of standards developed
in partnership by the U.S. Department of Health and Human Services and the American
Academy of Pediatrics and is aligned with accreditation standards of the National
Association for the Education of Young Children (NAEYC). CFOC presents common
health and safety standards which can reduce conflicting standards sometimes found
when considering multiple funding streams; it is for this reason the CCDF references
CFOC as a baseline to health and safety standards.
Table 1 outlines each of the categories covered in CFOC; within each category
are a set of standards specific to that topic. It demonstrates how national recommended
standards are firmly founded in health and safety practices with a few addressing minimal
educational and program requirements needed for a child’s learning. Nevertheless,
CFOC does not have the authority to mandate all standards for achieving foundational
quality programming, leaving each state to determine the licensing methods that work
best for their jurisdictions (Bromer, McGabe & Porter, 2013; Crowley et al., 2013; Lapp
Payne & Testa, 2014).
Table 1: Health and Safety Standards for Licensed Childcare (CFOC, 2011)
Standards Category Topics within the Category
Staffing Ratio/group size/age, recruitment background checks, qualificationsprofessional development, substitutes and consultants
Program activities Developmental activities, supervision and discipline, health education, parent/guardian relationship
Health promotion Health promotion, hygiene, cleaning, sanitizing
Dissertation Proposal Components – 1_Sonya Stevens 20
and protection disinfection, health protection, care plans and adaptation and illness management
Nutrition and food service
General requirements, requirements for special groups, staffing, meal service, seating and supervision, food from home, learning experiences, kitchen and equipment and food safety
Facilities, supplies, equipment and environment
Overall building requirements, outdoors, furnishing and equipment, play areas, playground & transportation
Infectious diseases (organized by type)
Detection and documentation, unimmunized children, immunizations of teachers and caregivers, informing parents, caregivers, authorities and communities, exclusions, attendance, staff education and records maintenance
Children with special health needs and disabilities
Inclusion, enrollment process, developing a service plan, coordination and document, re-evaluation, facilities assessment and additional standards
Administration Governance, policies, human resource management and records
Licensing and community action
Regulatory policies, licensing agency, facility licensing, role of the Health Department, caregiver and teacher support, public policy and resource management
Quality childcare is a necessity for millions of American families. Since it is also
the primary environment where many children learn, it also plays an important role in
closing the achievement gap and bringing families out of poverty. With so many children
in licensed care, it is critical that state monitoring systems provide an accurate baseline of
protection and quality programming using reliable, valid evaluation instruments and well
defined evaluation procedures (Alkon et al., 2015; Kayira, 2016).
Effects of Program Quality on Child Growth and Development
While quality has yet to be universally defined, recent research has suggested a
clear and definable benefit for children to attend high quality early learning programs
(Côté, Mongeau, Japel, Xu, Séguin & Tremblay, 2013; Kirby et al., 2015; Li et al., 2015).
Dissertation Proposal Components – 1_Sonya Stevens 21
Li et al. (2015) found that children who attend high quality childcare demonstrate
increased cognitive, language and social skills prior to beginning kindergarten.
Additionally, Yazejian et al., (2015) found that children who received high quality of care
earlier had higher kindergarten entry scores and dual language learners (DLL) show
greater gains than do English-only children.
Dual language learners and economically disadvantaged children were found to
have reduced levels of achievement gaps when they have consistent access to high
quality childcare (Yazejian et al., 2015). Other findings suggest DLLs show measurable
gains in social competencies when participating in quality preschool programs (Halle,
Whittaker, Zepeda, Rothenberg, Anderson, Daneri, &. Buysse, 2014). This is in part due
to the fact DLLs are able to increase language skills thereby increasing social standings
with English speaking peers. This is accomplished through quality programming which
benefits all children but specifically those learning English, giving them a better
advantage for success in later education (Yazejian et al., 2015).
Evidence showed a clear and definable value for the investment into
strengthening policy for the quality of care for early childhood (Heckman, 2011; Li et al.,
2015; Yazejian, 2015). In fact, evidence in favor of quality improvement is so conclusive
the Department of Defense (DOD) recognized that having reliable, high quality childcare
is an important part of combat readiness planning for military personnel (Floyd &
Phillips, 2013). To ensure quality, the military practices four categories of oversight;
certification including unannounced inspections, accreditation through the National
Association for the Education of Young Children, high education requirements for
Dissertation Proposal Components – 1_Sonya Stevens 22
teaching staff, and above average pay rates to ensure low staff turnover (Fiene, n.d.;
Floyd & Phillips, 2013).
Research has also provided evidence of the financial benefits of increased time
and exposure for children to attend high quality early education programs (Yazejian,
Bryant, Freel & Burchinal, 2015). One of the first studies to demonstrate this concept
was done by Barnett in 1993 (Campbell-Barr, 2012). This longitudinal study
demonstrated that for every dollar spent on quality early educational programming, there
was $7.16 saved (Campbell-Barr, 2012). Savings were seen through higher grade point
averages and reduction in the level of special education services over the child’s
educational career leading to higher earning as adults and lower levels of law
enforcement interventions (Campbell-Barr, 2012).
Investments to increase stable higher quality childcare, such as increased subsidy
for families to find and maintain quality childcare, have also been shown to provide the
community with a marked return on investment (Arteaga, Humpage, Reynold & Temple,
2014; Lipscomb, 2013). For example, the state subsidies that help family’s access quality
childcare have been shown to helped low income families stabilize their own income. In
addition, providers tend to increase investment into their own programs thereby
increasing the program quality (Lipscomb, 2013).
Oversight and Evaluation of Childcare Programs
The evaluation of childcare providers begins with a state mandated licensing
system which measures the compliance to state regulations in which childcare centers
must operate (Crowley et al., 2013). One type of monitoring system sometimes used is
the Quality Rating and Improvement Systems. An increasing number of states are
Dissertation Proposal Components – 1_Sonya Stevens 23
employing QRIS constructs to rate providers for quality levels above that required by
state licensing mandate (Boller, Paulsell, Grosso, Blair, Lundquist, Kassow, Kim &
Raikes, 2015). Nevertheless, providers have a choice to work toward an accreditation
system awarded by either a national or state specific agency. The accreditation process is
often considered an indicator of childcare quality because it examines many of the same
and often additional components found in QRIS. Nevertheless, providers have a choice
to work toward an accreditation system awarded by either a national or state specific
agency (Winterbottom & Piasta, 2015).
Quality Rating Systems. There are two quality rating tools commonly used to
evaluate program quality of licensed childcare centers; the Early Childhood Environment
Rating Scale-Revised (ECERS-R; Harms, Clifford, & Cryer, 1998) and the Classroom
Assessment Scoring System (CLASS; Pianta, La Paro, & Hamre, 2008). The CLASS
uses observations and a scoring system with a common metric and vocabulary for trained
professionals to examine instructional quality including teacher–child interactions
throughout early learning environments (Rodriguez & Garza, 2014). Likewise, ECERS-
R-R is widely used as an observational method to measure the quality of an early learning
classroom based on documenting relations between childcare quality and elements of
child development (Mayer & Beckh, 2012; La Paro, Hamre & Pianta, 2012). ECERS-R-
R has been used, adapted and validated in more than 20 countries, although it is limited
within a cultural context as studies on its effects have been done independently of cultural
and theoretical backgrounds (Mayer & Beckh, 2016).
While these tools are widely accepted and used throughout state QRIS systems, as
well as studied for reliability and validity, they are not without their faults. One
Dissertation Proposal Components – 1_Sonya Stevens 24
significant failing of the environmental rating systems is a limited knowledge around the
tools’ technical adequacy and the high need for inter-rater reliability (La Paro et al.,
2012). Additionally, within the ECERS-R there remains a concern regarding the
response process validity, structural validity, and criterion validity. By using a stop
scoring method (meaning when the environment rates low in one area the rating stopes
for the section without further credit given for aspect in higher quality that may be
observable in the environment), some of the higher quality indicators will be missed
(Mayer & Beckh, 2016). This will lead to structural validity issues and criterion issues
due to a heavy emphasis put on one set of factors and a tendency to underestimate the
associations between quality care and child outcomes (Mayer & Beckh, 2016). Finally,
evaluations of QRIS systems have focused on rating system only and not on the
components assisting quality improvement (Boller et al., 2015).
Over the past several decades, with pressure from policy makers to increase child
outcomes and public accountability for success in education, the early learning
community has recognized the educational and developmental benefits to ensuring
quality in childcare that rises above minimal health and safety measures (Elicker,
Ruprecht, Langill, Lewsader, Anderson & Brizzi, 2013). To this end, the majority of
states have begun the process of implementing childcare QRIS as a means to encourage,
measure, and assess the quality of childcare programs (Boller et al, 2015). This national
movement gained momentum when the Department of Education introduced the Race to
the Top Grant requiring state agencies awarded the grant to implement a QRIS system.
Unfortunately, there is no evidence demonstrating that one type of QRIS is better
than another (Boller et al., 2015; Elicker et al., 2013). Additionally, details of quality
Dissertation Proposal Components – 1_Sonya Stevens 25
have yet to be universally defined, although there is some agreement on the general
qualities of high quality childcare. Some of these areas are classroom curriculum
founded in research including elements of social, emotional and cognitive development
as well as appropriate child and program interactions and assessments, strong
professional development requirements, teacher support systems including child to staff
ratio and class size, programming that meets the needs of diverse learners, family
engagements, and availability of consistent and enough quantity of care (La Paro,
Thompson, Lower, Kintner-Duffy & Cassidy, 2012; Wechsler, Melnick, Maier & Bishop,
2016). These qualities, amongst other, are found in popular QRIS rating tools such as the
ECERS-R and CLASS (La Paro et. al, 2012; Rodriguez & Garza, 2014).
La Paro et al. (2012) expressed some cautions when using quality indicators to
increase regulations and other accountability systems in early education. Because
program quality is so strongly tied to child outcomes, there is concern that current
assessments are far too general and global in nature without providing enough
information on how programs can improve. In other words, the system of quality
improvement is moving faster than a common definition and improvement constructs can
be put into place. There is limited understanding by stakeholders as to what exactly is
being measured and how to improve program quality which can potentially lead to
increased spending with weakened returns. Karoly, Zellmaon and Perlman (2013) also
found that the systems are designed to rate the overall quality of the program and often do
not reflect the individual qualities of the classrooms.
The limitations of the rating systems are not the only concern that should be
considered. Hotz & Xiao (2011) argued that imposing regulations on quality may have
Dissertation Proposal Components – 1_Sonya Stevens 26
unintended negative influence on the market for childcare services. It is possible stronger
regulations reduce the number of childcare centers in the lower-income markets which is
not offset by increased workers or increased family home providers. This may be due to
the fact it costs more money than a provider in a low income neighborhood may be able
to charge for services to employ and maintain a higher quality program (Hotz & Xiao,
2011). It may also be due to a lack of social acceptance for the need for higher quality
care (Artz & Welsch, 2014).
Artz and Welsch (2014) found that, even with the complication of limited quality
measures, quality programming has limited impacts on the cost of childcare. Cost is
primarily determined by the degree of market forces, the influences of geographic regions
and, to a small degree, the quality of the program (Artz & Welsch, 2014). This may be
because the consumer may not understand and/or recognize the attributes and benefits of
quality childcare; therefore, there is little incentive for a provider to invest in higher
quality standards (Artz & Welsch, 2014).
Licensing. Childcare programs are required to have and maintain a childcare
license as the first step in quality rating (Kirby, Caronongan, Malone & Boller, 2015). In
fact, this is a beginning requirement for all QRIS rating tools and accreditation programs;
without licensure it is not possible to become rated on the quality rating scale or be
considered for accreditation (National Association for the Education of Young Children,
2016). The process of evaluating licensed childcare remains critical because children are
a vulnerable population and often do not have the communication skills nor knowledge to
sufficiently advocate for their right to a safe, healthy and developmentally appropriate
care situation (Kayira, 2016; Lapp Payne & Testa, 2014; Moloney, 2016).
Dissertation Proposal Components – 1_Sonya Stevens 27
The monitoring of licensed childcare centers for the purpose of continued state
licensure requires impartial, objective and independent observations and documentation
and adequate follow up (Kayira, 2016). Additionally, stakeholders need to understand
annual licensing oversight is more than simply a way to keep children safe but also as
tool to reach and maintain program objectives and a foundational level of quality goals
(Strain et al., 2012). According to NAEYC’s position statement, public regulations
ensure children’s rights to environments protecting them from harm and that provide an
outlet for healthy development are the minimum requirement for a childcare business to
operate (Crowley et al., 2013).
Even though every state regulates licensed providers differently and with a variety
of regulations among states, there exists general agreement of licensing standard
categories across the nation. The licensing regulations center on reducing environmental
hazards in which to protect children from injury, illness and abuse (Connors & Morris,
2014). Many states have begun to include regulations which address some level of
developmentally appropriate programming, requirements for communication with
families, staffs experience requirements, and professional development, and some limited
educational components (Connors & Morris, 2014). For example, Table 2 provides the
eight sections of licensing regulations from Washington State. Each topic includes
subtopics of the category and is assigned individual administrative code numbers. Table
2 is summation of the Washington State regulations monitored during routine monitoring
visits as well as complaint inspections (when there is an external complaint regarding a
facility) and health and safety re-checks after violations are found and corrected by the
provider. It is important to note that Washington State has separate Washington
Dissertation Proposal Components – 1_Sonya Stevens 28
Administrative Code (WAC) for licensed family home providers and school age
providers. While each set of regulations are similar, the example below is specific to
licensed childcare centers (DEL.com, 2016).
Table 2: Washington Administrative Code Chapter 170-295: Minimum Licensing Requirements for Childcare Centers
Standards Category Topics within the Category
Licensing
Staffing
Licensing authority, definitions, who needs licensed, subsidy eligibility, other required regulations, Waivers, dual license, application requirements, personal characteristics, capacity determination, initial and non-expiring licensing, denial, civil fines, revocation, probationary license, location of facility,
Director requirements, lead teacher requirements, assistant requirements, volunteer requirements, ongoing training requirements, orientations, first aid and CPR, HIV, AIDS and blood borne pathogen training, TB testing requirements.
Program Play materials, equipment and activities, length of time a child can be in a center, staff interactions, behavior management and guidance practices, rest periods, evening and night time care requirements, offsite, kitchen activities, special requirements for infants and toddlers, outdoor play area.
Health and nutrition Required health policies and procedures, hand washing, illness exclusion, medications (requirements, consent, documentation and storage), use of bulk medications, left over medications, delivery of medications, promotion, hygiene, cleaning, sanitizing disinfection, health protection, care plans and adaptation and illness management, milk requirements, meals and snacks (food types, sources, service, storage and thawing), kitchen materials and equipment, eating and drinking equipment.
Care of young children
Minimum age, nutritional needs, Bottles including formula and breast milk (preparation, storage, cleaning), feeding infants and toddlers, toileting, sleep equipment, safe sleep practices, diaper changing, nurse consultant, change of clothing
Safety and environment
First-aid supplies, safety maintenance, disaster planning, cleaning and sanitizing, water activities, maintenance and storage of janitorial supplies, water health, sewage and liquid waste, fence requirements, toilets, handwashing sinks and
Dissertation Proposal Components – 1_Sonya Stevens 29
bathing facilities, laundry, sleep and nap equipment, children’s storage space, facility temperature, pesticides, animals
Agency practices
Records, reporting, and posting
Discrimination, religious activities, American Indian children, child abuse and neglect, prohibited substances, unsupervised access
Child files, immunizations, attendance records, record availability and electronic records, facility records, policies, reporting, and postings
Administration Governance, policies, human resource management and records
Licensing and community action
Regulatory policies, licensing agency, facility licensing, role of the Health Department, caregiver and teacher support, public policy and resource management
One of the common trends revealed in an analysis completed by the National
Center for Childcare Quality Improvement (NCCCQI) (2013) is the predominate
inclusion across all states of increased safe sleep practices, increased preservice training
requirements for center directors and all staff training hours, increased background check
requirements, and added health and safety requirements. Enforcement trends included an
increase in inspection frequency, publicizing licensing information on websites, and use
of electronics during licensing visits. The report by NCCCQI (2014) outlined strategies
used by states to identify and limit harm to young children including educational
campaigns, increasing data systems, increasing monitoring visits and technical assistance,
and more.
There are cautions when making regulatory design overly complicated and strict.
Maloney (2016) found, teachers in England overwhelming reported an increase in
concern around inspector inconsistencies and their lack of understanding for the needs of
early learners. Because of this lack of confidence toward the inspectors, the weight of the
Dissertation Proposal Components – 1_Sonya Stevens 30
findings was lessoned in relevancy (Maloney, 2016). Therefore, there is concern that
policies and inspections become more about simply controlling the environment rather
than including systems to support childcare providers within the early learning
community (Maloney, 2016). It is for these reasons individual states must continuously
assess monitoring systems to include holistic programming support.
All states require some routine inspections to monitor compliance of state
mandated childcare regulations but no two systems are alike (Alkon et al., 2015; Crowely
et al., 2013). Much of the research completed over the past three decades defines three
key theoretical approaches to monitoring licensed childcare for improved health and
safety in group care as well as limited requirements addressing social and emotional
development (Fiene, 2014; Fiene & Carl, 2010; Fiene & Nixon, 1985). Common
methodologies include routine unannounced inspections with a combination of standard
weighting (the classification of regulation importance) and/or a key indicator system
(standards with statistical evidence of reliability or probability of full compliance or non-
compliance) and differential monitoring (the frequency of inspections) (Fiene, 2014;
Crowley et al., 2013). Most states use either a weighted system of monitoring or an
indicator system, but not both (Fiene, 2014).
Differential monitoring is a regulatory method for determining the frequency and
depth of monitoring based on an assessment of a facility’s history of compliance with
licensing standards (Fiene, 2013). A differential monitoring system can be used to
recognize a provider’s strong record of compliance with abbreviated, or less frequent,
inspections (Fiene, 2013). For example, Oklahoma uses a differential monitoring system
requiring three monitoring visits per year for providers with a clear history of compliance
Dissertation Proposal Components – 1_Sonya Stevens 31
while providers with serious noncompliance issues may receive as many as 12 monitory
visits each year with intensive technical assistance (NCCCQI, 2014). This type of
monitoring system was also shown to be implemented at increasing frequency across the
nation within the past decade (NCCCQI, 2013).
A licensing key indicator system is defined as specific rules statistically
predicting overall compliance with all the licensing rules (Fiene & Carl, 2010). It is
important to note, key indicators are not necessarily those rules most frequently out of
compliance, nor place clients at the greatest risk (Fiene & Carl, 2010). Fiene found the
13 areas most likely to predict compliance include child abuse prevention,
immunizations, adult child ratio, group size, staff qualifications and training,
supervision/discipline, fire drill, medication administration, emergency plan/contact,
outdoor environment, inaccessibility of toxic substances, and proper hand
washing/diapering (NCCCQI, 2013). The methodology outlines that if one of the
indicators within the key categories are found to be non-compliant than it probable other
areas within the program will also be out of compliance (Fiene, 2010).
For example, Washington State uses a combination of both the key indicator
system and the differential monitoring system (Hyde, 2011; NCCCQI, 2014). Feine’s 13
Key indicators (2010) are used as the core of an abbreviated checklist used for
monitoring visits by providers with high levels of compliance (NCCCQI, 2014). Being
found in non-compliance on one of the key indicators triggers the licensors to move to a
comprehensive checklist making the system both a differential and key indicator system
(DEL.com, 2015; Hyde, 2011; NCCCQI, 2014).
Dissertation Proposal Components – 1_Sonya Stevens 32
The key indicator system was predicted to reduce workload gradually as licensors
would have additional time to provide technical assistance thereby increasing areas of
compliance (Hyde, 2011). Other advantages of the key indicator system were thought to
include a systems responsiveness to changes, consistency between monitoring visits,
coverage of all regulatory areas, and a potential to be a cost effective monitoring method
(Fiene & Nixon, 1985; Fiene, n.d.). Nevertheless, research has not demonstrated the
efficiency of monitoring theory including the social acceptance of the monitoring system
and the effects of licensor- licensee relationship on monitoring compliance.
In addition to ensuring children’s health and safety, states are working on
additional oversight methods to incorporate important elements of program quality that
have foundational programming requirements to optimize children’s ability to learn and
develop (Crowley et al, 2013; Li et al., 2015). Licensed oversight agencies are working
with quality rating administrators to provide holistic evaluations of all childcare
providers. In fact, the 2014 Childcare Development Block Grant (CCDBG)
reauthorization included specific requirements for monitoring agencies of all types to
coordinate efforts (Maxwell, Sosinsky, Tout & Hegseth, 2016). Even with the
coordinated efforts, state licensing requirements and monitoring tools remain restricted to
state administrative codes and revised codes passed only by state legislative bodies. The
main focus of state licensing standards remains centered on the health and safety
standards that provide the foundation for quality standards.
An overwhelming majority of directors reported using the state licensing
regulations as the main standards guiding program expectations and curriculum planning
rather than using early learning standards. Early learning standards have been developed
Dissertation Proposal Components – 1_Sonya Stevens 33
by national and state organizations such as NAEYC, Head Start, etc. These standards
outline the kinds of things children should be learning and doing as a result of early
learning programming. Juxtaposed to early learning standards, licensing regulations
provide the foundational guidelines for children to meet learning standards by ensuring a
foundational level of quality; those regulations that ensure children are cared for in safe
and healthy environments (Ackerman & Sansanelli, 2010).
Even though there are no federal guidelines for states to design licensing
regulations there is a high relative consistency in licensing regulations across the country
in the area of minimum health and safety standards and a common understanding that
health and safety standards are the foundation of quality programming (Connors &
Morris, 2015). Because directors overwhelming uses these regulations as the guide to
their programming, it is critical to ensure licensing regulations are effective in providing
foundational quality programming and reduce instances of licensing violations
(Winterbottom & Jones, 2014). Every state has a different method in assisting programs
move beyond basic licensing compliance to quality improvement programs.
Social Validity of the Evaluation Process. Bronfenbrenner’s ecological theory
(1978) is used to provide the framework in which the importance of social acceptance in
childcare oversight is justified. The value society places on childcare programs plays a
critical role in licensing oversight. The ecological systems theory identifies four
environmental systems that influence human development; the microsystem (individual
traits and social interactions affecting individual perceptions including family, friends,
teachers and so on) the mesosystem (the system including the environment in which
children live such as home, church, school and the neighborhood), the exosystem (the
Dissertation Proposal Components – 1_Sonya Stevens 34
cultural context such as media, dominant industries and political climate) and the
macrosystem (the greater social context such as culture, beliefs, laws and ideologies)
( van Dijken, Stams & Winter, 2016; Kohl, Recchia, & Steffgen, 2013). Each of the
systems interact with one another to make up the totality of a child’s learning influences.
In order to ensure children have healthy and safe environments in which to learn,
licensing systems must consider each of the systems within the ecological theory in order
to be affective. For example, the interactions of peers and family interactions influence
the entire school climate (Kohl et al., 2013). If parents do not trust the school system to
ensure their child’s safety, children will likely adopt those feelings limiting learning
success. Likewise, if providers do not trust the outcomes of monitoring visits as an
effective tool for improvement, the regulatory outcomes are less likely to be important to
the provider. Additionally, if providers have limited investment and communication to
the licensing agency, the licensing agency may hold negative views about the provider
community.
Many childcare licensing issues cross the ecological systems including
community culture, ideological factors, political climate, socioeconomic factors, and
family structure. Van Dijken et al. (2016) explained how a high value for identifying
and understanding protective factors to young children and the role of the exosystem such
as neighborhoods and childcare communities can limit child injury and maltreatment.
When providers understand and incorporate the context of keeping children safe as a
foundation of early learning, licensing systems are able to effectively implement
protective factors through appropriate law development and oversight methods inclusive
of each of the ecological systems (Kohl et al., 2013).
Dissertation Proposal Components – 1_Sonya Stevens 35
In 1978, Wolf introduced the concept of social validity. The concept states that
society needs to validate critical work in order to ensure it is what they really want, that
the results of the work justify the means to get there, and that they are satisfied with the
work being done. By ensuring the social buy-in of childcare licensing tools, the work
licensors and providers do to ensure the health and safety of children is not only socially
valid but also relevant. Therefore, monitoring licensed childcare centers requires
impartial, objective and independent observations and documentation and adequate
follow up ensuring the childcare industry’s commitment to providing the best care and
early education possible for young children (Kayira, 2016). On the other hand, it is not
clear if the monitoring tools used to conduct childcare evaluations are consistent, reliable
and valid for the purpose for which they are used, and perceived by all stakeholders to be
valuable and useful
Many children spend a great deal of time in formal childcare; therefore, it is
critical that early childcare programs provide environments that are safe, healthy and
provides quality for learning (Crowley et. al, 2013; Houston, 2015). The tools used to
assess early learning programs must be valid and reliable in order to provide a firm
foundation of health and safety standards that ensure that children are ready to learn
(Crowley et al., 2013). Developing reliable and valid assessment tools that provide a
clear understanding of the inspections relevance and significance to achieving program
goals can only increase the benefits of early childhood programming (Li et al., 2015;
Strain et al., 2012).
Dissertation Proposal Components – 1_Sonya Stevens 36
Summary
Balancing the need for regulation to contribute to the health and safety of young
children as well as improve quality in licensed childcare without providing excessive
oversight is not an easy task. With so many children being cared for in licensed facilities
it is critical licensors and providers understand that regulatory processes are reliable,
compatible and effectively evaluated to meet their needs and will effectively aid to lessen
the unintended effects of higher standards (Moloney; 2016; Strain et al., 2012). Moloney
(2016) explained how when early educators do not value the results of regulations due to
adversarial relationships, or believe they are being unfairly regulated, the results have
little to no meaning. One way to ensure this does not happen is by concentrating on the
consistencies of licensing as a logical way to improve the quality of health and safety of
early learning environments. In this way the licensing tools’ usefulness provides a clear
understanding to all stakeholders how licensing guidelines can be used to develop a
partnership around the best evaluation of programming and quality improvement (Wolf,
1978).
Chapter 3: Research Method
When concentrating on only the standards licensed facilities must meet, and not
evaluating the reliability and validity of the monitoring process, the effects of annual
monitoring on quality program development cannot be determined. The purpose of this
mixed method program evaluation is to evaluate the reliability and validity of the focused
monitoring tool used in Washington State, as well as its value and usefulness in
maintaining quality and effective programming in licensed center Childcare programs.
This section describes the research methods and study design as well as how data will be
Dissertation Proposal Components – 1_Sonya Stevens 37
collected and analyzed to answer the following research questions: (1) How do
stakeholders describe the value, usefulness, and effects of state administrated focused
monitoring; (2) Is the focused monitoring observation tool a valid and reliable measure of
the foundational health and safety concerns that must be met by state licensed early
childhood programs?
The Washington State Department of Early Learning (DEL) made changes to the
monitoring system in 2011 to increase efficiency and improve relationships between
licensors and providers (Alkon et al., 2015; Hyde, 2011). Changes to the licensing
system included the use of an electronic checklist during monitoring visits and the use of
a differential key indicator model (Hyde, 2011). Policy (located in Appendix B)
currently states that when one of the key indicators is found to be non-compliant during
an abbreviated inspection (Appendix C), the licensor must move into a longer and more
inclusive checklist (Appendix D) (DEL.wa.gov, 2016). Differential monitoring was
intended to focus on the indicators proven to keep children safe in licensed Childcare
when using the abbreviated checklist, saving both time and resources (Fiene & Carl,
2010).
The key indicator-differential monitoring system was predicted to reduce
workload gradually which would give licensors additional time to provide technical
assistance, build trusting relationships with providers, and increase compliance with
quality programming standards (Hyde, 2011). With so many children in licensed
Childcare, it is critical Washington State uses a reliable and valid monitoring checklist
system.
Dissertation Proposal Components – 1_Sonya Stevens 38
Research Method(s) and Design(s)
The program evaluation research design will be guided by the Context, Input,
Process, and Product Evaluation (CIPP) (Stufflebeam & Shinkfield, 2007). CIPP
provides a framework for integrating program evaluation with program management and
development. Using this framework, three broad tasks are performed for each type of
evaluation: (1) delineating the information needed for decision making; (2) obtaining the
information; and (3) synthesizing the information to make programmatic decisions.
The context evaluation involves the identification of problems and needs that
occur in a specific setting and provides the basis for developing objectives that will lead
to program improvement. The input evaluation involves collecting information about the
resources and strategies needed to accomplish program goals and objectives and
determining constraints, which requires a wide range of knowledge about possible
resources and strategies and their effectiveness in achieving the desired outcomes. The
process evaluation requires the collection of evaluative data to monitor the program as it
operates day to day, which allows for the identification of problems in their early stages
so they can be solved before they cause irreparable damage to the project but can also
provide useful information about the strengths and weakness of the project that may
account for the observed outcomes. The product evaluation is conducted to determine
the extent to which the goals of the program have been achieved by developing measures
of the program goals, administering them, and then using the results to make decision
about continuing or modifying the project.
CIPP (Stufflebeam & Shinkfield, 2007) was the model chosen to guide the
evaluation because the goal of the study is to allow multiple inputs from all stakeholders
Dissertation Proposal Components – 1_Sonya Stevens 39
in order to understand Childcare monitoring consistency between licensors in action (Fry
& Hammer, 2012). Table 3 presents a timeline that will guide the program evaluation
and shows that the project will be initiated by engaging representative stakeholders (e.g.,
childcare centers, licensors, and Department of Early Learning leadership). The process
will begin with a context evaluation: The intent of this stage is to assess and choose an
appropriate strategy to identify and resolve program problems (Frye & Hemmer, 2012).
An Ad Hoc focus group consisting of licensors and licensing supervisors from across the
state as well as a representative from Information Technology (IT) will identify the
problems and needs of the current licensing checklist system and create the objectives
that will lead to program improvements. This will take place over a two-week period
with one to two meetings each week.
Next, an input evaluation will utilize the same Ad Hoc committee to identify the
WAC and RCW that will be placed on the core items of the focused monitoring checklist
as well as the procedures licensors will use in conjunction with the tool. The
identification of WAC and RCW will utilize a collection and analysis of those items most
cited within the current checklist system over that previous 12-month period leading to
significant risk to children. Significant areas of risk will be identified through those
citations that have led to injury or illness of children and or adverse actions to the
childcare license such as summary suspension, civil penalties, or revocation. Information
will be gathered through the FamLink database. The input evaluation will take place
over a 2 to 3- week period of time.
The process evaluation will consist of a pilot where 4 to 8 sites will be evaluated
using the focused monitoring tool and identified procedures over a one-month period of
Dissertation Proposal Components – 1_Sonya Stevens 40
time. In order to identified the usefulness of the focus monitoring tool for licensor (social
validity), post focus groups of licensors and their supervisors will be gathered to discuss
the findings as a baseline for the formative and summative assessments. Interviews of
the providers will also be used to assess the social validity of the focused monitoring tool
from the licensee perspective. Finally, the final months of the study will consist of the
product evaluation where the researcher will gather and analyze the data to determine the
extent to which the goals of the program have been achieved. The results will be used to
make recommendations of the reliability and validity of the focused monitoring tool as
well as any recommendation to the continuation and needed modifications of the project
(Stufflebeam & Shinkfield, 2007).
Table 3: Project timeline
Dependent on IRB approval in September 2016
September-November(a) Context evaluation
Facilitate internal (licensing staff and supervisors) focus meetings to identify current challenges and needs of the monitoring system
December(b) Input evaluation
Identify and contact providers for participation Identify and contact center licensor from each region
for participation. Facilitate focus meetings to design the focus
monitoring tool (four meetings minimum) Collect data from the DEL database to create the
checklist specific to each site previously identified
January-February(c) Process evaluation
Conduct licensor training for the use of the monitoring tool
Conduct monitoring visits with the focused monitoring tool
Conduct post visit interviews with the licensors and providers
February-April(b) Product Evaluation
Complete final dissertation project Present finding to the appropriate
stakeholders/defense
Dissertation Proposal Components – 1_Sonya Stevens 41
Overall, descriptive statistics and input analysis will be conducted to the target
sample population in the form of focus groups and interviews pre and post pilot
implementation. This will gather information regarding intended changes as well as the
relations between monitoring procedures and licensor reliability through literature review
and focus groups. Licensors and providers will be identified and invited to participate in
a controlled study to discover how using the focused monitoring checklist system effects
the consistency of annual regulatory oversight. The overall design of the study requires
an understanding of how licensor reliability and validity affects the outcomes of health
and safety for children in care; making the evaluation method appropriate for this study.
Population
According to a report by Childcare Aware of Washington (Washington’s source
for Childcare resource and referral), there were more than 23,500 requests for referrals
for families seeking licensed care in Washington State in 2015 (Kelly, 2016). Of the age
of children seeking care, about half were for children 3 years and younger. Of those who
provided demographical information, roughly half the children needing care were from
single-parent families and more than 80% considered themselves low income. Licensed
facilities available to meet the need totaled more than 5,700 including Childcare centers,
family home providers and school age programs with a Childcare capacity of over
166,000 (Kelly, 2016).
Childcare centers are located throughout Washington State and is inclusive of
both urban and rural settings. According to Washington States’ Office and Financial
Management Population Trends Report (2015), Washington has a population of over 7
million individuals with more than 15 main languages spoken; The most prevalent used
Dissertation Proposal Components – 1_Sonya Stevens 42
in Childcare centers being English, Spanish, Russian, Somali and Vietnamese. The
licensing division is separated into four different regions. Region 1 (Eastern Region) is
the largest geographically covering all 20 counties west of the Cascade mountains. This
region provides oversight from four licensing offices located Yakima, Wenatchee, Tri-
Cities and Spokane and is responsible for approximately 1548 childcare providers
speaking. Region 2 (Northwest) contains the five northwestern counties in the state and
provides oversight from four offices located in Bellingham, Mount Vernon, Everett and
Bellevue. Region 2 is responsible for approximately 1412 licensed providers. Region 3
(Southwest) contains 13 counties northwest of the San Juan Islands down to the Oregon
border and has about 1431 licensee. Region 3 has three main licensing offices in
Olympia, Tacoma, and Vancouver with three satellite offices in Aberdeen, Kelso and
Port Angels. Finally, Region 4 (South King County) is located in South King County and
provides oversight by three licensing offices in Renton with approximately 1366
licensees (DEL, 2015). Figure 1 shows the licensing map by region and office.
Figure 1: DEL Childcare Licensing Map
To be a licensor in the state of Washington applicants must have at minimum of
one-year experience as a social worker or have a master’s degree in social services,
human services, behavior sciences or a related field as well as related work experience.
Dissertation Proposal Components – 1_Sonya Stevens 43
Also a licensor may be hires if they have a bachelor’s degree in those same fields of
study and three years of equivalent experience. Additionally, licensors are expected to
have knowledge of child development, risk assessments, interview techniques, service
delivery, community resources and others. Childcare licensors in Washington State vary
in age, gender and ethnicity and all are fluent in English.
Sample
Four licensing field offices, one from each of the four regions of Washington
State (Eastern, northwest, South King County and Southwest) will be selected to
participate in the study. One or two licensors from each office will be recruited for a total
sample of 4-8 licensors. The sample will be generated from data gathered through DEL’s
internet database called Toolkit. The licensor Toolkit is an electronic system that is used
to track due dates of required licensor actions including complaint inspection timelines as
well as due dates for annual monitoring visits, health and safety rechecks, and more.
Individuals vary in gender, age and ethnicity and all speak English proficiently.
Four to eight different center licensors will be recruited from varying locations
around Washington State. Provider sites will be identified by searching through the
licensor Toolkit to select multiple sites from the participating licensor database with
annual monitoring visits due during the study timeline. Office supervisors will be
consulted to identify those sites who might be interested in the participating in the study
and then the researcher will contact the site administrator to recruit the provider
participants. It is possible not all providers are English speakers; therefore, it may be
important to consider enlisting translators for the interview process. Provider interviews
Dissertation Proposal Components – 1_Sonya Stevens 44
will be conducted in the providers’ place of business or by phone as to not require
participants to travel or unduly interrupt daily operations.
Materials/Instruments
Context. In order to apply the study to the appropriate context, archival data will
be presented at the first round of Ad Hoc meetings including foundational information
uncovered during the literature review process to create a shared understanding of the
monitoring methodology and process. Additional data for the context evaluation will be
collected within the DEL internal databases previously described to adequately identify
the strength and weaknesses within the current annual monitoring system. This two-
week process will allow the identification of needed program improvements and the
creation of program objectives.
Input. Input evaluation data will be collected through focus group meeting notes,
electronic input from licensing database and post-pilot interviews. A second round of
meetings with key stakeholders, including program leaders and licensing staff will be
used to identify procedures that will be utilized in the input phase of the study as well as
identify and incorporate the political climate factors that could influence the success of
the study. Data collection in this phase of the study will include both formative and
substantive measures in the form of document analysis from meting notes and interviews.
A second set of archival data will be collected from the DEL internal data base FamLink
to identify the focused monitoring checklist contents. FamLink is a Web based system
used to track monitoring information recorded from licensors including date and time of
monitoring visits, narrative notes and documented areas of non-compliance in the form of
Facility Licensing Compliance Agreements (FLCA) (Del.wa.gov, 2016). Field notes
Dissertation Proposal Components – 1_Sonya Stevens 45
written by the licensors and information provided on the FLCA associated with the
identified participants will be collected to provide a partial baseline for the focused
monitoring tool including but not limited to specific standards in which a provider has a
history of non-compliance and citations leading to adverse actions such as civil penalties,
summary suspension and license revocation.
Narrative data will be collected through pre-participant interviews to establish a
baseline of social validity of a focused checklist system. Interviews will be completed
with each of the participating licensors and providers containing 15 questions, open and
close ended, pertaining to monitoring visits, the use of the differential checklist system,
and the value of Washington Administrative Code (WAC). Questions will be both open
and closed ended such as: What are the expectations you have during monitoring visits?
What are the Washington Administrative Codes and/or Revised Codes of Washington
(RCW) that justifies a move from an abbreviated checklist to a comprehensive checklist?
What is the process of completing a checklist upon entry into the site and why? Still to
be developed includes interview protocol and finalized questions as well as any survey
materials and apparatus.
Process. The first step of the process evaluation is to conduct licensor training in
which to evaluate the reliability and validity of the monitoring tool. This will be done on
a one-day workshop with the presentation of the guidelines identified by the Ad Hoc
committee through paper and PowerPoint presentations. Once this is completed, the pilot
using the focused monitoring system will be implemented with specific measures of
inter-rater reliability considered.
Dissertation Proposal Components – 1_Sonya Stevens 46
Data in the process evaluation will be collected by information from the focused
monitoring checklists system as well as provider post visit surveys. Current licensing
checklists are on a data base called Electronic Licensing Forms (ELF) and may not be
able to be altered to fit the needs of a focused monitoring checklist. Therefore, the tool
used for the pilot will most likely be in a paper format of the checklist system designed
by the Ad Hoc committee with specific input from the IT representative with a focus on
future implementation abilities. Completed forms will be submitted to the researcher and
will be stored on a secure state computer hard drive with a backup on the researchers
personal USB drive with any confidential information redacted.
Post interview questions will then be used to collect additional data regarding the
usefulness and validity of the focused monitoring tool. They will take place using an
unstructured format in order to allow for a wide range of input via phone calls or in
person interviews. Interviews will be completed with each of the licensors and providers
containing approximately 15 questions, open and close ended, pertaining to the focused
monitoring visits. Questions will be both open and closed ended such as: How did the
monitoring visit meet your expectations? What was the value you perceived about the
focused monitoring system? What were the challenges found while completing the
focused monitoring system? What was the process of completing a checklist upon entry
into the site and why? Still to be developed includes interview protocol and finalized
questions as well as any interview materials and apparatus.
Product. The product evaluation will measure the extent to which the goals of
the program have been achieved. This section of the study is where the licensor activities
are documented and assessed by the researcher. This study will focus on a close
Dissertation Proposal Components – 1_Sonya Stevens 47
examination of the oversight outcomes and any other relevant groups or individuals
affected by the study with a special focus on the social validity and consistency of
licensors focused monitoring outcomes (Frye & Hemmer, 2012; Strain et al., 2012;
Stufflebeam & Shinkfield, 2007). Additionally, it will evaluate the effects of the changes
on practice and outcomes including the positive and negative effects along with possible
short and long term effects (Frye & Hemmer, 2012).
The tools used to accomplish this task will include ?. Results will inform
program improvement of Childcare licensing monitoring tools, training procedures and
the successes and challenges of a collaborative monitoring process.
Data Collection, Processing and Analysis
Context. Background summation and analysis will be conducted by the
researcher over a period of one month during the first stage of the four-month study and
will done from the researcher’s place of study. It will include an analysis and written
summation in a single written document of existing documents inclusive of previous
licensing analyses, Ad Hoc meeting notes, policies regarding the current monitoring
process, stakeholder input and any other narrative information collected in the context
portion of the study. This will then be shared with the focus group to create a shared
understanding of the process goals as well as the methodology for the focused monitoring
system.
Input. Ad Hoc meeting notes will be complied and used as the baseline analysis
for the creation of the focused monitoring tool using. Subsequent data will be collected
through CIPP surveys including both closed- and open-ended questions. The pre and
post interviews will be conducted of the research participants by the researcher at the
Dissertation Proposal Components – 1_Sonya Stevens 48
participants’ place of business as to not place undue burden on the participants.
Interviews will be taped and transcribed verbatim and entered into the coding software
database called NVivo™ for organization and analysis. This will be done over a three-
week time period following the first month of the project by the researcher. Descriptive
analysis will be applied to the data collected from the CIPP survey close-ended questions.
The data collected from the open-ended question will be analyzed using the NVivo™
coding analysis software with data categories found within significant statements.
Codes will be taken from primary data collection and may include themes from
key words or phrases such as: Indictors, Fiene indicators, administrative code,
compliance, non-compliance, technical assistance, interruption/interruptive, routine,
safety, Facility Licensing Compliance Agreement and so on. These words will then be
clustered together into five main categories such as regulation (RE), technical assistance
(TA), routine (RO), health (HE), and safety (SA). Codes within the categories may
include critical indicators (CI), non-critical indicators (NCI), Fiene indicators (FI),
differential monitoring (DM), Washington Administrative Code (WAC), and so on.
Codes will be adjusted as needed based on the interview outcomes. Data will be
compiled and cross compare in order to establish commonalities in practice and thought
which will be incorporated into the focused monitoring tool.
Process. Once the focused monitoring process is developed, licensors will be
trained to use the tool to monitor identified sites following a developed protocol over a
one-month time frame at the licensed facility. Licensors will conduct the unannounced
monitoring visit within the required timeline using the paper format of the monitoring
checklist system. The checklist will contain a series of standards to be observed during
Dissertation Proposal Components – 1_Sonya Stevens 49
the site visit within each of the main identified sections. The focused checklist may
include 2 to 3 WACs that put children at the highest risk (weighted), 2 WACs identified
as Key Indicators, 1-3 WACs with a history of compliance issues specific to that site, and
at least 1 random WAC for a total of no more than 8 WACs in each of the eight standard
areas (Staff requirements, cleanliness and sanitation, programming, environment, etc.).
The amount and combination of WACs to be included on the checklist may be altered
during the input phase of the study. Each of these items will be marked by the licensors
as compliant, non-compliant, or not observed. If any of the identified WACs are found to
be non-compliant in a section, the licensor will complete the comprehensive checklist for
that section only and then move onto the next section repeating the process for each
section. Once the checklist is complete the licensors will complete the electronic version
of the Facility Licensing Compliance Agreement (FLCA) with the provider in accordance
to DEL monitoring policy.
Licensors will record visit results and narratives within the current electronic
database and send an electronically scanned copy of the focused monitoring checklist to
the researcher via secured state email. Information will be gathered under the new
operating systems to identify critical consistencies of the monitoring visits during each
licensors monitoring visit. This will include, but may not be limited to the length of visit,
the amount and type of technical assistance provided by the licensor, and the amount and
type of areas found in non-compliance and so on. Information will be transferred onto an
Excel spreadsheet and stored on the researchers secured hard drive. Post-interviews will
be conducted by the researcher for analysis.
Dissertation Proposal Components – 1_Sonya Stevens 50
Product. Narrative data will be gathered through post interviews conducted by
the researcher with the providers and licensors as well as the licensors written narratives.
This will take place in the final two months of the six-month study and at the
participants’ place of work as to not place undue burden on the participants. Post
monitoring visit interviews will be conducted using the same guidelines as the interviews
in the input phase of the study. Once again, interviews will be taped and transcribed
verbatim and entered into the coding software database called NVivo™ for organization
and analysis. Descriptive analysis will be applied to the data collected from the close-
ended questions. The data collected from the open-ended question will be analyzed using
the NVivo™ coding analysis software using the same data categories used previously.
This will allow for a cross analysis of the data to evaluate the effects of the focused
monitoring system on programming and general value of the monitoring system.
Summary
The responsibility for compliance at all times ultimately lies with the licensed
Childcare provider. Previous research has concentrated on the validation of tools such as
standardized health and safety instruments, key indicators and differential monitoring
(Alkon et al., 2015; Fiene, 2014; Fiene, 2013). However, the best application of this
program evaluation study is to re-assess and make improvements of existing licensing
oversight monitoring tools in an attempt to improve licensor consistency through focused
technical assistance.
What is unique to this mixed method academic research project is it is centered on
how these systems can be used in conjunction with a focused monitoring system. This
study is designed to concentrate on how using the indicators and weighted indicator
Dissertation Proposal Components – 1_Sonya Stevens 51
system could potentially be used in conjunction with oversight in only the areas found in
non-compliance and if focused monitoring visits will increase social validity and
programming quality for children. As more and more children are entering into the
licensed Childcare systems it is critical to re-assess and maintain systems that provide
appropriate oversight, and ensuring each child’s health and safety.
Dissertation Proposal Components – 1_Sonya Stevens 52
References
Ackerman, D. J., & Sansanelli, R. A. (2010). The source of childcare center preschool learning and program standards: Implications for potential early learning challenge fund grantees. Early Childhood Research & Practice, 12(1).
Administration for child and families. (2016). OCC Fact Sheet. Retrieved from http://www.acf.hhs.gov/programs/occ/fact-sheet-occ
Alkon, A., Rose, R., Wolff, M., Kotch, J. B., & Aronson, S. S. (2015). Health and safety checklist for early care and education programs to assess key national health and safety standards. Maternal and Child Health Journal, doi:10.1007/s10995-015-1809-8
Amirkhanyan, A. A., Kim, H. J., & Lambright, K. T. (2014). The performance puzzle: understanding the factors influencing alternative dimensions and views of performance. Journal of Public Administration Research & Theory, 24(1), 1-34.
Arteaga, I., Humpage, S., Reynolds, A. J., & Temple, J. A. (2014). One year of preschool or two: Is it important for adult outcomes?. Economics of Education Review, 40221-237. doi:10.1016/j.econedurev.2013.07.009
Artz, B., & Welsch, D. M. (2014). Childcare quality and pricing: evidence from Wisconsin. Applied Economics, 46(35), 4276-4289. doi:10.1080/00036846.2014.955256
Batra, S. (2013). The psychosocial development of children: Implications for education and society--Erik Erikson in context. Contemporary Education Dialogue, 10(2), 249-278.
Boller, K., Paulsell, D., Grosso, P. D., Blair, R., Lundquist, E., Kassow, D. Z., & ... Raikes, A. (2015). Impacts of a childcare quality rating and improvement system on childcare quality. Early Childhood Research Quarterly, 30(Part B), 306-315. doi:10.1016/j.ecresq.2014.10.001
Bromer, J., McCabe, L. A., & Porter, T. (2013). Special section on understanding and improving quality in family childcare: Introduction and commentary. Early Childhood Research Quarterly, 28(4), 875-878. doi:10.1016/j.ecresq.2013.08.003
Campbell-Barr, V. (2012). Early years education and the value for money folklore. European Early Childhood Education Research Journal, 20(3), 423. doi:10.1080/1350293X.2012.704764
Côté, S. M., Mongeau, C., Japel, C., Xu, Q., Séguin, J. R., & Tremblay, R. E. (2013). Childcare quality and cognitive development: Trajectories leading to better
Dissertation Proposal Components – 1_Sonya Stevens 53
preacademic skills. Child Development, 84(2), 752-766. doi:10.1111/cdev.12007
Côté-Lussier, C., & Fitzpatrick, C. (2016). Original article: Feelings of safety at school, socioemotional functioning, and classroom engagement. Journal of Adolescent Health, 58543-550. doi:10.1016/j.jadohealth.2016.01.003
Crowley, A. A., Sangchoon, J., & Rosenthal, M. S. (2013). Health and safety of childcare centers: An analysis of licensing specialists' reports of routine, unannounced inspections. American Journal of Public Health, 103(10), e52-e58. doi:10.2105/AJPH.2013.301298
Connors, M. C., & Morris, P. A. (2015). Comparing state policy approaches to early care and education quality: A multidimensional assessment of quality rating and improvement systems and childcare licensing regulations. Early Childhood Research Quarterly, 30(Part B), 266-279. doi:10.1016/j.ecresq.2014.05.006
Del.wa.gov. (2016). About licensed childcare. Retrieved from http://www.del.wa.gov/care/about/Default.aspx
D’Souza, J., & Gurin, M. (2016). The universal significance of Maslow’s concept of self-actualization. The Humanistic Psychologist, 44(2), 210-214. doi:10.1037/hum0000027
Elicker, J., Ruprecht, K. M., Langill, C., Lewsader, J., Anderson, T., & Brizzi, M. (2013). Indiana Paths to QUALITY™: Collaborative evaluation of a new childcare Quality Rating and Improvement System. Early Education & Development, 24(1), 42-62. doi:10.1080/10409289.2013.736127
Fiene, R. (2013). Differential monitoring logic model and algorithm (DMLMA)©: A new early childhood program quality indicator model (ECPQIM4)© for early care and education regulatory agencies. Middletown, PA: Research Institute for Key Indicators. Retrieved from http://www.researchconnections.org/childcare/resources/24950/pdf.
Fiene, R. (2014). Georgia childcare licensing study: Validating the core rule differential monitoring system. Research Institute for Key Indicators. Middletown; PA. Retrieved from http://www.researchconnections.org/childcare/resources/30431
Fiene, R. (n.d.). The relationship of accreditation and licensing standards, training, and child development outcomes. Retrieved from http://harrisburg.psu.edu/capital-area-early-childhood-training-institute/publications
Fiene, R., & Carl, B. (2010). 13 key indicators of childcare quality: Childcare quality indicators (CCQ1 – CDPES2 PC1) scale. Retrieved from http://harrisburg.psu.edu/sites/default/files/capital-area-early-childhood-training-
Dissertation Proposal Components – 1_Sonya Stevens 54
institute/13keyindicatorsofchildcarequalitychildcarequalityindicatorsccqi-cdpes2pc1scale.pdf
Fiene, R., & Nixon, M. (1985). The instrument based program monitoring information system and the indicator checklist for childcare. Childcare Quarterly, 14(3), 198-214. doi:10.1007/BF01113748
Floyd, M. L., & Phillips, D. A. (2013). Childcare and other support programs. Future of Children, 23(2), 79-97.
Frye, A. W., & Hemmer, P. A. (2012). Program evaluation models and related theories: AMEE Guide No. 67. Medical Teacher, 34(5), e288-e299. doi:10.3109/0142159X.2012.668637
Halle, T. G., Whittaker, J. V., Zepeda, M., Rothenberg, L., Anderson, R., Daneri, P., & ... Buysse, V. (2014). Review: The social–emotional development of dual language learners: Looking back at existing research and moving forward with purpose. Early Childhood Research Quarterly, 29734-749. doi:10.1016/j.ecresq.2013.12.002
Harms, T., Clifford, R. M., & Cryer, D. (1998). Early Childhood Environment Rating Scale, Revised Edition. New York, NY: Teachers College Press.
Heckman, J. J. (2011). The economics of inequality: The value of early childhood education. American Educator, 35(1), 31-35.
Hotz, V. J., & Xiao, M. (2011). The impact of regulations on the supply and quality of care in childcare markets. American Economic Review, 101(5), 1775-1805. doi:10.1257/aer.101.5.1775
Hudson, G., Miller, G. G., & Seikel, K. (2014). Regulations, policies, and guidelines addressing environmental exposures in early learning environments: a review. Journal of Environmental Health, 76(7), 24-34.
Huston, A. C., Bobbitt, K. C., & Bentley, A. (2015). Time spent in childcare: How and why does it affect social development?. Developmental Psychology, 51(5), 621-634. doi:10.1037/a0038951
Hyde, B. (2011). Childcare licensing in Washington: A report to the Washington State Legislature. Washington State Department of Early Learning. Retrieved from http://www.del.wa.gov/publications/research/docs/LicensingProvisoReport11511.pdf
Karoly, L. A., Zellman, G. L., & Perlman, M. (2013). Understanding variation in classroom quality within early childhood centers: Evidence from Colorado's
Dissertation Proposal Components – 1_Sonya Stevens 55
quality rating and improvement system. Early Childhood Research Quarterly, 28645-657. doi:10.1016/j.ecresq.2013.05.001
Kayira, T. (2016). Fundamentals for effective monitoring of child-care institutions. Commonwealth Law Bulletin, 42(1), 105-124. doi:10.1080/03050718.2016.1143380
Kirby, G., Caronongan, P., Malone, L. M., & Boller, K. (2015). What do quality rating levels mean? Examining the implementation of QRIS ratings to inform validation. Early Childhood Research Quarterly, 30(Part B), 291-305. doi:10.1016/j.ecresq.2014.08.006
Kelley, S. (2016). 2015 Childcare data report: Trends, childcare supply, cost of care, & demand for referrals. Retrieved from http://wa.childcareaware.org/about-us/data
Kohl, D., Recchia, S., & Steffgen, G. (2013). Measuring school climate: An overview of measurement scales. Educational Research, 55(4), 411-426.
Lahti, M., Elicker, J., Zellman, G., & Fiene, R. (2015). Approaches to validating childcare quality rating and improvement systems (QRIS): Results from two states with similar QRIS type designs. Early Childhood Research Quarterly, 30(Part B), 280-290. doi:10.1016/j.ecresq.2014.04.005
La La Paro, K. M., Hamre, B. K., & Pianta, R. C. (2012). Classroom Assessment Scoring System toddler.
Lapp-Payne, A. & Testa, M. S. (2014). Protecting children's health and safety in every setting. Exchange (19460406), (220), 8-12.
Ledford, J. R., Hall, E., Conder, E., & Lane, J. D. (2016). Research for young children with autism spectrum disorders: Evidence of social and ecological validity. Topics in Early Childhood Special Education, 35(4), 223-233.
Li, W., Farkas, G., Duncan, G.J., Burchinal M.R., & Vandell, D. (2013). Timing of high-quality childcare and cognitive, language, and preacademic development. Developmental Psychology Association, 49(8), 1440-1451. doi:10.1037/a0030613
Lipscomb, S. T. (2013). Increasing access to quality childcare for children from low-income families: Families' experiences. Children and Youth Services Review, 35411-419. doi:10.1016/j.childyouth.2012.12.020
Mayer, D., & Beckh, K. (2016). Examining the validity of the ECERS-R–R: Results from the German National Study of Childcare in Early Childhood. Early Childhood Research Quarterly, 36415-426. doi:10.1016/j.ecresq.2016.01.001
Dissertation Proposal Components – 1_Sonya Stevens 56
Meloy, M. E., Lipscomb, S. T., & Baron, M. J. (2015). Linking state childcare and child welfare policies and populations: Implications for children, families, and policymakers. Children and Youth Services Review, 5730-39. doi:10.1016/j.childyouth.2015.07.008
Miller, A. (2013). Potential dangers of unlicensed daycares. CMAJ: Canadian Medical Association Journal, 185(18), 1566. doi:10.1503/cmaj.109-4643
Moloney, M. (2016). Childcare regulations: Regulatory enforcement in Ireland. What happens when the inspector calls?. Journal of Early Childhood Research, 14(1), 84-97. doi:10.1177/1476718X14536717
NCCCQI. (2013). Trends in childcare center licensing requirements and policies for 2011. Retrieved from https://childcareta.acf.hhs.gov/resource/research-brief-1-trends-child-carecenter-licensing-regulations-and-policies-2011
National Association for the Education of Young Children. (2016). Eligibility requirements. Retrieved from http://www.naeyc.org/academy/pursuing/eligibilityreq
National Center on Childcare Quality Improvement. (2014). Contemporary issues in licensing: Monitoring strategies for determining compliance: Differential monitoring, risk assessment, and key indicators. Fairfax, VA: National Center on Childcare Quality Improvement. Retrieved from https://childcareta.acf.hhs.gov/sites/default/files/1408_differential_monitoring_final_1.pdf.
Palley, E. (2012). Expected struggles: U.S. childcare policy. Children and Youth Services Review, 34628-638. doi:10.1016/j.childyouth.2011.12.007
Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008). Classroom Assessment Scoring System manual pre-K. Baltimore, MD: Brookes
Raikes, H., Torquati, J., Wang, C., & Shjegstad, B. (2012). Parent experiences with state childcare subsidy systems and their perceptions of choice and quality in care selected. Early Education & Development, 23(4), 558-582. doi:10.1080/10409289.2011.574268
Rodriguez, B. J., & Garza, S. (2014). Test review: Classroom Assessment Scoring System manual: Toddler. Journal of Psychoeducational Assessment, 32(6), 573-576. doi:10.1177/0734282914523085
Sandstrom, H., & Chaudry, A. (2012). ‘You have to choose your childcare to fit your work’: Childcare decision-making among low-income working families. Journal of Children & Poverty, 18(2), 89. doi:10.1080/10796126.2012.710480
Dissertation Proposal Components – 1_Sonya Stevens 57
Scopelliti, M., & Musatti, T. (2013). Parents' view of childcare quality: Values, evaluations, and satisfaction. Journal of Child & Family Studies, 22(8), 1025. doi:10.1007/s10826-012-9664-3
Somers, T. S., Harvey, M. L., & Rusnak, S. M. (2011). Making childcare centers SAFER: a non-regulatory approach to improving childcare center siting. Public Health Reports (Washington, D.C.: 1974), 126 Suppl 134-40.
Strain, P. S., Barton, E. E., & Dunlap, G. (2012). Lessons learned about the utility of social validity. Education & Treatment of Children, 35(2), 183-200.
Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluation Theory, Models, and Applications. Jossey-Bass, an imprint of Wiley.
Tucker-Drob, E. (2012). Preschools reduce early academic-achievement gaps: A longitudinal twin approach. psychological science, 23(3), 310-319.
Villanueva, K., Badland, H., Kvalsvig, A., O'Connor, M., Christian, H., Woolcock, G., & ... Goldfeld, S. (2016). Perspective: Can the neighborhood built environment make a difference in children's development? Building the research agenda to create evidence for place-based children's policy. Academic Pediatrics, 1610-19. doi:10.1016/j.acap.2015.09.006
van Dijken, M., Stams, G., & de Winter, M. (2016). Can community-based interventions prevent child maltreatment?. Children and Youth Services Review, 61149-158. doi:10.1016/j.childyouth.2015.12.007
Washington State Legislature. (2016). Revised Codes of Washington. Retrieved from http://apps.leg.wa.gov/rcw/
Washington State Legislature. (2016). Washington Administrative Codes. Retrieved from http://apps.leg.wa.gov/wac/
Winterbottom, C., & Jones, I. (2014). National accreditation and its role in early education: An analysis of Florida’s Gold Seal Quality Child-Care program and licensing standards. Journal of Early Childhood Research, 12(1), 64. doi:10.1177/1476718X13492942
Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214.
Yazejian, N., Bryant, D., Freel, K., & Burchinal, M. (2015). High-quality early education: Age of entry and time in care differences in student outcomes for English-only and dual language learners. Early Childhood Research Quarterly, 3223-39. doi:10.1016/j.ecresq.2015.02.002
Dissertation Proposal Components – 1_Sonya Stevens 58
Zellman, G. L. and Fiene, R. (2012). Validation of quality rating and improvement systems for early care and education and school-age care, research-to-policy, research-to-practice brief OPRE 2012. Washington, DC: Office of Planning,
Dissertation Proposal Components – 1_Sonya Stevens 59
Appendices
Appendix A: Analysis of Licensor Compliance
Dissertation Proposal Components – 1_Sonya Stevens 60
Appendix B: DEL Monitoring Policy
Dissertation Proposal Components – 1_Sonya Stevens 61
Dissertation Proposal Components – 1_Sonya Stevens 62
Appendix C: DEL Abbreviated Monitoring Checklist
Dissertation Proposal Components – 1_Sonya Stevens 63
Dissertation Proposal Components – 1_Sonya Stevens 64
Dissertation Proposal Components – 1_Sonya Stevens 65
Dissertation Proposal Components – 1_Sonya Stevens 66
Dissertation Proposal Components – 1_Sonya Stevens 67
Dissertation Proposal Components – 1_Sonya Stevens 68
Appendix D: DEL Comprehensive (Long) Monitoring Checklist
nsive
Dissertation Proposal Components – 1_Sonya Stevens 69
Dissertation Proposal Components – 1_Sonya Stevens 70
Dissertation Proposal Components – 1_Sonya Stevens 71
Dissertation Proposal Components – 1_Sonya Stevens 72
Dissertation Proposal Components – 1_Sonya Stevens 73
Dissertation Proposal Components – 1_Sonya Stevens 74
Dissertation Proposal Components – 1_Sonya Stevens 75
Dissertation Proposal Components – 1_Sonya Stevens 76
Dissertation Proposal Components – 1_Sonya Stevens 77
Dissertation Proposal Components – 1_Sonya Stevens 78
Dissertation Proposal Components – 1_Sonya Stevens 79
Dissertation Proposal Components – 1_Sonya Stevens 80