qris standards learning table
DESCRIPTION
QRIS Standards Learning Table. Session #3: Efficiency in Monitoring: Streamlining Documentation. Introductions and Updates. Introduce the state team (Name, title, agency) AL, CA, CT, GA , HI , NV, OR, VI - PowerPoint PPT PresentationTRANSCRIPT
National Center on Child Care Quality Improvement
QRIS Standards Learning Table
Session #3: Efficiency in Monitoring: Streamlining Documentation
2
Introductions and Updates• Introduce the state team (Name, title, agency)
AL, CA, CT, GA, HI, NV, OR, VI• Describe what your state team has been doing with regard
to your QRIS since our last call. It could be related to the homework or other points of interest in your work.
• Share strategies your state is using to bring these concepts and materials back to workgroups within the state. (agendas, topics, etc.)
• If a certain resource or idea has been particularly helpful, tell us about that.
Key Point
Even a QRIS that appears simple can become complex and expensive to administer unless steps are taken to streamline the documentation procedures for standards and sources of evidence.
Sources of Evidence
For each standard you must:• Clarify if/when documentation is required
Example: If you’ve already seen a source of evidence in the past, do you need to see it each year?
• Specify what documentation can be accepted to verify compliance.
Efficiency Opportunity: Current Assessment Tools as Source of Evidence
• Some Program/Classroom Assessment tools measure the same content.
• Some Program/Classroom Assessment tools measure criteria included in a state’s QRIS.
• Thus, a QRIS could use an Assessment tool – such as ERS or PAS as the source of evidence.
Do the common tools measure the same concepts?
ECERS-R FCCERS-R CLASS PAS BAS
General Cognition Social & Emotional Development
Approaches to Learning Heath/Physical Development Business Practices Family Involvement Internal Communication Leadership/Management ?
Efficiency Opportunity: Self-Report
• What standards are most appropriately verified by self-report?
• What are effective procedures for validating self-reporting? Is random sampling appropriate?
• What documentation needs to be available for review?
Efficiency Opportunity: Automation
• How can automation streamline the monitoring process?– Links to data-bases for licensing, registry, CACFP,
subsidy, accreditation– Electronic scoring/reporting of ERS, CLASS, PAS/BAS– Director portals or on-line applications (in real time)
that enable programs to upload evidence– Provider-focused platforms that include downloadable
tools/templates to support compliance
Automation: Learning Table States
Results from both sessions:• Links to Registry: AR, DE, NH, OR, GA
(OK, CA, PA developing)• Links to Licensing: DE, KY, NM, OK, TX, OR, PA• Links to PreK Monitoring: NM• Links to Head Start Performance Review: AR, DE, OK• On-line Upload of Documentation: AR, NM
Case Study: Maine
Michel Lahti, PhDUniversity of Southern Maine
QUALITY FOR ME – THE BASICS
• Licensing compliance• Membership in MRTQ Registry• Online application based upon a self-evaluationOnce the on-line application is submitted, the provider
immediately receives feedback from the Quality for ME system regarding the anticipated Step level
• Portfolio of documentation (random)• On-site Observations (random)
The General Approach• Web-based application • Linkage to licensing database and PD Registry
– Relieves burden for all applicants– Improves data quality in QRS application– Feedback loop also improves data quality in linked database
• Criteria cross-walked with Accreditation criteria• Self-report on remaining items
– About 50 specific questions if no Accreditation– Reduced to just 5-10 questions depending on Accreditation
• Immediate feedback on how to move to next step in each area
• Individual and aggregate reports shared with R&R centers to facilitate technical assistance
QRS Step in Each of Eight Areas:• compliance history/licensing status• learning environment/developmentally appropriate
practice• program evaluation• staffing and professional development• administrative policies and procedures• parent/family involvement• family resources• authentic assessment
Criteria for achieving steps cross-walked with standards for the following:
• NAEYC Accreditation• NAEYC Candidacy• National Association of Family Child Care Providers Accreditation• National After School Association Accreditation• American Montessori Society Accreditation• Head Start: Zero Non-compliance Issues at Last Review / All
Non-compliance Issues at Last Federal Review Resolved
Program LicensingMeDHHS, Augusta program license # contact info capacity license status license expiration type of program …
Maine Roads To Quality (Prof Dev Registry)Univ of Southern Maine, Portland provider ID provider education provider training record license # of program where provider employed …
Quality Rating SystemUniv of Maine, Orono program license # self-reported data calculated data …
Maine Roads To Quality (Prof Dev Registry)Univ of Southern Maine, Portland program license # accreditation …
Key Data Linkages
Improves Data Quality at Linked Databases
Immediate Feedback to Applicant
Figure xx. Example of automatic immediate scoring report provider
Step Report
Section Name Steps Compliance History/Licensing Status 1
Learning Environment/Developmentally Appropriate Practice
2
Program Evaluation 1
Staffing and Professional Development 1
Administrative Policies and Procedures 4
Parent/Family Involvement 2 Community Resources 4
Child Observations 2
Overall the Program is at Step 1
Immediate Feedback to ApplicantFigure xx. Example of detailed automatic immediate feedback to provider on how to achieve next steps in each area of QRS evaluation (abbreviated).
Recommendations
Compliance History / Licensing Status
Current step is #1. In order to move to step #2:
Your facility must have no substantiated serious violations in the past year.
Learning Environment / Developmentally Appropriate Practice
Current step is #2. In order to move to step #3:
At least 50% of lead teachers (per program site) working with children ages 3-5 must have completed the training on implementing curriculum based on Maine’s Early Childhood Learning Guidelines.
Program Evaluation
Current step is #1. In order to move to step #2:
Your program must provide an opportunity to identify strengths and weaknesses that is inclusive of staff, families, and administrators
Staff must be given feedback regarding the yearly self assessment
Staffing and Professional Development
Current step is #1. In order to move to step #2:
Your program must hold staff meetings on monthly or more basis Are at least 50% of your lead teachers must be at a level 5 or above on the Maine Roads to Quality
Direct Care Career Lattice
Administrative Policies and Procedures
Current step is #4. This is the highest step. Congratulations!
Parent / Family Involvement
Current step is #2. In order to move to step #3:
Parents of infant and toddlers must be provided with a written daily communication about their child’s day
Community Resources
Current step is #4.
(specific recommendations for each of 8 areas)
Data Usage…
• Monitor Enrollments and Characteristics of Programs
• ERS Scores – Focus on Areas of Strength and Improvement
• Monitor Program Progress through Step Levels• Monitor Supports to Programs• Infrastructure for Evaluation Projects:
– Comparing QRIS to non-QRIS Sites– Investigate QRIS Standards: Use of Child Level
Assessments– Validation Study
Lessons Learned from Maine
• Intention is to Build a System, an Infrastructure to Help Align ECE Programming
• Develop Working Partnerships with State Program Administrators and University Research Staff
• System Operation Requires Ongoing Attention - Keep it Valid and Reliable
• Importance of Translating Data from QRIS Monitoring into Information for Decision-making
BENEFITS TO JOINING QUALITY FOR ME…
• Ability to accept Child Care Subsidy and receive a payment differential based upon Step Level
• Assistance in paying for Accreditation fees and cohort supports (some facility improvement grants)
• On-site technical assistance• Scholarships to pursue early childhood education
degrees• Tax credits for parents and providers
Automation of QRIS Implementation
Results from both sessions and other states included: • WELS (FL, NY, MS)• MOSAIC (MI, CA)• BRANAGH (LA)• State - Developed Systems (AZ, ME, GA, PA)
Georgia’s Online SystemBackground
Design - 2011-2012 Launched 1/2012
Equal emphasis on Process Quality (ERS) and Structural Quality (Program Portfolio
In-house design and development of online system to manage all of Quality Rated from process to data
ApplicationTraining/Technical Assistance – Registration to trackingPortfolio Submission – CQI PlansIncentives ManagementResourcesReports and Data
Communication
Quality Rated Components
Research Questions – Data Dictionary
• Validation and research guided development of online system
• With TA support from FPG– Developed logic model– Developed validation and evaluation model– Created data dictionary– Created reports
Validation Plan by Phase
Phase ValidationPhase 1Completed 12/2011
Content Validation (standards)
Phase 29/1/12 > = to 30 portfolios submitted
Inter-rater reliability (portfolio/ERS) Distribution of program standards and criteria Correlating components Portfolio accuracy - elements verified at ERS visit (long and short form)ERS distribution
Phase 33/2013
Distribution of rating levels- rating distributions by program type (HS, rural vs. urban types of children served, etc.)Testing of various scoring structures/weighting and cut-offs
Phase 32014
Meaningful differentiation of quality levels with another outside variable i.e. CLASS
Evaluation Plan
Type of evaluation questions
Year 12012-2013
Year 22013-2014
Year 32014-2015
Year 42015-2016
Year 52016-2017
Participation x x x x xRetention x x x x xMotivation x x x Quality Supports (TA, Training, Incentives, Tiered Reimbursement, Bonus Packages)
x x x
DECAL Resources and Processes
x x x x
Statewide Quality Improvement
x
Logic ModelMission Statement Quality Rated will improve the quality of early education and school age care programs through ‐
aligning and coordinating system-wide initiatives.
Long Term Goals
Years 5 and beyond
Families choose higher rated programs
25% of children receiving subsidy are in rated programs
Some programs increase quality by at least 1 level51% of Georgia counties have 51% or more of eligible programs participating Quality of ECE programs statewide has improvedFor programs rated within the last 4 years:• Enrolled programs continue to participate (measure)
Intermediate GoalsYears 3-4
Families choose Quality Rated Programs
20% of children receiving subsidy are in a Quality Rated program
For programs rated within the last 2 years:• Retention at 75%• Some programs increase quality by at least 1 levelRecruitment: • 25% of registered FCC homes participating in Quality Rated• 40% of licensed GDCH and Centers participating in Quality Rated• 50% of programs that receive CC subsidy are participating in Quality Rated
Short Term Goals
Years 1-2
Families have access to Quality Rated Programs
10% of children receiving subsidies are participating in a Quality Rated Program
Recruitment:• 10 % of registered FCC homes participating in Quality Rated• 15 % of licensed GDCH and Centers participating in Quality Rated• 75 % of programs participating are rated• 25% of programs serving subsidized children are participating in Quality
RatedDevelop a process to recruit and serve exempt but eligible entities (school districts w/GA Pre-k, Head Start, DOD)
CHILDREN and FAMILIES PROGRAMS
Output # of programs participating # of programs rated with quality improvement plan
# of programs receiving support
Activity Programs volunteer to participate in Quality Rated, ratings assigned, supports to improve quality (TA , PD, bonuses, incentives, tiered reimbursement, public awareness campaign)
Input Stakeholder developed Quality Rated standards, public and private funding, research findings
Ongoing Validation and Evaluation
Online Site Users
• Quality Rated staff• Technical assistance staff• Resource and referral agencies• Programs enrolled in QR• Incentive partners• Research team• Parents in 2013 – will see levels
Lessons Learned in Georgia
• Keep it simple• Resources make all of the difference• Transparency• INVOLVE THE RESEARCHERS
Web-Based Supports for Providers
The Raise Quality Tab on ECESharedResources.org:http://national.ecesharedresources.net/index/
• SharedSource PA, • Child Care Tennessee, • New Mexico Early Learning Alliance• Oregon• Maine
Efficiency Opportunity: Multi-Site Centers
• How is documentation streamlined for multi-site centers?
• What information can be gathered from the central office?
• What must be gathered at each site?• What standards might be revised given a
multi-site management framework?
QRIS Administration withMulti-Site Centers: State Examples
• New Mexico – Verify documentation at central office
• Oklahoma – Head Start participation• Georgia – Cohort structure
Verification: Effective and Efficient?
Standards Think Tank Participants thought the most effective and efficient verification methods were:• Objective Third Party Observation/Assessment• Electronic Link to Licensing, Registry or other
Official Database• Self-Report with Verification of Random
Sample
Questions, Reflections, Comments?
Next Session - Homework1. When you consider your state’s initial QRIS standards or in the early stages of implementation, what types of considerations are your team discussing?• Ease of administration of the QRIS• Standards that provide administrative or research data (e.g., must enroll in the state Professional
Development Registry)• Ease of participation for early care and education programs in QRIS (e.g., Do the standards begin very low to
entice enrollment? How rigorous is the highest level?)• Research base for the standards• Standards that address emerging issues (e.g., diversity, child assessment, reflective practice)• Alternate pathways for various provider types (under what circumstances for which types of providers)• Other2. What data or research did you use to guide your selection of QRIS standards and what type of data are you collecting to guide future revisions?• Information from participants in the QRIS (programs, providers, parents)• Data from your QRIS management system• State Research • National Research• Other
Thank You
NCCCQI does not endorse any non-Federal organization, publication, or resource.
Follow-up Contacts: [email protected]
[email protected] [email protected]
[email protected] [email protected]
[email protected]@icfi.com
www.qrisnetwork.org
National Center on Child Care Quality Improvement