1

EEC Board MeetingJune 11, 2013

Quality Rating and Improvement System (QRIS)Review of Standards Removal

Board Vote Feb 12, 2013

Quality Rating and Improvement System (QRIS)

Massachusetts has developed high quality benchmarks, which are aligned with state early childhood standards (MA Early Learning Guidelines for Infants & Toddlers, Preschool Learning Guidelines and the Pre-K Common Core of the MA Curriculum Frameworks), as well as some Head Start, National Association of Educating Young Children accreditation, and other early learning and development standards.

2

There are five categories in QRIS:

1.) Curriculum and Learning a. Curriculum, Assessment and Diversity b. Teacher/Child Relationships and Interactions

2.) Safe, Healthy Indoor and Outdoor Environments

3.) Workforce Qualifications and Professional Development a. Designated Program Administrator Qualifications and

Professional Development b. Program Staff Qualifications and Professional Development

4.) Family and Community Engagement

5.) Leadership, Management and Administration

3

QRIS is a Comprehensive Assessmentthat Encompasses All Programmatic Aspects

Three Different Program Types

The five categories and two sub-categories remain consistent , however the individual standards have been adapted for effectiveness in three different program types:

Center-Based and School Based (CBSB) Family Child Care (FCC) After School and Out of School Time Programs (ASOST)

4

5

Development of the QRIS Standards Was a Community Effort

2008:• EEC began conducting early research to set the stage for the design of the

MA QRIS. • EEC received guidance from EEC Board and EEC Advisory Team (February -

March 2008)• Initial QRIS stakeholder team developed the scope, purpose and mission of

MA QRIS. Evolved into MA QRIS

2009:• ASOST Stakeholders Group convened• Draft posted for public input- 377 survey respondents (March -June 2009 )• EEC conducted presentations to over 900 members of the field about QRIS• EEC reconvened the QRIS External Stakeholder team (June 2009)• Initial QRIS Standards were reviewed and revised – evidence based and

measurable (November 2009 – January 2010)• Feedback gathered from stakeholders at events with The CAYL Institute

(December 16, 2009)

2010 :•Feedback provided on QRIS Standards at Advisory Team Meeting & Stakeholders Meeting (January 2010) •EDC hired to review and streamline standards (reduce duplication, remove criteria with no research base)

2012:•Pilot for the QRIS Validation Study begins•Environmental Rating Scales (ERS) observation visits begin for 426 self-assessed Level 3 and 4 programs

6

Recent QRIS Successes Six Program Quality Specialists were hired and trained on the Quality Rating

and Improvement System

Pilot for the QRIS Validation Study started

FY13 QRIS Improvement Grants were awarded for durable goods and program planning

Many Center-Based and School Based, Family Child Care, and After School and Out of School Time programs sharing with PQS that they have been reflectively reviewing their program practices in a more comprehensive and thoughtful manner

Programs have become more familiar with the use of research-based measurement tools

Pilot study is in development with NAEYC to reduce duplication of effort by accredited Center-Based programs

Work has started on a more user-friendly version of the QRIS Program Manager (on-line application system)

Communication and collaboration with national organizations has improved Approval granted to reassess RTT benchmarks Coordination of QRIS and QRIS related programs will be managed

within the EEC Policy Department

Program Quality Specialists and Education Specialists Support QRIS

In collaboration, EEC staff has:

• Reviewed over 1,400 QRIS applications between October 1, 2012 and May 1, 2013

• Verified over 700 Level 2 applications between October 1, 2012 and May 1, 2013

• Completed approximately 200 site visits • Facilitated QRIS Training Sessions throughout the state• Provided daily QRIS support via the telephone and email• Created resources for QRIS users • Presented QRIS information at dozens of early education venues

throughout the state to educators, administrators and stakeholders• Held monthly TA with EPS Coaches/Mentors working with programs on

QRIS

Quality Rating and Improvement System Program Engagement

Since the launch of the on-line, QRIS Program Manager (QPM) in January 2011 5,149 programs (unique count) have created a total of 6,704 QRIS applications (final status) using the QPM system. 5,571 of those applications have been granted a QRIS Level. (Data as of May 3, 2013).

Additional QRIS Data: There are 363 Programs that have participated in QRIS that

have since closed for business. Note: Metro (region 4) has been dissolved and towns have been reallocated to

regions 2, 3, 5 and 6.

BACKGROUND ON STANDARDS REMOVALIn 2012, EEC contracted with University of Massachusetts Donahue

Institute (UMDI) to conduct the QRIS Validation Study.UMDI provided EEC with a measurement map of standards to be

included and excluded in the QRIS Validation Study.

CLARIFICATIONThe UMDI research team did make recommendations to include and

exclude some standards in the QRIS Validation Study. The UMDI research team did not suggest that any standards be

removed from the Massachusetts QRIS.

10

QRIS Standards Removed

On February 12, 2013, the EEC Board voted to “remove” QRIS standards from the system. EEC would like the Board to consider new information about this decision, and rescind the February vote.

The three reasons for removal were:

1.) Already measured in an evidenced based tool used in QRIS2.) Not linked to child outcomes 3.) Not measurable

11

12

Standard Category Family Child Care

Center Based

After School and Out of School Time

Already Measured in an evidenced based tool used in QRIS

8 11 9

Not linked to Child Outcomes

9 3 3

Not Measurable 4 12 3

Total Removed 21 26 15

Total in QRIS (excluding Level 1) 58 79 71

Implementation of the standards removal, based on the Board vote, has not been consistently implemented to date. The QRIS standards resource document for educators has been modified to note the Board vote on February 12, 2013.

13

14

15

The QRIS Program Manager (the on-line application system) does not yet support the standards removal Board vote.

BACKGROUND ON REMOVAL of standards measured elsewhere

The standards proposed to be removed are either measured elsewhere through evidence based tools, or are currently collected data (i.e. PQ Registry, BAS, PAS).

JUSTIFICATION FOR KEEPING IN QRIS

The QRIS Standards form the foundation of QRIS, not the measurement tools. If EEC decides in the future to change the measurement tools, the standards could be lost. The value of the measurement tools should not supersede the standards.

16

BACKGROUND ON REMOVAL of standards that do not have a link to child outcomes or are not measurable

Some standards have no link to child outcomes thus it is unclear of their connection to quality, while other standards are not measurable. Reasons for these two categories may include:

Some of the standards -• are considered vague• have compound criteria• are a simple yes/no measure

JUSTIFICATION FOR KEEPING IN QRIS

These standards may still have value in the QRIS system with revisions.

17

RECOMMENDATIONS FOR REVISIONS TO QRIS STANDARDS

Changes to the QRIS should be done thoughtfully and be informed by:

• Data from the QRIS Validation Study (UMDI)• Data from the Reliable Rater visits (Wellesley College)• Feedback from the field, stakeholders and community• Best practices shared with other QRIS states• Input from Program Quality Specialists and other EEC staff whom

support QRIS• Contributions from the EEC Board

18

Resources

Standards Removal Documents:

http://www.eec.state.ma.us/docs1/board-materials/20130212-qris-background-discussion.pdf

http://www.eec.state.ma.us/2013/20130212-qris-standards-removal.pdf

QRIS Resource Tools for Educators:http://www.mass.gov/edu/birth-grade-12/early-education-and-care/qris/massachusetts-qris-standards.html

19