effectively communicate pvaas with your board and community pennsylvania value-added assessment...
TRANSCRIPT
Pennsylvania Value-Added Assessment System (PVAAS):
Effectively Communicate PVAAS with your Board and Community
PVAAS Statewide Core TeamFall, 2011
Session Description This 60 minute session will provide strategies and
approaches for discussing PVAAS with your local school board and community, as well as communicating PVAAS results. Approaches that have been used by actual districts in Pennsylvania will be discussed. Materials and resources to use as tools for communication about PVAAS to your local school board and community will be reviewed. Opportunities for questions and discussion will be provided.
There are trainer notes included for this presentation.
Agenda Documents & Suggested Handouts to
use with School Board Members Considerations for Communicating
PVAAS to School Boards and Your Community
PVAAS Public Reporting Site Examining the School Board
PowerPoint template Questions
Suggested Approach for InitialSession with School Board
Keep it simple…keep it conceptual Discuss what PVAAS is and is not
• Start with a focus on Scatter Plots: Achievement vs. Growth Summarize important reports!
• Suggest starting with Reading and Math only• District Value-Added Report• Value-Added Summary Report (building level
results)• District (Single Grade) Projection Summary Report• Performance Diagnostic Summary Report (if needed)
Go For The BIG IDEAS!
Suggested Approach for Discussion with Community Members
Keep it simple…keep it conceptual Discuss what PVAAS is and is not Start with a focus on Scatter Plots: Achievement vs.
Growth (Across Grades Only!) Summarize important information!
• District & School Value-Added Across Grades Only
• School Search Average Growth Index Across Grades Only
Go For The BIG IDEAS!
Supports and Resources to Prepare & Present at
PVAAS to School Board Members and/or Community
Show: DVD for School Boards
PVAAS for School Board Members
by School Board Members
Tom Gentzel, PSBA 2 Superintendents
School Board Members 12 minutes
Email [email protected] for a copy of the DVD
Suggested Resources & Documents for Board Members
Copy of PPT Presentation
Guides to PVAAS Reporting 2011• Guide to PVAAS Reporting for LEAS 2011• Guide to PVAAS Public Reporting 2011
Suggested Resources & Documents for Board Members
Copies of Key District Reports• District Value-Added Summary Reports• District Value-Added Reports• District Projection Summary Reports
District Scatter Plots• Web-based Scatter Plots (demonstrate on
public reporting site) Grade level or Across Grades
Example
PDE PVAAS Webpage
Help Menus, PPT Presentations with Trainer Notes, Resource Materials, Video Clips, & District Case Studies
SAS PVAAS Website
New & Archived Webinars, Podcasts, Guide to PVAAS Reporting for LEAs, Guide to PVAAS Public Reporting,
& Intermediate Unit Supports
https://pvaas.sas.com
Key Concepts for EVERY
PVAAS Session
Key Concepts: ANY PVAAS Session
Achievement Progress Measures students’
performance at a single point in time
Compares student performance to a standard
Critical to students’ post-secondary opportunities
Highly correlated with demographic factors, such as socioeconomic status
Measures students’ progress across time/years
Compares student performance to his/her own prior performance
Critical to ensuring students’ future academic success
Not correlated with demographic factors
Key Concept:ANY PVAAS Session
PVAAS reporting reflects the effectiveness of your district’s or school’s Standards-Aligned System
PVAAS reporting reflects the district’s or school’s system regarding curriculum, assessment, and instruction
Key Concepts: ANY PVAAS Session
There is NO relationship between demographic variables such as socioeconomic status and growth.
• There are high achieving schools making high growth;• There are high achieving schools making low growth;• There are low achieving schools making high growth; • There are low achieving schools making low growth.
Growth reporting reflects what WE do with students in terms of academic growth in schools/districts.
-- VAAS can remove the effects of factors not under the control of the school. (McCaffrey, Lockwood, Koretz & Hamilton, 2003; Ross, Wang, Sanders, Wright & Stringfield, 1999a; Wright, Horn & Sanders, 1997).
Key Concept:ANY PVAAS Session
PVAAS District and School Value-Added Reports do NOT provide a measure of teacher effectiveness!
Often people ask about the meaning of these reports, especially in situations where there is only one classroom teacher in a grade level. This is NOT a report on the effectiveness of the classroom teacher, even in that situation!
Teacher effectiveness reporting would be done using a different statistical analyses than the one used for reporting on district or school effectiveness. Additional data would be collected as well.
DECISION POINTS
Who? - What? -When? - How?
Decision Point: Audience
• What are the backgrounds of school board members?
• Do the board members have experience with growth data/PVAAS?
• Do the board members have any background knowledge/foundation in Statistics?
Decision Point: Intended Goal for Session (s)
PVAAS Overview?
Review of Actual District/School Data?
Part of Planning Process?• Comprehensive Planning• School Improvement Planning
Decision Point: Focus of Data
District? School? Grade? Subgroups? Which subjects?
• Math, Reading, Science, Writing Remember, less is more!
Decision Point: Reports NeededDistrict-wide Focus
• Scatter Plots• District Value-Added
Think ahead if there is only one building per grade
• Value-Added Summary/Individual schools• District (Single Grade) Projection Summary
Reports• (Optional) Performance Diagnostic Summary
If your audience is interested in growth of students at different achievement levels
Examples of Reports
Talking About Your Results
Think about your message before sharing your results! What do the results
mean? Do you have a plan of
action regarding the areas needing improvement (Yellow and Red)?
What is your plan of action for attending to the Green, Light Blue, and Dark Blue areas?
Talking About Your Results
Think about your message before sharing your results!
What do the results mean? Do you have a plan of action regarding the areas
needing improvement (Yellow and Red)? What is your plan of action for attending to the
Green, Light Blue, and Dark Blue areas?
Decision Point: Session Facilitators(s)
Principal Team of Principals District Administrator(s) District Administrator(s) + Principal(s) Support from Intermediate Unit PVAAS
Contacts PVAAS Statewide Team for PDE
• Regional sessions for district administrators and School Board members
OTHER CONSIDERATIONS
PVAAS Projections are an Approved Way for Schools
to Make AYP(Growth Model)
PVAAS Public Reporting Site Value-Added (Growth) Reports at District and
School Levels• Math, Reading, Science, and Writing• Grades 4 through 8, and 9-11
School Search Capability• Allows users to find and view the achievement
and progress of groups of students in public schools across the Commonwealth of Pennsylvania.
• Can search for similar schools based on grade levels tested and various demographics.
Web-based scatter plots
Preparing for Parent Requests
Parents may ask about data or students.
What is available at the student level? Projections!• Percent likelihood of a student reaching Basic,
Proficient and/or Advanced levels on future PSSA
Some PA districts have shared PVAAS student projections with parents.
Plan in advance for how to respond such requests
Presenting PVAAS to an Individual School Board Power Point Template
with Trainer Notes
Pennsylvania Value-Added Assessment System (PVAAS):A Compass for Improvement
Presented at [District] School Board Meeting[Date]
DVD for School Boards
PVAAS for School Board Members
by School Board Members
Tom Gentzel, PSBA 2 Superintendents
School Board Members 12 minutes
TWO Types of PVAAS Information
Today
Looking Back/Evaluation…
Value-added Growth Reports For Groups of Students
Looking Forward/Planning…
PVAAS Projection Reports For Individual Students and Groups of Students
PVAAS IS: Analysis of existing assessment data
• Longitudinal PSSA scores in available content areas
Math, Reading, Science, Writing• Provides reliable estimates
A measure of a school’s effectiveness
Critical Relationship: Growth Data vs. Achievement
Data
PVAAS:
Is not another test
Critical concept• Is not a comparison of
single test scores
Review of Literature
Is there a relationship between… Demographics vs. Achievement?
YES
Demographics vs. Growth?NO
Demographics are inherently controlled for in PVAAS due to the use of all longitudinal data on students!
High Achievement
Average Achievement
Low Achievement
Current Data Collection Approaches
Achievement is based on each student’s ending point.
PVAAS Adds Another Variable To Achievement…GROWTH!
Below Growth
Standard
At or AboveGrowth
Standard
Growth is based on each student’s starting point!!
Viewing Achievement and Progress Together
PVAAS Web-based Scatter Plots
Scatter Plot Quadrants and Their Meanings
Quad 2 - SlippingHigher
AchievementLower Growth
Quad 1 - Excelling
Higher Achievement
Higher Growth
Quad 4 - Improving
Lower Achievement Higher Growth
Quad 3 - Underperforming
Lower AchievementLower Growth
Example: Regional Scatter Plot
Example: Regional Scatter Plot: Same Schools, Different Subject
Web-based Scatter Plotson PVAAS Public Site
Example
District Scatter Plots
Math and Reading
District Scatter Plot – Grade 4Math and Reading
District Scatter Plot – Grade 5Math and Reading
District Scatter Plot – Grade 6Math and Reading
District Scatter Plot – Grade 7Math and Reading
District Scatter Plot – Grade 8Math and Reading
District Scatter Plot – Grade 11Math and Reading
Overview of PVAAS Reporting
PVAAS Reporting District Level Reporting School Level Reporting Grade Level Reporting
• Reading (Grades 4-8, 11)• Math (Grades 4-8, 11)• Science (Grades 4, 8, & 11)• Writing (Grades 5, 8, & 11)
Subgroup Reporting (ex. IEP, ED, ELL) Student Level Reporting
• Projections to Future PSSA Performance• Reading, Math, Science, Writing• Grades vary based on subject area
Have all of our groups of students met the standard for PA Academic Growth?
Grades 4-8?Grades 9-11?
Value-Added Growth Reports
PVAAS Value-AddedGrowth Descriptors
DB
Significant evidence that the district/school exceeded the standard for PA Academic Growth
LBModerate evidence that the district/school exceeded the standard for PA Academic Growth
GEvidence that the district/school met the standard for PA Academic Growth
YModerate evidence that the district/school did not meet the standard for PA Academic Growth
RSignificant evidence that the district/school did not meet the standard for PA Academic Growth
District/School Value-added ReportGrades 4 through 8: Math & Reading
Question for Educators:Are all grade levels meeting the standard for PA Academic Growth?
NOT on Public Site
District/School Value-added ReportGrades 9-11 Math & Reading; Science; & Writing
Question for Educators:Are all grade levels meeting the standard for PA Academic Growth?
NOT on Public Site
District Value-added Summary Report
Question for Educators: Which schools at which grade levels are exceeding the standard for PA Academic Growth?
NOT on Public Site
How well are we supporting our high achieving students or low achieving
students?
Performance Diagnostic Summary Reports
Performance Diagnostic Growth Descriptors,
Grade 4-8, Math/Reading
Exceeded the standardfor PA Academic Growth
Met the standardfor PA Academic Growth
Did not meet the standard for PA Academic Growth
Performance Diagnostic Summary Report
Questions for Educators: How effective are schools across the district in making academic progress with groups of students at different predicted performance levels? Are there difference in growth patterns across schools?
NOT on Public Site
What is the likelihood that students will be reach a Proficient level or
higher on a future PSSA? What is their likelihood of reaching an
Advanced level?
PVAAS Projection Summary ReportsPVAAS Student Projection Reports
Student Projection Methodology
Is this the same methodology as the one used to estimate growth?• No, the projection methodology is a separate modeling
process that focuses on individual students.
What data are used in this methodology?• ALL available longitudinal data in both Reading and Math
are used in projection calculations for all four subjects.
What students have projections? In what subjects?• Projections are available for all students who have a
minimum of 2 years of historical data. The next slide details the grade levels and subjects in which projections are reported.
PVAAS Fall 2011 Reporting: Projections to Basic, Proficient, Advanced
Math and Reading: 3 4 or 5 4 5 or 6 5 6 or 7 6 7 or 8 7 8 8 11
Writing: 3 5 4 5 5 8 6 8 7 8 8 11
Science: 3 4 4 NA 5 8 6 8 7 8 8 11
Grade PSSA
Last Taken
Grade PSSA
Last Taken
Grade PSSA
Last Taken
NOT on Public Site
District (Single Grade)Projection Summary Report
PVAAS Student Projection Report
NOT on Public Site
PVAAS Help Menus
Questions:
PVAAS Materials or Statewide Implementation
717-606-1911
PVAAS Report Web Site
https://pvaas.sas.com
www.pde.state.pa.us
333 Market Street Harrisburg, PA 17126
