paiu cc meeting pvaas update november 2014 kristen lewald, ed.d. statewide pvaas director for pde...
TRANSCRIPT
PAIU CC MeetingPVAAS UpdateNovember 2014
Kristen Lewald, Ed.D.
Statewide PVAAS Director for PDE
Lancaster-Lebanon IU13
Topics
Guide for Reflection on Teacher Specific Reporting
Demo Access Accounts Common Questions from Admins and
Teachers Feedback from Teachers Feedback from 499
Superintendents/Designee Help Menu System PVAAS Reporting/Website Updates
NEW: Guide for ReflectionPVAAS Teacher Specific Reporting
Guide for Reflection:Process for Review and
Planning
Steps to Review Reports and
Develop Plan of Action
Guide for Review and Planning
Two Sets of Guiding Questions:
Organized Based on Domains
1. Planning and Preparation
2. Classroom Environment
3. Instruction
Organized Based on CIAO!
1. Curriculum
2. Instruction
3. Assessment
4. Organization
Demo Account AccessPVAAS Teacher Specific Reporting
Demo Account Access
Share with LEAs
Common Questions from Admins and Teachers
PVAAS Teacher Specific Reporting
Why No Report for a Teacher Who Rostered?
PVAAS Teacher Specific Reporting
Common Questions
Students Included in Reporting
To actually be included in PVAAS Teacher Specific reporting, a student MUST:
• Have a PSSA or Keystone score from the most recent year
• NOT be a foreign exchange student• NOT be a first-year ELL student• NOT be Proficient or Advanced on a PRIOR
Keystone exam• NOT be claimed at less than 10% instructional
responsibility• NOT have tested with the PASA (alternate
assessment)
Minimum N Counts
Must have at least 11 students
Must have an “Active N Count” of at least 6 students/6.0 FTE Students
Why No Individual Student Growth Measure?
PVAAS Teacher Specific Reporting
Common Questions
Why No Individual Student Growth Measure?
Student History Report
Growth is about a GROUP of students NOT an individual student.
Error would be too large for an individual student to know if they made growth or not.
This report should NOT be used or interpreted as growth for an
individual student!
Why No Predicted Score NOW?PVAAS Teacher Specific Reporting
Common Questions
Why No Predicted Score Available NOW for SY14-15?
A student’s predicted score is based on the student’s own testing history and on the average performance of students in the same cohort statewide who have a similar testing history.
In other words, for each assessment we analyze with the predictive model, we look at the past testing histories of students across the state who took that assessment in the most recent year.
Because the predicted score is based on the performance of other students statewide who took the assessment in the same year, it’s not possible to generate the predicted score prior to the assessment.
What would the score tell you that a PVAAS projection/% of likelihood does not?
Why Different?School Value-Added vs.
Teacher Value-Added Reporting
PVAAS Teacher Specific Reporting
Common Questions
Why Different?School Value-Added vs. Teacher
Value-Added Reporting
2 Primary Reasons:
PVAAS School Reporting uses full academic year/Oct 1 as a requirement for including students
PVAAS Teacher Value-added reporting uses % Student + Teacher Enrollment and % Shared Instruction to determine the weighting of each student on each teacher’s PVAAS teacher Specific Reporting
Why Different?Composite v. Teacher Value-
AddedPVAAS Teacher Specific Reporting
Common Questions
What is the Composite?
A combined growth measure across a teacher’s PVAAS reported subjects/grades/courses within SY13-14.
For SY13-14, we only have one year of reporting so the Composite Score is a combined growth measure across a teacher’s PVAAS reported subjects/grades/courses for SY13-14 only.
In future years, the Composite Score will represent a combined growth measure across a teacher’s PVAAS reported subjects/grade/courses across years, up to and including 3 consecutive school years. 1 year Composite
2 Year Composite
3 Year Composite/3 Year Rolling Average1 Pager in Develop-
ment
Teacher Value Added Summary
The composite is a combined measure of all tested subjects, grades, and Keystone content areas in which a teacher received a Value Added report.
Composite: How Calculated
To calculate the composite, a simple average is taken of all of the teacher's individual index values for up to three years.
Then the average is multiplied by the square root of the number of individual index values that went into the average.
This step is a necessary step that accounts for the fact that more data was used to generate the average than was used to generate each individual index, which affects the standard error for the composite.
Why Different?Composite v. Teacher Value-Added
The growth color indicators are categories or ranges of indicators of growth. We cannot “average” categories. In other words, green + red ≠ yellow; likewise green + dark blue ≠ light blue.
For PVAAS, the assumption is that the achievement level of the teacher's group of students is maintained (green) UNLESS there is enough evidence in the assessment data to say otherwise.
Why Different?Composite v. Teacher Value-Added
It is all about the amount of evidence in the student assessment results – the assumption is that the achievement level of the teacher's group of students is maintained (green) UNLESS there is enough evidence in the assessment data to say otherwise.
The more data available, the more evidence we have to see if the group of students exceeded the growth standard OR not.
There is more evidence when all data was combined for a composite score. With data from multiple subjects (or multiple years when that’s available) included in the Composite, there’s more evidence/more data.
The composite is not really an “average,” but rather an accumulation of the evidence/data towards meeting, exceeding, or falling short of the standard for PA Academic Growth.
Why Different?Teacher Value-Added v.
Diagnostic ReportingPVAAS Teacher Specific Reporting
Common Questions
Diagnostic Reports do NOT factor in the percent of Instructional Responsibility
Students are weighted equally
Reflect the growth of a group of students who may have had more than one teacher with instructional responsibility in that specific subject/grade or course
Why Different?Teacher Value-Added vs. Diagnostic
Reporting
Methodology QuestionsPVAAS Teacher Specific Reporting
Common Questions
More Info About Methodology
PVAAS Methodologies, Measuring Growth & Projecting Performance – PVAAS login page, under Professional Development
Virtual Learning Modules – eLearning Link
Value Added Reports – Math/Reading
Value Added Reports – Science/Keystones
Teacher Specific Reporting
EVAAS Statistical Models in PA – PDE PVAAS site, Methodology Page
Full Day TOT Sessions for AdminsPVAAS Teacher Specific Reporting
Participation
450 LEAsOver 1700 Participants
Misunderstanding
PVAAS 3 Year Rolling Average3 Consecutive Years of PVAAS
Teacher ReportingAny state assessed
subject/grade/courseDoes not need to be in the same
subject/grade/course
Half-Day Teacher Sessions at IUsPVAAS Teacher Specific Reporting
Half-Day Sessions with Teachers
– We Need Your Support! Before session
Setup of session – registration, handouts, technology
Make sure all participants can get online
Make sure all participants log onto PVAAS site - check username and password and PPID before training starts
Download session PPT
Welcome Participants to your IU/Orient to Facility
During Session Rotate around room during online work
Gather questions if not comfortable answering
Engage with participants
Ideas for Follow-up IU services
After Session Gathering and Packing Up Materials
Copies of Session Feedback Forms
Email Final Registrant List
Feedback from TeachersPVAAS Teacher Specific Reporting
What will you tell your colleagues about PVAAS?
Emails to 499 Superintendents/Designees
PVAAS Teacher Specific Reporting
Asked 4 Questions
How is it going overall? Were most of the reports in sync with
what principals expected? Any/many surprises for the principals?Any questions or information needed
by you or your team?
“Best” Teacher
Qualities such as: students engaged in teacher’s classroom, using technology with students, positive feedback from students and/or parents, meets with students before/after school, goes to a student’s activities and sporting events, contributes to the school community, contributes to the district community, etc. Could a teacher do all of these things listed above, but students
may not be academically growing? is that possible? Why?
Could a teacher do all of these things listed above AND their students ARE academically growing? is that possible?
Why?
NEW: PVAAS Help Menu System
PVAAS Teacher Specific Reporting
New PVAAS Help Menu System System
Updates to PVAAS Keystone Reporting
PVAAS District/School Reporting
Updates to Both PVAAS Reporting Sites:
PVAAS Restricted/Password Protected Site: Thursday, Nov. 20th Updates to Keystone Reporting
PVAAS Public Site: Thursday, Dec. 4th
Updates: PVAAS Password Protected Site
• Keystones district/school value-added reports• Keystones district/school performance diagnostic reports• Keystones district/school quintile diagnostic reports• PVAAS SPP/AGI reports• PVAAS DPP/AGI reports• Changes to students included on drill down in above reports based on new business rules for Keystone reporting
