pvaas school consultation fall 2011
DESCRIPTION
PVAAS School Consultation Fall 2011. PVAAS Statewide Core Team [email protected]. Consultation Guide Pg. 1. Purpose of Consultation. Goal : To model a process for the inspection, analysis and interpretation of your 2011 data Focus : PAAYP, eMetric, and PVAAS Data for Reading and Math - PowerPoint PPT PresentationTRANSCRIPT
PVAAS 2011 Consultation Guide
2
Purpose of Consultation• Goal: To model a process for the inspection,
analysis and interpretation of your 2011 data
• Focus: PAAYP, eMetric, and PVAAS Data for Reading and Math
• This process is designed to give a school the “30,000 foot view” of the status of achievement and growth– Whole school data– Grade level data– Subgroup data
Consultation Guide Pg. 1
10/9/2011
PVAAS 2011 Consultation Guide
3
Two key questions to keep in mind as you examine your data!
Based on the system’s prior year achievement goals/targets:
– Is the system (school/district) serving all students adequately?
– Is the system (school/district) serving all students equitably?
Purpose of Consultation
10/9/2011
PVAAS 2011 Consultation Guide
4
Agenda• The Power of Two: Web-Based Scatter Plots……15
min• PVAAS Reports………………………………………35
min– Value-Added Reports, Performance Diagnostic Reports,
School Projection Summary
• PA AYP Report………………………………………..5 min
• PSSA Data Interaction Reports, eMetric ………...20 min– Performance Level Summary, Reporting Categories
Summary, Matched Comparison Report• Summary and Next Steps………………………….15
min
10/9/2011
The Power of Two!
Considering Achievement and Growth TOGETHER!
C.G. Pg. 2
PVAAS 2011 Consultation Guide
Quad 2 - SlippingHigher
Achievement,Lower Growth
Scatter Plot Quadrants and Their Meanings
10/9/2011 6
Quad 1 - Excelling
Higher Achievement,Higher Growth
Quad 4 - Improving
Lower Achievement, Higher Growth
Quad 3 - Underperforming
Lower Achievement,Lower Growth
C.G. Pg. 2
PVAAS 2011 Consultation Guide
NEW! Web-Based Scatter Plots
7
C.G. Pg. 2
https://pvaas.sas.com
10/9/2011
PVAAS 2011 Consultation Guide
8
Questions For Later Analysis1. Based upon the position of the schools in
district, is the system serving all students adequately in the chosen subject area?
2. How are the schools in the district performing in comparison to all the schools in the state?
C.G. Pg. 2
10/9/2011
PVAAS 2011 Consultation Guide
9
Consultation Guide• Decide on focus for your team to make the most
of the time:– What subject you/others will focus on – Reading or Math?– What grade level(s) within the subject?– What subgroup within the grade and subject?
• Purpose is to model a process that can be used to determine the status of achievement & growth in your school.
• The next step is to begin to investigate WHY you are getting these results.
C.G. Pg. 2
10/9/2011
PVAAS 2011 Consultation Guide
10
PVAAS ReportsC.G. Pg. 7
https://pvaas.sas.com
10/9/2011
PVAAS 2011 Consultation Guide
11
TWO Types of PVAAS Information
Today
Looking Back/Evaluation…Value-added Growth Reports For
Groups of Students
Looking Forward/Planning…
PVAAS Projection Reports For Individual Students
and Groups of Students
C.G. Pg. 7
10/9/2011
PVAAS 2011 Consultation Guide
12
School Value-Added ReportsThese reports provide information on how effective a school was in supporting groups of students to at least meet the standard for PA Academic Growth.
C.G. Pg. 7
10/9/2011
PVAAS 2011 Consultation Guide
NEW! Legends and Color Codes
• Alignment of colors and interpretative legends across all Value-Added Reports (all subjects and grade levels) – New 5 color-coding scheme
• Same for Reporting 4-8 and 9-11!– All legends are consistent in color and language– Use of the data becomes easier as there is no
confusion about what the colors indicate• New 5 Color Codes
– Allow for more differentiation among schools exceeding the standard for PA Academic Growth
– Allow for more differentiation in Science and Writing reporting, as well as grades 9-11 Reading and Math reporting
1310/9/2011
PVAAS 2011 Consultation Guide
1410/9/2011
PVAAS 2011 Consultation Guide
NEW! PVAAS Value-addedGrowth Descriptors
1510/9/2011
PVAAS 2011 Consultation Guide
16
Meeting the standard for PA Academic Growth
Growth Measure= 0• Grades 4-8 - A group meets the standard
for PA Academic Growth when they maintain their relative achievement level from one year to the next in relation to the statewide distribution (2006 baseline).
• Grades 9-11 – A group meets the standard for PA Academic Growth when their actual achievement meets their predicted achievement- based on the average schooling experience.
10/9/2011
PVAAS 2011 Consultation Guide
17
Questions1. Are the students in your school making the
growth needed to meet the school’s/districts achievement goals/targets?
2. Describe the pattern of growth within each grade over the last three years.
3. What patterns do you see when examining the entering achievement level of students at each grade level over the past years (looking at achievement vertically for the grade)?
4. What do the Growth Measures tell you about the system’s success at meeting the needs of students at their achievement level?
C.G. Pg. 7
10/9/2011
PVAAS 2011 Consultation Guide
18
School Value-Added Reports
Navigate & investigate your Value-Added Reports and answer the questions in your guidebook.
C.G. Pg. 7
10/9/2011
PVAAS 2011 Consultation Guide
19
Performance Diagnostic Report
The Performance Diagnostic Report allows you to view growth patterns disaggregated by predicted PSSA performance levels (Below Basic, Basic, Proficient, and Advanced).
C.G. Pg. 8
10/9/2011
PVAAS 2011 Consultation Guide
20
Interpretation of the Performance Diagnostic
Report
Zero (0) Line
Growth
Met standard for PA Academic Growth
Green
Less Than standard for PA Academic Growth
Pink
Exceeded standard for PA Academic Growth
Blue
What the whiskers tell us…
C.G. Pg. 8
10/9/2011
PVAAS 2011 Consultation Guide
A
B
C
21
Patterns of Growth
C.G. Pg. 8
10/9/2011
PVAAS 2011 Consultation Guide
22
Questions1. How does the growth pattern for the current
year (blue bars) compare to the growth pattern for previous years (gold bars) in same grade and subject?
2. What patterns of growth do you see across grades in each subject area?
3. What do these patterns suggest about the system’s success at meeting the needs of students at varying achievement levels?
4. What might these patterns suggest about the system’s curriculum, instruction, materials & resources, or interventions?
C.G. Pg. 8
10/9/2011
PVAAS 2011 Consultation Guide
23
Performance Diagnostic Report
Navigate & investigate your Performance Diagnostic Reports for each grade level and complete the questions in your guidebook.
C.G. Pg. 8
10/9/2011
PVAAS 2011 Consultation Guide
24
Questions for a Subgroup1. What patterns of growth do you see for
the selected subgroup across grades in each subject area?
2. Does the pattern of growth for the selected subgroup mirror the pattern of growth for the grade as a whole? Make note of any concerns.
3. What do these patterns suggest about the system’s success at meeting the needs of students at varying achievement levels within each subgroup?
C.G. Pg. 9
10/9/2011
PVAAS 2011 Consultation Guide
25
Performance Diagnostic Report - Subgroups
Navigate & investigate your Performance Diagnostic Reports by subgroup for each grade level. Complete the table and answer the questions in your guidebook.
C.G. Pg. 9
10/9/2011
PVAAS 2011 Consultation Guide
26
TWO Types of PVAAS Information
Today
Looking Back/Evaluation…Value-added Growth Reports For
Groups of Students
Looking Forward/Planning…
PVAAS Projection Reports For Individual Students and Groups of Students
10/9/2011
PVAAS 2011 Consultation Guide
Projections to Proficient or Advanced
27
Grade PSSA Last Taken
3 4 or 54 5 or 65 6 or 76 7 or 87 88 11
Project to these Grades
10/9/2011
PVAAS 2011 Consultation Guide
28
How do I know that the projections
are accurate?• The projection is precise, and is created using a model that
has been reviewed and approved by four different peer review panels and the GAO (US Government Accountability Office).
• The most recent 2008 growth model proposal to USDOE includes information regarding the statistical model and projection reliability study.(
http://www.ed.gov/admins/lead/account/growthmodel/pa/index.html)
• Recent studies have confirmed that the PVAAS projections (even as far as 3 years into the future) are more reliable at looking at the future performance of a student than the most recent PSSA score.
10/9/2011
PVAAS 2011 Consultation Guide
29
School (Single Grade) Projection Summary Report
This report provides educators with the probability of students scoring at or above a Proficient level on a future PSSA.
C.G. Pg. 10
10/9/2011
PVAAS 2011 Consultation Guide
30
Questions1. What percentage of students fall in 70%-100%
probability range of reaching proficiency? What does this suggest about the degree to which the core program is meeting the needs of the majority of your students?
2. What percentage of students fall in the 40%-70% probability range of reaching proficiency? What does this suggest about the needs for supports or interventions for students that are at risk of not reaching proficiency?
3. What percentage of student fall in the 0- 40% probability range of reaching proficiency? What does this suggest about the need for intensive supports and interventions for students in this range?
C.G. Pg. 10
10/9/2011
PVAAS 2011 Consultation Guide
31
School (Single Grade) Projection Summary Report
Navigate & investigate your Single Grade Projection Summary Reports for each tested grade and answer the questions in your guidebook.
C.G. Pg. 10
10/9/2011
PVAAS 2011 Consultation Guide
32
Questions for a Subgroup1. In what probability range do most of the
students in this subgroup fall in each tested grade? Greater than 70%, Between 40% to 70%, or Less than 40%?
2. What do the data suggest about the degree to which your core program is meeting the needs of the selected subgroup of students?
3. What interventions, supports, and/or services have been provided for these students?
C.G. Pg. 11
10/9/2011
PVAAS 2011 Consultation Guide
33
School (Single Grade) Projection Summary Report - Subgroups
Navigate & investigate your Single Grade Projection Summary Reports by Subgroup for each tested grade and answer the questions in your guidebook.
C.G. Pg. 11
10/9/2011
PVAAS 2011 Consultation Guide
34
PA AYP
http://paayp.emetric.net/
C.G. Pg. 3
10/9/2011
PVAAS 2011 Consultation Guide
35
PA AYP Report• Public Site• Has a school met the AYP targets?
– Academic Performance in Reading & Math– Test Participation in Reading & Math– Attendance or Graduation
• Which student groups count for AYP purposes?– A total of 40 or more students in the tested
grades in the school
C.G. Pg. 3
10/9/2011
PVAAS 2011 Consultation Guide
36
Questions1. Did your school make AYP for performance
for the whole school and all measurable subgroups? (72% Reading, 67% Math)
2. Did your school make AYP for participation for the whole school and all measurable subgroups?
3. Did the school make AYP for Attendance or Graduation?
C.G. Pg. 3
10/9/2011
PVAAS 2011 Consultation Guide
37
PAAYP Data Table
Navigate & investigate your PAAYP Data Table and answer the questions in your guidebook.
2011 AYP TargetsReading – 72%Math – 67%
C.G. Pg. 3
10/9/2011
PVAAS 2011 Consultation Guide
38
PSSA Data Interaction – eMetric
https://solutions1.emetric.net/pssa/
C.G. Pg. 4
10/9/2011
PVAAS 2011 Consultation Guide
39
eMetric• Password-protected site• Grade level data
– Multiple years• Student groups within grade levels
– Ethnicity– Gender– ED, IEP, LEP– Title I
C.G. Pg. 4
10/9/2011
PVAAS 2011 Consultation Guide
40
eMetricC.G. Pg. 4
10/9/2011
PVAAS 2011 Consultation Guide
41
Graphical Summary: Performance Levels
• Provides summary reports in the form of graphs, including bar graphs, pie charts, histograms, or line graphs.
• Allows for selection of subject areas, demographic variables, multiple grades, and multiple years.
C.G. Pg. 4
10/9/2011
PVAAS 2011 Consultation Guide
42
Questions1. Did every grade meet the AYP performance
targets for reading and math? (72% Reading, 67% Math)
2. Did every relevant subgroup within each grade meet the AYP performance targets for reading and math? (72% Reading, 67% Math)
3. Does the pattern of performance for the selected subgroup mirror the pattern of performance for the grade as a whole?
C.G. Pg. 4
10/9/2011
PVAAS 2011 Consultation Guide
43
Navigate & investigate your Performance Levels Summary for each grade level and subgroup and answer the questions in the guidebook.
Graphical Summary: Performance Levels
C.G. Pg. 4
10/9/2011
PVAAS 2011 Consultation Guide
44
Graphical Summary: Reporting Categories
• Provides a bar graphs which display the percentage of score points earned for each reporting category.
• Allows for selection of different subject areas and demographic variables at each grade level.
C.G. Pg. 5
10/9/2011
PVAAS 2011 Consultation Guide
45
Questions1. At each grade level, did all students
perform satisfactorily (at least 60% correct) in all reporting categories in the targeted subject area?
2. Does the pattern of performance across reporting categories for the selected subgroups mirror the pattern of performance across reporting categories for the grade as a whole?
C.G. Pg. 5
10/9/2011
PVAAS 2011 Consultation Guide
46
C.G. Pg. 5
Navigate & investigate your Reporting Category Summary for each grade level and subgroup and answer the questions in the guidebook.
Graphical Summary: Reporting Categories
10/9/2011
PVAAS 2011 Consultation Guide
47
Matched Comparison
• Compares PSSA performance levels for two years to determine if students have moved to higher PSSA performance levels in the subject area this past year.
C.G. Pg. 6
10/9/2011
PVAAS 2011 Consultation Guide
48
Questions For Later Analysis
1. In each grade level, have more students moved to higher PSSA performance levels, remained in the same performance levels, or moved to lower performance levels?
2. How do these results impact your school’s achievement goals?
C.G. Pg. 6
10/9/2011
PVAAS 2011 Consultation Guide
49
Matched Comparison
Navigate & investigate your Matched Comparison reports for each grade levels and answer the questions in the guidebook.
C.G. Pg. 6
10/9/2011
PVAAS 2011 Consultation Guide
50
Summary of Essential Findings
• Based on your system’s achievement goals/target, how would you answer the two main questions?– Is the system (school/district) serving all
students adequately?– Is the system (school/district) serving all
students equitably?• List three findings from today’s session
under the subject area you investigated to share with the group.
C.G. Pg. 12
10/9/2011
PVAAS 2011 Consultation Guide
51
Next Steps• Share the findings from this session with
staff Or, even better….
• Replicate the process with staff– Whole school – half focusing on math; half
focusing on reading– Grade level teams – focusing on math and
reading for their grade only– Departmental – focusing on subject area
C.G. Pg. 13-14
10/9/2011
PVAAS 2011 Consultation Guide
52
What about other data?• What other data sources are
available for staff? – Benchmark Assessments (4-Sight, MAPS,
Acuity)– CDT – Classroom Diagnostic Tool– DIBELS, AIMSweb, G-MADE, GRADE, Study
Island, STAR– District/School Common Assessments
• Who has access to these data?• How often is it shared?
C.G. Pg. 13
10/9/2011
PVAAS 2011 Consultation Guide
53
Why are we getting these results?
C.G. Pg. 13-14
10/9/2011
PVAAS 2011 Consultation Guide
54
How can we support you in the future?
• Discuss with your team what types of support you might find useful from your IU PVAAS contact(s), as well as any other resources or materials needed from the PDE PVAAS Statewide Team.
• Detach and return page 19 of the Consultation Guide to your session facilitator.
C.G. Pg. 19
10/9/2011
PVAAS 2011 Consultation Guide
55
Questions:
PVAAS Materials or Statewide Implementation
717-606-1911
PVAAS Report Web Sitehttps://pvaas.sas.com
10/9/2011
www.pde.state.pa.us
56PVAAS 2011 Consultation Guide10/9/2011