north carolina career technical education assessments: using assessment and data for continuous...
TRANSCRIPT
North CarolinaCareer Technical Education
Assessments: Using Assessment and Data for
Continuous Growth and Improvement
Tammy Howard, PhD
Director, Accountability Services
North Carolina Department of Public Instruction
General Overview of the System
• 215 Courses at the State Level
• 215+ End of Course Assessments
• Ability to Deliver Assessments Online or Paper/Pencil
• System to Capture and Validate Credentials and Certifications
2
3
Vision and Concept
• System Initially Designed in the Early 1990’s
• Originally Conceptualized as a Method to Reward Teachers for Continuous Improvement
• Recognized Nationally as Model for Formative and Summative Assessment
• Aligned to Carl D. Perkins Career and Technical Education Act Technical Attainment Performance Indicators
4
Assessment Models• Types of Test Item Banks
– Field Test
– Classroom (Teacher – Formative)
– Secure (Accountability – State End-of-Course)
• Types of End-of-Course Summative Assessments
– Developed
– Adopted
– Industry-Based
• Pre- and Post-Assessment Model
• EVAAS
5
Curricular Alignment
• All Courses in NCCTE are required to have:– “Blueprints” of Essential Instructional Standards
Aligned to:– Instructional Content
Aligned to:– Formative Assessment
Aligned to:– Summative Assessment
6
Validity, Reliability, and Analysis
• Validity Required for All Secure and Classroom Test Items
• Reliability Study Required for Each New Assessment
• Data Analysis Performed at State and District/Local Level
• State-level Item Analysis
• Scale Scores Determined After Reliability and Item Analysis
7
Growth Analysis• In the early 1980s, the EVAAS approach to measuring
growth was founded at the University of Tennessee Knoxville by Dr. William Sanders
• Revolutionized the way educators and policymakers viewed schooling effectiveness and the ability of students to make growth
• In 2006, EVAAS was implemented statewide as a school improvement resource.
• In 2012, EVAAS became a formal part of North Carolina’s teacher evaluation and accountability after recommendation by WestEd and UNC researchers.
8
How is growth measured?
• Predictive-based model– Based on students’ prior testing history
• all previous tests for which correlation has been established
• Students must have three prior test scores in any grade/subject
– What is the difference between students’ expected score and observed score?
• Use all available testing history for each student to minimize impact of measurement error
• Use standard errors to address uncertainty inherent in any growth model and protect against misclassification
9
Inclusion in Growth Model (EVAAS)Each CTE assessment must have sufficient:• Stretch in the scales
– To measure growth of both high- & low-achieving students
– No floor or ceiling effects• Reliability in the scales
– Preferably have at least 40-50 test questions per subject/grade/year
• Number of districts/schools/teachers administering test – Preferably 25 at each level
10
Inclusion in Growth Model (EVAAS)Each CTE assessment must have sufficient:• Correlation with curricular objectives• Predictive relationship
– Results of each CTE test must be related or correlated with results of other tests from different grades/subjects
– i.e. prior math or reading test scores and the CTE test score were not strongly related for Carpentry, but they were for Biomedical Technology
11
EVAAS Reporting• Dozens of reports for use in school improvement
– Reflective analytics, such as value-added and diagnostic reports for districts, teachers and schools
– Proactive analytics, such as student projections– Comparison reports, such as value-added summary and
scatterplots
• Roster verification for the student-teacher linkages in teacher value-added reports
• Help supports, such as video clips, online ticketing system, and help pages
• Available through a secure web application with customized access
12
Sampling of CTE Assessments Receiving EVAAS Reporting
• 50 CTE assessment met criteria for inclusion in growth model
Source: http://www.dpi.state.nc.us/docs/effectiveness-model/evaas/resources/vam-assessments.pdf
13
14
15
16
17
18
For More Information….
Contact:
Trey Michael - [email protected]
Tammy Howard - [email protected]
Tom Tomberlin - [email protected]