assessment system overview center for education overview for the ncate boe team april 18-22, 2009
TRANSCRIPT
Assessment System Assessment System OverviewOverview
Center for EducationOverview for the NCATE BOE Team
April 18-22, 2009
Assessment concerns
Does the assessment system provide information about
• Aggregated evidence about candidate knowledge, skills and dispositions?
• Disaggregated evidence about areas of strength and weakness in individual programs?
• Formative,on-going feedback to candidates, faculty, administrators and PK-12 partners?
• Summative feedback about program outcomes?• Benchmarked information about candidate proficiency
and progress linked to transition points?
Characteristics of the current assessment system
Assessments are aligned with• the conceptual framework • professional standards of PDE • or specialized professional associations
Assessments are tied to transition points.
The content of instruction is aligned with assessments.
Multiple methods and multiple evaluators are used.
Challenges in AY 2006
• Integrating information that we already had in order to have a “system”
• Refining data collection• Acquiring new skills (rethinking rubrics, achieving
alignment)• Creating new assessments for national program
recognition• Implementing TaskStream at all levels• Creating a collective assessment culture by through
collaboration and cross-disciplinary participation• Understanding that assessment is everyone’s job and
everyone has to be involved
What are the broad elements of the
assessment system?
• Assessment tools and strategies used to evaluate candidate proficiencies and faculty performance
• Evaluators, including faculty, administrators, school partners, and candidates, as well as other members of the professional educational community
• The proficiencies that are evaluated• Procedures and time-tables for collecting evidence
at each level of the assessment system
Organization of the assessment system
• See next slide
Databases
• ACCESS Database (2003 - 2006, transiting to a revised database in 2007)
• ACCESS Database (2007- present)• PRAXIS Database• Enrollment/Completion Database• Databooks (collections of required course
assessments for key courses in all programs)• TaskStream DRF’s• Mobile clinical evaluations using PDA’s & ACCESS• Miscellaneous databases needed to answer specific
questions
ACCESS Databasesrelevant evidence linked to transition
points2003 ACCESS database
(candidates for initial teaching licenses only)
• Admission to Widener• Admission to teacher
certification• PPST scores• Writing samples (yearly)• Academic progress• Student teacher
placements• PRAXIS scores• FBI/Criminal history
compliance
2007 ACCESS database (all candidates including candidates for graduate level certificates and degrees)
• Admission to Widener• Admission to certification• PPST scores • Writing samples • Academic progress• Student teaching/clinical
practice placements• Proficiency in student
teaching/clinical practice
• PRAXIS scores• FBI/Criminal history
compliance (as relevant)
PRAXIS database
• Pass rates by programs (Title II - HEA reports)• Total scaled scores disaggregated by program and
by level– Initial teaching license baccalaureate level– Initial teaching license post baccalaureate level– Specialist certificates– Advanced certificates for other school personnel
(principals and supervisors)• Category scores also disaggregated by program
and program level
Enrollment/Completer Database
• Headcounts of candidates • Undergraduate degrees and certificates• 5-year program candidates• Master degree candidates• Initial teaching, specialist and other school
personnel in graduate level certification programs
• Doctoral degree programs• School Psychology candidates
Databooks: Evidence and indicators
• Assessment narratives• Directions to students• Rubrics and scoring guides for every student
This data is compiled for each course by the NCATE office on EXCEL spreadsheets.
Data is then reported at semester assessment meetings to all unit faculty as a comprehensive set of data that indicates the percentages of candidates who scored each level of proficiency in each key course.
TaskStream: Evidence and indicators
• Candidate work samples• Rubrics aligned with standards or the unit
conceptual framework• Evidence of candidate proficiency• Evaluator feedbackThis evidence is assessed on a continuing basis by
students and faculty. Candidate performance is aligned with professional standards, PDE standards and the unit conceptual framework, according to relation to proficiency levels. The evidence can be aggregated to study program outcomes.
Miscellaneous Databases
• These are individual collections of data that address specific questions.
Who are the evaluators?
• Administrators• Advisory Boards• Cooperating faculty in PK-12 schools• Faculty committees• Center for Education Faculty • Arts and Sciences Faculty• Faculty who supervise candidates in student
teaching, practica and internships• Candidates
Administrators assess
• Faculty proficiency in teaching, scholarship and service to their professions
• Graduate/alumni satisfaction• Employer perceptions
Advisory boards assess
• Knowledge of content• Knowledge of pedagogy• Pedagogical skill• Dispositions• Employer perceptions
Cooperating faculty assess
• Knowledge of content• Knowledge of pedagogy• Pedagogical skill• Dispositions
Faculty committees assess
• Knowledge of content• Knowledge of pedagogy• Pedagogical skill• Dispositions• Readiness for next steps in licensure or degrees• Candidate work samples• Student learning• Faculty proficiency in teaching, scholarship and
service to their professions
Arts and Sciences faculty assess
• Knowledge of content
Center for Education faculty
assess • Knowledge of content• Knowledge of pedagogy• Pedagogical skill• Dispositions• Readiness for next steps in licensure or degrees• Candidate work samples• Student learning• Faculty proficiency in teaching, scholarship and
service to their professions• Graduate/alumni satisfaction• Employer perceptions
Faculty who supervise candidates assess
• Knowledge of content• Knowledge of pedagogy• Pedagogical skill• Dispositions• Readiness for next steps in licensure or degrees• Candidate work samples
Candidates assess• Student learning• Faculty proficiency in teaching• Graduate/alumni satisfaction
Has the unit benefited from continuous
assessment? • Faculty as a unit are aware of candidate proficiencies and program content.
• Assessment is now a unit-wide enterprise because all faculty attend all assessment workshops and share assessment information.
• Skills, assessment language, and enthusiasm for assessment has increased.
• Time has been created to have cross-disciplinary discussions for the first time.
• Faculty in Higher Education and Human Sexuality Education are using the NCATE model to develop standards and assessment for their programs.
What are the next steps?
• Embedding large parts of the system within the Office of Certification
• Sustaining gains in collaboration throughout the unit and the PEU
• Refining our tools• Modifying assessments in line with recent
changes in PDE accountability mandates at all levels of certification
• Continuing the integration of the assessment of non-licensure programs
• Adapting our methods to the proposed Widener assessment database