MN CTEDDIData Driven Improvement

MN CTE Professional Development2011-12

What is CTEDDI?

• A professional development model for CTE educators.

Why CTEDDI?

• Goal: Increase knowledge and skills for faculty & teachers in the use and interpretation of data.

• Intended Outcome: Teachers & Faculty make informed instructional improvements that enhance student success.

Instructional Improvement Cycle

5 Steps

Collect Data

Analyze Data

Verify & Triangulate

Design Action Plans

To improve Learning & Instruction

Implement Plan &Review Outcomes Ongoing

MN CTEDDI Partners

• Minnesota State Colleges and Universities

• Minnesota Department of Education• National Research Center for Career

Technical Education• National Occupational Competency

Testing Institute (NOCTI)

MN CTEDDI Project

• Part of on-going process for implementing and sustaining the use of technical skill assessments (TSA)

• Strive to influence continued improvement and systemic change in Minnesota’s secondary and postsecondary CTE programs.

MN Team Leaders

• Debra Hsu, Lead Facilitator• Michael Mitchell, Facilitator• Denise Roseland, Assessment Coordinator

TSA Implementation process

• Year 1: Identify core common competencies & state approved assessments by pathway

• Year 2: Implement TSA’s in approved programs of study begin MN CTEDDI Project

• Year 3: Implement TSA’s in pathways and continue MN CTEDDI

• Year 4 & Year 5: Continue implementation of TSA’s in pathways. Review, Revise & Assess CTE programs using data driven improvement model.

Phase I Project Timeline

• October/Nov - Consortium Leaders Training Face-to-Face at Consortium Sites

• Oct - Consortia leaders identify pathways and prepare teachers/faculty

• Oct/Nov- Teachers/Faculty administer technical skills assessment tests

• November 29, 2011 Data Driven Improvement Training Seminar for Leaders,

Teachers/Faculty – MN CTE & NRCCTE North Metro Location TBD• December-April - Monthly Webinars with Leaders, Teachers/

Faculty – • May - Final Report of MN CTEDDI Project with NRCCTE

State Team Responsibilities

• Provide skill-building training and technical assistance to local leaders and participants

• Purchase NOCTI pre-tests for participating consortia

• Foster virtual and face-to-face collaboration that encourages on-going professional learning for participants

• Collect and report participant data for national research project

Local Responsibilities• Participate in training and on-

going professional development• Support and facilitate processes

for teachers and faculty to gain access to necessary data other than assessment results

• Implement TSAs in pathways/classrooms selected

• Develop and implement consortium-wide learning improvement plan based on CTEDDI processes

• Participate in national research project data collection

Team Leader To Do List

• Initial 3 hour organization meeting (that’s today!)• Identify Faculty/Teachers & Pathways• Become a NOCTI client and order NOCTI pretests or

select and order other appropriate technical skill assessments

• Support teachers/faculty as they administer tests to students & work to gain access to other data

• Complete & collect Pre-workshop Information Form• Join Online Learning Community

Team Leader To Do List

• Attend November 29 Large Group Workshop provided by Pat Kelly, NRCCTE; John Foster, NOCTI & MN CTEDDI Leaders

• Complete & collect Post-workshop Feedback form

• Attend monthly webinars Dec-April• Complete & collect Mid-Project Feedback Form• Complete & collect Final Feedback Form

Faculty/Teacher To Do List

• Administer tests to students• Gather other relevant data• Complete Pre-workshop Information Form• Attend November 29 Large Group Workshop

provided by Pat Kelly, NRCCTE; John Foster, NOCTI & MN CTEDDI Leaders

• Complete Post-workshop Feedback form• Begin using CTEDDI skills in classroom

Faculty/Teacher To Do List

• Join Online Learning Community• Attend monthly webinars Dec-April• Complete Mid-Project Feedback Form• Complete Final Feedback Form

MORE ABOUT THE ROLE OF DATA…

• School/campus • Program• Teacher

Understanding Your Data

• Class• Student • Standards

• National• State • District/System

What are the sources of data currently available to

teachers/faculty in this consortium?

Let’s list some… – – – – – –

• Achievement gaps• Differences between subgroups• Trends • Progress

What the Data Reveal

Given the available data, what kinds of improvements might teachers/faculty

be able to make?

Given the available data, what kinds of data are missing that would allow them

to make more powerful changes?

ARE NOCTI TESTS REQUIRED?

Carpentry Pretest Areas Pretest % CorrectStudent 1

Posttest %CorrectStudent 1

Safety 54.6 --

Tools and Accessories 30.0 --Blueprint Reading and Estimation 41.1 --

Foundations, Forms, and Concrete 36.4 --Rough Framing 53.8 --

Exterior Finish 36.4 --

Interior Systems Installation 40.0 --Interior Finish 27.3 --Carpentry Related Mathematics 35.9 --

Total 40.0

Using Data from Similar Tests

National Nurse Aide Assessment• Activities of Daily Living• Basic Nursing Skills• Restorative Skills• Emotional and Mental Health• Spiritual and Cultural Needs• Communication Needs• Client Rights• Legal and Ethical Behavior• Member of HealthCare Team

NOCTI Nursing Assisting Test• Basic Nursing Assisting Skills• Infection Control• Communication Skills• Personal Care Skills• Mental Health and Social Service

Needs• Client’s Rights• Basic Restorative Devices• Employability Skills

Carpentry Pretest Areas Pretest % CorrectStudent 1

Posttest %CorrectStudent 1

Safety 54.6 --

Tools and Accessories 30.0 --Blueprint Reading and Estimation 41.1 --

Foundations, Forms, and Concrete 36.4 --Rough Framing 53.8 --

Exterior Finish 36.4 --

Interior Systems Installation 40.0 --Interior Finish 27.3 --Carpentry Related Mathematics 35.9 --

Total 40.0

Using Data from Similar Tests

• Both tests align to a national standard (e.g., NOCTI; or industry standard, e.g., NIMS, National Nurse Aide Assessment Program)

• Reports include subpart scores for major categories (competencies) within the occupation

• The blueprints have several content areas in common

ANY ASSESSMENT USED FOR THIS PROJECT MUST PROVIDE A TEACHER

OR FACULTY MEMBER WITH COMPETENCY-BASED RESULTS FOR

EACH STUDENT

Example from a Carpentry Test

Safety

Tools and Accessories

Blueprint Reading and Estimation

Foundations, Forms and Concrete

Rough Framing

Exterior Finish

Interior Systems Installation

Interior Finish

Carpentry Related Mathematics

Carpentry Pre-test Areas Pre-test % CorrectStudent 1

Post-test % CorrectStudent 1

Safety 54.6 --Tools and Accessories 30.0 --

Blueprint Reading and Estimation 41.1 --

Foundations, Forms, and Concrete 36.4 --

Rough Framing 53.8 --

Exterior Finish 36.4 --

Interior Systems Installation 40.0 --

Interior Finish 27.3 --

Carpentry Related Mathematics 35.9 --

Total 40.0

Individual Carpentry Pre-Test

Carpentry Pre-test Areas

Student 1 % Correct

Student 2 % Correct

Student 3 % Correct

Student 4 % Correct

Student 5 %

Correct

Student 6 %

Correct

Class Average % Correct

Safety 54.6 45.5 36.4 27.3 27.3 54.5 40.9

Tools and Accessories

30.0 20.0 10.0 20.0 10.0 30.0 20.0

Blueprint Reading and Estimation

41.1 17.6 23.5 23.5 29.4 35.3 28.4

Foundations, Forms, and Concrete

36.4 54.5 45.5 45.5 36.4 54.5 45.5

Rough Framing 53.8 38.5 30.8 38.5 30.8 53.8 41.0

Exterior Finish 36.4 36.4 27.3 36.4 36.4 45.5 36.4

Interior Systems Installation

40.0 60.0 40.0 60.0 60.0 80.0 56.7

Interior Finish 27.3 9.1 18.2 27.3 9.1 27.3 19.7

Carpentry Related Mathematics

35.9 45.5 27.3 36.4 36.4 54.5 39.4

Total 40.0 34.0 28.0 33.0 29.0 46.0 35.0

Carpentry Class Pre-Test

So does it become “teaching to the test”?

Given that technical skill assessments are aligned with national standards and/or employer-expected competencies, an assessment is ONE WAY to provide assurance that program instruction is preparing students for workplace demands.

When an assessment is combined with other relevant data that examine multiple facets of student performance, confidence that students are prepared improves dramatically.

Alignment with other Improvement efforts

• CTEDDI is not designed to be another stand-alone effort to improve instruction

• It has been designed to fit within other improvement efforts or strategic efforts (i.e. school reform initiatives, HLC accreditation, etc.)

MORE ABOUT THE ONLINE COMMUNITY

Main Screen