Summer Institutes

2013

ChangingTeacherPractice

ChangingStudentOutcomes

How to use Professional Development to Ensure Improved Student Outcomes

This presentation was adapted from Learning Forward, 2012

Our Outcomes • Define evaluation in relationship to professional

learning; • Examine the process of teacher change and its

impact on student learning; and • Acquire strategies, tools, and resources to

assist in evaluating professional learning - Building PD Capacity Toolkit (link)

Our Essential Questions• How can evaluating professional learning leverage

school, school system, and state improvement effort?

• How will I align professional learning objectives to measurable short, medium and long-term results for educators and students?

• How will I collaborate with others to construct a framework that outlines a detailed plan for evaluation?

• How do I incorporate evaluation into my work and normative practice?

Group Norms• We’re all in this boat together, so lets agree to…

• Listen as an Ally

• Value Differences

• Maintain Professionalism

• Participate Actively

• If we start sinking, and we need some help, we’ll be clear about whether we need a bucket or a boat.

The Standards for Professional Learning Professional learning that increases educator effectiveness and results for all students uses a variety of sources and types of student, educator, and system data to plan, assess, and evaluate professional learning.

-Standards for Professional Learning, 2011

http://www.ncpublicschools.org/docs/effectiveness-model/ncees/standards/prof-learn-quick-ref.pdf

Link to Student Results

DuFour Questions• What do we want students to learn?

• How will we know if they have learned it?

• What will we do if they have?

• What will we do if they haven’t?

Apply this to Professional Learning:

What does it mean in your district: Full implementation vs. Full Participation

• How do you know you have full implementation?

Full Implementation - Evaluation A systematic, purposeful process of studying, reviewing, and analyzing data gathered from multiple sources in order to make informed decisions about a program. – Killion, 2008

The systematic investigation of merit and worth. – Thomas Guskey, 2000

Which best describes your experience?

A BExternally driven and designed Internally driven and designed

Summative evaluations only Planning, formative, and summative evaluation

Event-based Program-based Looking for answers/solutions form others

Discovering or creating solutions and alternatives with others

Feared Embraced

Filed/shelved Used

Done as an afterthought Planned from the beginning

Documentation Evaluation

Process-focused Results-driven

Presentation of results Reflective dialogue

Shifting PerspectivesFrom To

Externally driven and designed Internally driven and designed

Summative evaluations only Planning, formative, and summative evaluation

Event-based Program-based Looking for answers/solutions form others

Discovering or creating solutions and alternatives with others

Feared Embraced

Filed/shelved Used

Done as an afterthought Planned from the beginning

Documentation Evaluation

Process-focused Results-driven

Presentation of results Reflective dialogue

Your Evaluation Process

Write 4-5 sentences that describe the evaluation process/steps you currently use as a leader responsible for assisting others with evaluating professional learning.

Tech Tool Idea:

Penzu – online writing journal

Group Think – Table talk, share out or Padlet

1. What aspect of evaluating professional learning do you find essential?

2. How do we know that the professional learning is making its way to the classroom?

How and Why of EvaluationGood evaluations are the product of thoughtful planning, the ability to ask good questions, and a basic understanding about how to find valid answers. In many ways they are simply the refinement of everyday thinking. Good evaluations provide information that is sound, meaningful, and sufficiently reliable to use in making thoughtful and responsible decisions about professional development processes and effects (Guskey & Sparks, 1991).

Lead Box EvaluationsSuperman X-ray

Input Output

Actions Results

Glass Box Evaluations

?A simplistic approach to professional learning evaluation that fail to amplify the underlying theory and operation of the professional learning program.

A comprehensive approach to professional learning evaluation that illuminates how professional learning program components interact to

produce results.

Lead Box

ProfessionalLearningAction

Student Achievement

Results?Focus on outputs rather than what

occurs in the program or what is

presumed to be causing those

outcomes and why

Student Achievement

ResultsProfessionalLearning

CurriculumDevelopment

Nonacademicfactors

?

Lead Box

Focus on inputs and fail to shed light on HOW a

program’s activities and

resources interact to produce results.

Student Achievement

Results

ProfessionalLearning

Actions

Glass Box Focus on what

occurs and how it occurs within the

program.

Glass Box

Student Achievement

ResultsProfessional

Learning

Coaching/Follow-upInstructional ResourcesImplementation MonitoringStudent Assessment

Focus on illuminating

factors contributing to transformation

process.

Lead Box vs. Glass Box

What is the difference?

Thorough planning facilitates sound evaluation.

4 Key Components to Evaluating PD

Systematic(Established process)

How rigorous is the process?Is it conducted in accordance with standards and guidelines?

Standards(Predetermined criteria)

Does it have merit and/or worth?Does it meet predetermined standards of success?

Audience Who will use the evaluation?For whom is the evaluation being done?

Intended Uses

How will the evaluation be used?What decisions will be made as a result of the evaluation?

8 Smooth Evaluation Steps

Planning1. Determine

‘what’ to evaluate

2. Formulate Evaluation Questions

3. Construct Evaluation Framework

Conducting4. Collect Data

5. Organize, Analyze, & Display Data

6. Interpret Data

Reporting7. Disseminate

and Use Findings

8. Evaluate the Evaluation

3 Types of Evaluation

1. Planning – before program design to provide information on conditions or needs to address

2. Formative – during implementation to provide information on whether the program is working as designed

3. Summative – after completion to provide information on outcomes or overall impact

Tiers and Benchmarks

1. Multiple settings

2. Data sources – affective, quantity, performance data

3. Initial vs. embedded

4. 5 year plan for data collection

5. Universal Design for Learning (UDL)

Reflection• Review your 4-5 sentences about evaluation

• Look at the Padlet responses

• ‘Steal’ ideas to take back to your district

• Reflect on what you can do differently as result of your new knowledge

Change is Learning Concerns-Based Adoption Model : Developed by Bill Rutherford, Gene Hall, Shirley Hord, and Susan Loucks-Horsley

4 Components:

1. Stage of Concern – 7 stages of responses

2. Levels of Use – eight ranges of intervention use

3. Innovation Configuration – described actions

4. ‘Change’ facilitators – leaders of learning

Change Learning Exchange• Distribute numbered cards (#1-4) at your table.• Read and the corresponding article on change.• Prepare a two-minute talk about your article. Use any of

the following to prepare.– Why your focus area is important.– Implications of the change process.– Ways to facilitate your area of change. – Challenges you anticipate when helping others

understand this area of change.

Change Spotlight• Find a partner that read a different article.• Take two minutes each, discuss your article.• Focus on any of the following:

– Why your focus area is important.– Implications of the change process.– Ways to facilitate your area of change.– Challenges you anticipate when helping others

understand this area of change. • Listen for the timer to repeat the process.

Note to Self

What new insights did you gain as a result of your reading and discussion with others?

Share Out

Weebly Activity• Access the Toolkit

• Find the ‘Evaluation’ Tab

• Read and Complete the activities

Share big ideas as a group

34

Summary & Wrap-up

Set Standards for Acceptable Performance

• Specify how good is good enough

• Specify “success” in advance

• Provide a benchmark/baseline for comparison before and after professional learning

Teachers participate in collaborative learning experiences.

Teachers implement new learning in their instruction.

Student performance increases.

Evaluation Assumptions• The staff development program is data-driven, research-

based, and well-defined.

• The school, district, or regional agency has the capacity, including fiscal and human capital, to implement both the program and evaluation with fidelity to their designs.

• Key stakeholders in the school, district, or agency intend to use the evaluation results to make decisions about the program.

Toolbox

Debrief• How are the ideas presented today

CONNECTED to what you already knew?

• What new ideas did you get that EXTENDED or pushed your thinking in new directions?

• What is still CHALLENGING or confusing for you to get your mind around? What questions or wonderings do you now have?