renew evaluation

RENEW EvaluationSarah Hough, Ph. D. & Niki Sandoval,

M. A.

University of California, Santa Barbara

Purpose of RENEW Evaluation

• Are implementation goals met?• Are project activities moving participants toward

the specific pedagogical objectives of–Increasing participants’ understanding of mathematics

• Does the project help to retain teachers in the profession?

–Increasing participants’ ability to implement national standards in mathematics–Increasing participants’ understanding of equity issues and their ability to work with English Learners

RENEW Grows

Year 1

76 preceptees

30 preceptors

42 schools

8 districts

Year 2

107 preceptees 125 b. teachers

29 preceptors

33 schools

8 districts

Year 3

169 preceptees

56 preceptors

55 schools

9 districts

Current Participants

• 136 preceptees• 110 new teachers attended summer institutes

• 36 preceptors

• 57 schools • 9 districts

Activity Increases

0

50

100

150

200

250

Year II Year III

MEPSMEPS hoursVisitations

What New Teachers Say

Every time I attend a math institute I become more confident and empowered in math. I feel like a better teacher every second I am present in the institute. summer institute participant

I feel more confident when analyzing statistical data now based on the activities we did as a group. summer institute participant

Cross institute feedback in terms of • growth in leadership• What beginning teachers said they gained from the institutes.• How many of the BTs signed up for participation in RENEW• How many returning preceptees from previous years

Juggling Pedagogical Juggling Pedagogical Content KnowledgeContent Knowledge

Student Diversity

SubjectMatter

Beliefs About Math

Instructional Strategies

Beliefs About Teaching

Efficacy

Differences in Student Thinking

Curriculum

CONTENT

PEDAGOGY

STUDENT

Collecting Data About Beliefs Each year renew beginning teachers fill out a

questionnaire that includes:

Two graded response scales that measure beliefs about mathematics and beliefs about pedagogy

Beliefs About Mathematics and it’s Teaching Before

Participation

Math as procedures

Math as investigation

Teaching as “telling”

In line with PSSM

Not in line with PSSM

Change in Preceptee Beliefs

Math as procedures

Teaching as “telling”

Math as investigation

ß

Collecting Data About Learning Content

• Pre and post “statistics” concept maps from 22 beginning teacher participants at the UCSB institute were analyzed for change.

• Content Analysis--themes of institute were: measures of central tendency, graphical representations, probability, sampling and bias.

Content Analysis of Maps

GraphsVennDiagrams

Bar charts

T-charts

Tables

Statistics

Representing

Pie-charts

concept

Chunk

Math Content Analysis of MapsAppeared on Map as

ConceptAppeared on Map

as ChunkPre Post Pre Post

Measures ofCT

50% 73% 5% 43%

Graphing 68% 100% 33% 81%

Sampling 0% 50% 0% 38%

Probability 59% 77% 19% 33%

Bias 10% 68% 10% 29%

DataCollection

45% 64% 15% 31%

Data AnalysisandInterpretation

27% 50% 0% 38%

Example 3-pre map

Statistics

Median

Baseball players

Mean

Average Middle

When do we really use this?

Histogram

Bar graphs

ModeMost

Frequency

Races

Example 3-post mapStatistics

Median

Not always useful

Mean

Average Middle

Box and whisker

Bar graphs

Mode

Big 3

Most frequent

Stem and leaf

Reinforces results

GraphsVennDiagrams

Pie-chartsBar charts

T-charts

Analyze Data

Changes in Subject Matter StructureMean Pre

Mean Post

Range Pre

Range Post

#Concepts 24 35 7-39 13-58

Width+depth 15 21 8-29 12-37

#Chunks 4 6 0-10 0-18

#Links 0 1 0-1 0-5

Participant view of their own learning

• When participants were asked to reflect on their learning using the pre and post concept maps the majority of them agreed that they had added more concepts to their maps

“I ran out of room for all of my thoughts on my 2nd concept map! What a great visual assessment of my learning. I hadn’t realized just how much I really did learn this week until I saw the difference between Monday’s and Friday’s maps. ”

Participant view of their own learning

• And, that their maps were more sophisticated and connected

“My first graph does not have as many connections with the sub categories as the second one. I have a deeper understanding of topics now, for instance, that the mean could get affected by a very big number , while the median could stay the same.”

Change in PCK--Vertical Curriculum Knowledge

“Prior to this week’s activities, I attributed statistics more to a college level or advanced math level. However, now I understand and am excited to see how statistics applies to the elementary level of teaching and learning.”

Change in PCK--Horizontal Curriculum Knowledge

“What I have come away with is not only a better understanding of the content of statistics but more importantly, how to integrate it into my classroom. I can honestly say statistics was not a topic I covered in my first year of teaching but I feel I now have many ideas/topics that would be relevant to my grade level as well as have student interest. am very excited and plan to use Math By All Means –Probability Grades 1-2. this year I plan to use the textbook less and use Math By All Means to supplement the lesson ideas. ””

How RENEW Fosters PCK

• Math Activities

• Classroom connections(e.g. Good Questions for Teaching Mathematics)

• Understanding English Learners(e.g. Language oppression and

implications for learning of math)

Change in PCK--Understanding of Student

Diversity“Bias is a concept on my second map, but not on my

first. I had heard bout statistics being “used” to support an opinion but did not truly understand how the same information could say such different things. In the activity looking at the work force comparing gender and race, though are groups were given the same data, the results were quite different. Looking at sampling size, locations, etc. makes a huge impact on what the statistics on a given topic may say. “

Change in PCK--Understanding of Student

Diversity“One thing that was illustrated a lot was how

differently people think about the same thing, how we each approach and solve problems in various ways. Its great to learn different perspectives by working with others. The language and statistics activity was great for emphasizing how much more I need to be conscious of possible language barriers in my classroom as opposed to the students not understanding the material. I’m really glad that you incorporate social issues like race/ethnicity and sexism.”

Collecting Data About EfficacyEach year renew beginning teachers fill out a

questionnaire that includes:

Six graded response questions about efficacy

Efficacy to implement effective teaching practices

• Manage a class of students engaged in hands-on project work

Not adequately prepared

Very well preparedSomewhat prepared

Fairly Well prepared


• Help students take responsibility for their own learning





• Recognize and respond to student diversity





• Use strategies that specifically encourage minorities and females





• Lead a class of students using investigative strategies





• Encourage student interest in mathematics




Retention

• 93% of RENEW beginning teachers are still participating in RENEW after 1 or more years in the project.

How is RENEW developing PCK in BTs

• By providing intellectual and emotional support to the Preceptor cadres with whom they work

• By providing professional development that focuses on leadership to these experienced teachers.

The project is collecting data using multiple methods to trace the development of leadership capacity in preceptors

RENEW Evaluation on the web

http://renew.education.ucsb.edu

Research and Evaluation

Papers and Presentations, Reports

http://renew.education.ucsb.edu/

renew evaluation

Documents

math institute

participants ability

teacher participants

themes of institute

ucsb institute

beginning teachers

learningwhen participants

post concept