do you know what your students know? a principal’s guide to improving student achievement lani...

Do You Know What Your Students Know?

A principal’s guide to improving student achievement

Lani Seikaly, partnerHillcrest and Main, Inc.

http://hillcrestandmain.com and project director of MDK12

http://mdk12.org [email protected]

http://hillcrestandmain.com/

http://mdk12.org/

Lani Seikaly, Hillcrest & Main, Inc.

What do principals of low performing schools need to put in place to improve their student achievement?

Where do they need to focus their staff’s time and school resources?


Focus Question

What can you learn from your AYP Data?


What can you learn from your AYP data?

• Are you meeting the target?• Do the data help inform instruction?• If your results are similar next year, will

you meet the 2006 AMO? The 2007 AMO?

• What implications does this have for your school?

State assessments only inform schools where their students are performing at the time of testing.

Teachers must know where their students are at any point in the

school year.


Thanks to NCLB and the mandate to assess state standards, states and districts have a target that is known and relatively stationary.


Though standards-based education precedes NCLB by a decade, the accountability programs designed to meet AYP rules have been the catalyst for states and districts to take a closer look at the degree to which their teachers understand and teach the content standards and assessments align with their standards.


How long has your state had state content standards?

How long have they assessed them

How long has your school been

accountable for student performance on them?

What are the consequences for not meeting state performance standards?


Is the focus on standards resulting in improved

achievement?


"After more than a decade, it's fair to be asking whether the standards-based approach to education reform works. We're seeing pretty strong evidence that it does," said Education Week Research Director Christopher Swanson.


A report by the magazine Education Week released this month found that states that made the largest gains on the NAEP -- including Delaware, Massachusetts, North Carolina and Texas -- were also the most fervent and longest supporters of standards-based education reforms.


The report, "Quality Counts 2006," found that factors such as per-pupil spending and student demographics had less of an impact on student achievement than a state's history of raising expectations and standards.

http://www.edweek.org/ew/toc/2006/01/05/index.html


How do state standards and NCLB change expectations for what happens in our schools?


Standards-based reform changes everything

• Defines what we expect all students to know and be able to do.

• Expects educators to take all students to proficiency on those standards.


Before Standards

Teachers taught what they thought was important.

After Standards

Teachers are expected to teach the content standards.


Before Standards

Teachers had different expectations for different groups of students.

After Standards

Teachers are expected to take all students to proficiency.


Before Standards

The focus was on how well teachers taught.

After Standards

The focus is on how well students learn.


Before Standards

Only selected students were enrolled in higher level courses.

After Standards

All students are expected to have equity of opportunity for enrollment in higher level courses.


Standards-based education makes a huge difference in expectations for teachers. And so it becomes even more critical to know that teachers understand and are teaching the content standards identified for their grade level courses.


What activities do teachers need to engage in today that they might not have

done 10 years ago?


Align instruction and assessment with state content standard indicators

Know where your students are performing on the indicators

Diagnose what students know and still need to learn in relation to those indicators

Engage in grade level team examinations of student work

Work toward a common understanding of content standards


Focus Question

What are the reasons for low performance?


What are the reasons for low performance?

• Have you clarified your problem?

• What are your hypotheses for low performance?

• What do experts say?


“School improvement plans … are typically oversized, imprecise, and an obstacle to improvement. They set off a riot of activities -- which supplant the work of teachers to create, adapt, and evaluate lessons and strategies aimed at helping higher proportions of students master essential standards.”

Mike Schmoker in “The Real Causes of Higher Achievement”


To accurately identify the problem, teams need data about their instructional program.

There are five processes that need to be in place to hit any instructional target and should serve as the basis for an exploration of your instructional program.


How to Hit your Instructional Target

• Understand the target

• Teach the content standards

• Assess the content standards

• Monitor student progress

• Intervene with students not succeeding


Questions we need to ask …

Do all teachers Know the content standards and

indicators they are responsible for teaching?

Understand the intent of the indicators? Know what the state assessment is

testing and how it is scored? Recognize proficient performance? Align instruction and assessment with

content standard indicators? Diagnose what students know and still

need to learn and monitor their progress?


What are your hypotheses for any areas of low performance in your school or district?


I recently had the opportunity to work with 18 Title I schools in a large school district, 10 of whom had not meant AYP for two years and were on the “needs improvement” list.


I was asked to design and lead training for principals and their leadership teams that resulted in moving off (and or staying off) the “needs improvement” list.


The journey was bumpy

but we learned a lot about operationalizing school improvement research into an action plan that improved student achievement to meet the AYP target.


Problem clarification raised a number of questions for our schools.

• Were we teaching the content standards we were testing?

• Were the state content standards embedded in our district curriculum?

• Did anyone know where our students were on mastering the content standards on a daily or weekly level?

• Were we collecting classroom formative data?


“In some of our lowest-achieving schools, there is a patent mismatch between the real, taught curriculum and the actual standards that are assessed -- by state, standardized, or district assessments.”Mike Schmoker in “The Real Causes of Higher Achievement”


“For all our so-called common curriculum, very little has been done -- let's be honest -- to ensure that the taught and the tested curriculum are aligned.”

Mike Schmoker in “The Real Causes of Higher Achievement”


After surveying staff to better understand our problem, we also found:

• Staff didn’t understand the AYP target

• Staff didn’t align instruction and classroom assessment with content standards

• Staff didn’t monitor student progress

• Staff believed their instruction had less of an impact on student learning than demographics (particularly poverty and special needs)


After surveying principals to better understand our problem, we found:

• Principals didn’t know how to lead the data analysis discussion

• Principals didn’t expect teachers to collect monitoring data

• Principals didn’t expect teachers to use diagnostic data to inform their instruction


While conducting classroom walkthroughs, we found:

• Classroom work and assignments not aligned with content standards

• Inconsistent evaluations of student work between teachers

• Rubrics and scoring tools not consistent with content standard indicators

• Inconsistent evidence of team planning• Lack of evidence of regular teacher

feedback and student revisions


By the end of problem solving, it was painfully clear that

• Staff were not clear on what students were expected to know and do and what good performance looks like.

• Staff taught their curriculum with little if any information about what their students already knew and still needed to learn.

• Schools didn’t know what their students knew and didn’t know.

• Nonetheless, staff graded students regularly (usually based on sorting and comparing criteria rather than standards-based criteria).


What do our educational experts believe is important?


Though we have a good deal of research to draw on in identifying what effective schools and leaders do, translating the research into concrete, practical actions has been more elusive.


We agreed with Mike Schmoker that “The combination of three

concepts constitutes the foundation for results: meaningful teamwork; clear, measurable goals; and the regular collection and analysis of performance data.”


“Teachers show what they believe is worth learning by the way they allocate time in the classroom and by the emphasis they convey to their students about what is really important.”

From A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives

So where did we go from there?


AYP provides us a relatively stationary target.

However, to meet AYP, we need to monitor student level data at the classroom level on an ongoing basis


So we turned to developing a monitoring system that focused on those content standards assessed by the state and that would give us a better idea of where our students were on an ongoing basis.


Then we focused on how to monitor student progress.

• We asked teachers to collect monitoring data from their classrooms and submit it every two weeks.

• We held regular data analysis discussions to discuss what teachers were finding.


We encountered misunderstandings

• Some teachers felt the data they collected was an aside from their day-to-day instruction.

• Some schools thought discussing the monitoring data as a set of numbers rather than diagnostic information was what we wanted them to do.

• Some schools thought using quarterly assessments would give them the information they needed to inform instruction.


What did we learn?

“I think monitoring has improved learning in our school in many ways but probably the biggest thing is that it has focused us. We have learned that the rigor and challenge in our curriculum have changed, and we have to be more focused and consistent with what we’re doing.”

Laura McCutcheon, Staff Development Teacher


What did we learn?• Monitoring tools sharpened school and teacher

focus on what they were expected to assess and, therefore, teach.

• Teachers were willing to comply with the data collection but saw no relationship between the data and their instruction.

• Teachers had collected no formative assessment data, only summative data.

• When asked what the student appeared to know and be able to do, their monitoring data could not tell them.


After all of that, we still couldn’t answer the

questions…

• Where are each of our students in relation to the content standards they must attain?

• What do they know and are able to do?

• What do they still need to learn?


So now what?

Our instruction was better aligned with the content standards but we still needed to understand what our students knew and still needed to learn.


We read what Black and Wiliams

had to say

about the research that documents the strong link between improved student achievement and the use of formative data in their Phi Delta Kappan article, “In the Black Box: Raising Standards through Classroom Assessment.”


Black and Wiliam in their 1998 Phi Delta Kappan article,

“In the Black Box: Raising Standards through Classroom Assessment,”

assert,“There is a body of firm evidence that formative assessment is an essential component of classroom work and that its development can raise standards of achievement. We know of no other way of raising standards for which such a strong prima facie case can be made.”

http://www.pdkintl.org/kappan/kbla9810.htm

http://www.pdkintl.org/kappan/kbla9810.htm


If formative assessments are so critical, then how do we create

good assessments?


We examined the National Research Council book, Knowing What Students Know, which identified the key concepts on which good assessments are built.From Knowing What Students Know by James W. Pellegrino, Naomi Chudowsky, and Robert Glasser/


Every assessment, regardless of purpose, rests on three pillars:

• Cognition, a model of how students represent knowledge and develop competence in the subject domain

• Observation, tasks or situations that allow one to observe students’ performance

• Interpretation, an interpretation method for drawing inferences from the performance evidence thus obtained.

James W. Pellegrino, Naomi Chudowsky, and Robert Glasser


To effectively monitor student progress, staff needed to understand

the knowledge and cognitive domains of the content standard indicators

how students learn

how to provide students opportunity to demonstrate what they know

how to interpret student responses


Knowing What Students Know also suggests that

“Student work should focus adult-student and

adult-adult conversations to discuss standards.”

We also read about the importance of collaborative

examinations of student work.


If we are serious about meeting the NCLB goal to take all students to proficiency, then this is not solo work. Grade level teachers must regularly examine student work as a team.


Joan Richardson, editor of the National Staff Development Council newsletter, believes that

“The practice of having teachers work together to study student work is one of the most promising professional development strategies in recent years. Examining student work helps teachers intimately understand how state and local standards apply to their teaching practice and to student work.


Teachers are able to think more deeply about their teaching and what students are learning. As they see what students produce in response to their assignments, they can see the successes as well as the situations where there are gaps. In exploring those gaps, they can improve their practice in order to reach all students.”


The Aspen Workshop on High Schools recommended in its summary report for the Transforming High Schools Task Force that the continuous and collaborative examination of student work along with the personalization of schooling are the two critical strategies for transforming high schools at the local level.


“At the risk of sounding overly simplistic, the use of student work as the unrelenting focus of adult conversations can be the catalyst of fundamental changes in the educational experience of adolescents, and the transformation of teaching and learning at the high school level.”


“Assessment should be used not simply to judge how much kids know but to illuminate the nature of their knowledge and understandings in order to help kids learn.... Common sense tells us that on-going, classroom-based assessment can serve this

purpose.”

Niyogi, Nivedita S. 1995. The Intersection of Instruction and Assessment: The Classroom. Princeton, NJ: Educational Testing Service.


Kate Nolan, Director of Re-Thinking Accountability for the Annenberg Institute of School Reform, believes

“The process of studying student work is a meaningful and challenging way to be data-driven, to reflect critically on our instructional practices, and to identify the research we might study to help us think more deeply and carefully about the challenges our students provide us.”


Kate Nolan, Director of Re-Thinking Accountability for the Annenberg Institute of School Reform, believes

“Rich, complex work samples show us how students are thinking, the fullness of their factual knowledge, the connections they are making. Talking about them together in an accountable way helps us to learn how to adjust instruction to meet the needs of our students.”


So we understood

that to know where each of our students is in relation to content standards, we needed to regularly monitor student progress and examine student performance to inform our instruction. What would that look like? How would we operationalize that?


Focus Question

What does a team examination of student

work look like?


What does a team examination of student work look like?

• How do teachers define proficiency?• How do teachers diagnose strengths

and needs?• How do they record the diagnostic

information?• Who leads the discussion?• What questions need to be asked?


We created an examining student work protocol for our teams to diagnose student strengths and needs with the primary purpose of informing instruction.


Only when teachers know where their students are in relation to the objectives they are responsible for do they have the information they need to inform instruction.


Examining Student Work

Though teachers have always examined student work as part of their grading process, the new focus on accountability and standards has driven a more structured and collaborative examination of student work.


Examining Student Work

The focus of the examination has shifted from a summative evaluation of student performance to a diagnostic evaluation of student performance, teacher assignment, and implications for instruction.


The Examining Student Work Protocol asks teachers to

• Identify characteristics of proficiency on an objective using a specific assignment/assessment

• Diagnose student strengths and needs on the performance

• Determine next instructional steps based on the diagnosis


In the first part of the protocol,

a team of teachers work through the process of reaching consensus on what the team believes constitutes a proficient response on a selected text and question.


Part 1: Reaching Consensus about Proficiency

• What did you ask the students to do? • Which Maryland Content Standard indicator

and objective were you assessing? • What did you consider proficient

performance on this assignment? • Exactly what did students need to say or

write for you to consider their work proficient?


Stiggins argues that we really can’t assess accurately if we don’t understand the target:

“To assess student achievement accurately, teachers and administrators must understand the achievement targets their students are to master. They cannot assess (let alone teach) achievement that has not been defined”.

Stiggins, Richard J. 2001. “The Principal’s Leadership Role in Assessment.” NASSP Bulletin (January 2001): 13–26.


A pre-requisite to interpreting student work is a clear understanding of what you are looking for. What does a proficient response look like? What exactly do your students need to know and still need to learn?


And yet we found the greatest challenge was to reach consensus on what proficiency looked like on the assignment. This is particularly challenging on the reading standards where proficiency must be defined for each text being read.


Only after the team has agreed on what constitutes a proficient response are they able to diagnose student strengths and needs.


Teachers were surprised at how “off

the page” their teammates were.

Think of all the mixed messages our students are receiving when we haven’t defined proficiency on a standard / indicator in the same way.


It is not enough that an individual teacher defines proficiency.

It is critical that at least a grade level team has reached consensus on the definition of proficiency to ensure that all students are held to the same performance expectations.


In the second part of the protocol,

the team examines three student papers to determine if the response is proficient and to identify strengths, needs and instructional next steps.


Part 2: Diagnosing Student Strengths and Needs to Inform Instruction

• What did the student demonstrate that they knew?

• What misconceptions or wrong information did the student have?

• What did the student not demonstrate? • How would you find out if they knew it? • Based on the team's diagnosis of the student

performance, what do you do next with that student?

• What do you need to re-teach the class?


Each teacher will be asked to bring three samples of student work from the same assignment or assessment: a response at the top of the class, a response at the bottom of the class and a response in the middle of the class.


Teachers must shift their mindset from scoring (a summative examination) to diagnosing (a formative examination) student performance.


In many cases teachers have spent a great deal of time sorting student responses (either by letter grades or by rubric scores) and virtually no time diagnosing what students know and still need to learn.


Focus Question

What can you learn from examining student

work?


What can you learn from examining student work?

• What do teachers learn about students?

• What do teachers learn about their instruction?

• What do teachers learn about their team’s understanding of content standards?

What did our teachers say about what they learned?


Andrew Nelson, teacher at Harmony Hills E.S.

“I wish I had caught onto this earlier but at one of the meetings, it became apparent that kids weren’t reading the question. That was a big awakening to me because I was so focused on how to write the answer, we hadn’t spent time unpacking the question.”


Glenn Messier, teacher at Harmony Hills E.S.

“My teammates proficient responses were a little bit more advanced than what I was expecting and looking for. They were looking for a lot more in-depth answers. To get on the same page, I needed to raise the bar for my students.”


Ilise Wolf, teacher at Harmony Hills E.S.

“When you’re working on a team, I really find I get a lot of ideas from my teammates and support and feedback. What I might consider a good assignment for a child, another 2nd grade teacher might have a few extra words to add that would really make a difference to my students.”


Eunice Goring, Academic Support Specialist

“I’ve seen a lot of growth. Before they were all over the board about what a proficient response looked like. I’ve seen a lot of growth in teachers in terms of defining proficiency and being able to have a more realistic and rigorous view of what their students can do.”


How did the teams capture the data?

What did we learn?


Teachers came out of this examining student work process with

• Grade level consensus of what constitutes proficient work on the assignment

• Formative assessment data• Specific information to inform their instruction• Strategies for re-teaching• Deeper understanding of the intent of the

standard / indicator they were assessing.• Probing questions to ask students to better

understand where they were


We learned

When you have teachers collaborating with other teachers and interpreting student performance on their own assignments, you have raised the amount of investment teachers have in the process and in making changes to their practice.


We learned

Teachers began to reach consensus on what they are looking for in an assignment before they gave it to the students. Therefore, their teaching was also more aligned with the proficient criteria they had identified.


We learned

Principals needed to set clear expectations for their teams, identify end products and monitor whether they were met,


We learned

Most importantly, the process mandates the regular collection of student performance data that is both interpreted and analyzed for where the student needs to go next instructionally and used to modify instruction.


We learned that we could improve student achievement and meet

AYP.


Focus Question

How do principals structure the ongoing

examination of student work?


What structures need to be put in place?

Where will you find the time? What end products do you want

teachers to produce? How do you establish a culture for

learning communities? What capacity building do teachers

need?


How do we build teacher capacity to do this?

Shouldn’t we do that before we start the process?


This is on the job training.

The focus on interpreting student performance and determining what teachers need to do to support student performance allows teachers to examine their own practice through the lens of student needs rather than the lens of good versus mediocre teaching.


The fact that the process is ongoing allows teachers to build capacity over time rather than try to absorb everything in an upfront training.


What would the system need to include?

• Clear expectations for what teachers are expected to do and why

• Time for teams of teachers (teaching the same content standards) to meet weekly to reach consensus on proficient performances and to examine student work


What would the system need to include?

• A data recording system that demon-strates progress is being monitored

• A data capturing system that provides summary data on diagnosis of student performance and next instructional steps

• A monitoring system that provides the principal with evidence that the process is being implemented


Bottom line is

It is only when teachers and schools start to collect the data that provides them the necessary information about where a student is in relationship to the indicators they must master that effective data-driven decision making to improve student achievement will happen.


Focus Question

What expectations do you need to set?


What expectations do you need to set?

How often will you expect teachers to do this?

What will you expect teachers to do with the information?

What end products do you want them to produce?

How will you expect them to share their findings about student progress with you?

What’s negotiable; what’s not?


In his article, “The Learning-Centered Principal,” Rick DuFour describes his role to focus teachers on what students were learning in the following way:

“As principal, I played an important role in initiating, facilitating, and sustaining the process of shifting our collective focus from teaching to learning. To make collaborative teams the primary engine of our school improvement efforts, teachers needed time to collaborate.

http://www.ascd.org/frameedlead.html


Rick DuFour advises principals to• Provide time for collaboration in the

school day and school year. • Identify critical questions to guide the

work of collaborative teams. • Ask teams to create products as a

result of their collaboration. • Insist that teams identify and pursue

specific student achievement goals. • Provide teams with relevant data and

information.


DuFour continues:

“Teachers, accustomed to working in isolation, needed focus and parameters as they transitioned to working in teams. They needed a process to follow and guiding questions to pursue. They needed training, resources, and support to overcome difficulties they encountered while developing common outcomes, writing common assessments, and analyzing student achievement data.”


Staff members should be able to answer the following questions:

• Why does my principal expect me to participate in examining student work with my team?

• How will it benefit students? • Who’s in charge of ensuring this happens? • How often does my principal expect me to do

this? When? • How will my principal know that I am doing it? • What am I expected to produce as end products

to these dialogues? • What am I expected to do with these end

products?


How do you lead the process?

• What questions do you need to regularly ask?

• How do you monitor the process? How do you demonstrate its importance?

• How do you communicate your expectations?

• How are you using the information to inform staff development?

• How do you use regularly scheduled time to focus on the process?


Focus Question

How do you use student work to monitor student progress?


Reading Grade 6 Student Monitoring Plan Comprehension of Informational Text: Determine and analyze

important ideas and messages in informational texts

Objective Date/

Level

Comments Date/Level

Comments Date/level

Comments

Identify and explain the author’s text and intended audience

Identify and explain the author’s opinion

State and support main ideas and

messages


You can find more information on the School Improvement in Maryland Web:

• The Examining Student Work Protocol (http://mdk12.org/data/progress/using/m4w5/pr4/index.html)

• An online course, Using Data to Improve Student Achievement (http://mdk12.org/data/course)

• Monitoring Templates (http://mdk12.org/data/progress/developing/m4w2/pr2/index.html)

http://mdk12.org/data/progress/using/m4w5/pr4/index.html

http://mdk12.org/data/progress/using/m4w5/pr4/index.html

http://mdk12.org/data/course

http://mdk12.org/data/progress/developing/m4w2/pr2/index.html

http://mdk12.org/data/progress/developing/m4w2/pr2/index.html


Using Data to Improve Student AchievementUsing Data to Improve Student Achievement

Principals are increasingly more accountable for the achievement of their students. This online course is designed to build a principal's capacity to use data to improve student achievement.


Using Classroom Data to Monitor Individual Student ProgressUsing Classroom Data to Monitor Individual Student Progress

In this module, principals will enhance their skill in developing a system to monitor individual student progress and in supporting teachers in their collection, analysis, and use of classroom data to inform instruction and identify students who need interventions.


Lani Hall Seikaly Contact Information

• Email: [email protected]

• Phone: 703 867-3921

• Web site: http://hillcrestandmain.com

• Web site: http://MeetAyp.com

• Web site: http://mdk12.org

mailto:[email protected]

http://hillcrestandmain.com/

http://meetayp.com/

http://mdk12.org/

http://hillcrestandmain.com/index.html

http://hillcrestandmain.com/index.html


“We can, whenever and wherever we choose, successfully teach all children whose schooling is of interest to us. We already know more than we need to do that. Whether or not we do it must finally depend on how we feel about the fact that we haven’t so far.” Ron Edmonds

do you know what your students know? a principal’s guide to improving student achievement lani...

Documents

hillcrest main

state standards

standardsbased education

state content standards

state performance standards

standardsbased approach

state assessments

student performance