researching education as it happens using classroom ... · pdf fileusing classroom...

AMERICAN INSTITUTES FOR RESEARCH®

Researching Education as It Happens

Using Classroom Observations to Generate Quantifiable Data

American Educational Research Association

2004 Annual Meeting April 12–16, San Diego, Calif.

Researching Education as It Happens Using Classroom Observations to Generate Quantifiable Data

Submitted to:

American Educational Research Association

Submitted by:

James Stapleton Kerstin A. Carlson LeFloch

Amy E. Bacevich Beth Ketchie

April 2004

Table of Contents

Introduction................................................................................................................................... 1 Overview............................................................................................................................. 1

Project Background............................................................................................................. 1

Rationale ............................................................................................................................. 2

Data Organization......................................................................................................................... 2 Data Sources ....................................................................................................................... 2

Activity Units...................................................................................................................... 4

Procedure and Reliability.................................................................................................... 5

Observation Instrument ............................................................................................................... 6

Class and Demographic Information .................................................................................. 6

Narrative Log ...................................................................................................................... 6

Coding Structure ................................................................................................................. 7

Organization Codes............................................................................................................. 7

Engagement/Action Codes.................................................................................................. 8

Instructional Strategies Codes........................................................................................... 11

Materials Used Codes ....................................................................................................... 14

Results .......................................................................................................................................... 15

Selected Class Description and Behaviors Results........................................................... 15

Selected Instructional Strategies Results .......................................................................... 18

Summary of Findings........................................................................................................ 21

Conclusion ................................................................................................................................... 22

Benefits of Design............................................................................................................. 22

Future Adaptations............................................................................................................ 22

References.................................................................................................................................... 24

Appendix A. Demographic Information Capture Sheet ........................................................ 25

Appendix B. Narrative Log Sheet............................................................................................. 26

Appendix C. Organization and Engagement Coding Sheet................................................... 27

Appendix D. English Instructional Strategies Coding Sheet ................................................. 28

American Institutes for Research i

Chapter 1. Introduction

Appendix E. Math Instructional Strategies Coding Sheet ..................................................... 29

American Institutes for Research ii

Introduction

Overview

One difficulty faced by innovative educational efforts is proving effectiveness in areas outside of testable skills. Greater engagement on the part of students, use of higher order thinking strategies by students, better group awareness and dynamics, and more frequent use of a wider range of teaching strategies are some of the general outcomes that are more difficult to measure and prove but that are often claimed by alternative educational tools and programs. One strategy researchers use to access these claims is to observe and record classroom events in an effort to see firsthand the differences between classes using the materials or strategies in question and those not using them (Evertson & Green, 1986). The challenge for these observers is to create instruments that capture the processes mentioned above in a way that is reportable and useful for analysis. This paper presents one classroom observation instrument that attempts to do just that.

Project Background

The purpose of the study for which we created the classroom observation instrument was to evaluate two new supplementary curriculum packages, one for mathematics and one for English. The curricula cover grades 6 through 10 and were piloted in select districts across the nation in the 2002-2003 school year. The total programs involve extensive teacher training, multi-media materials, and follow-up support. The bulk of the programs are contained in a large package of printed material for the teachers consisting of background material, readings, activities, and projects formed into units designed to span the entire school year. After going through summer training, teachers implemented the materials in their classes as a pilot for possible continued use in the future.

In order to evaluate the programs overall, our team engaged in a number of data collection activities. We interviewed administrators and teachers individually, we held student focus groups, we collected and coded student work, and we looked at student achievement scores. In addition to receiving feedback from users, however, the client wished to specifically measure the effectiveness of the programs in terms of certain desired immediate educational outcomes. These outcomes included more interactive learning on the part of the students, more student-led and group work, a focus on inductive learning and student discovery of principles, more student reflection and interpretation, and greater student engagement. All of these, it was hoped, would be fostered through the materials and a higher frequency of teaching strategies aimed at encouraging all of these behaviors. As such, these instructional strategies were emphasized in the training that teachers using the programs received.

While user feedback could provide some insight into the degree to which the programs produced these events, such outcomes are not clearly measurable in terms of student scores or grades (Shavelson, Webb, & Burstein, 1986). While one would hope and assume that higher achievement would be one result of such events, the measurement of the actual occurrences of any of the outcomes and events listed above would clearly require careful observation at the

American Institutes for Research 1

Researching Education as It Happens: Using Classroom Observations to Generate Quantifiable Data

moment of instruction with an eye towards identifying and noting specific actions, organizing principles, and behaviors (Edmonds & Briggs, 2003). It was to convincingly get at the reliable measurement of such classroom events that the observation forms and coding system described in this paper were designed.

Rationale

Our rationale for relying in part on classroom observations arose from the desire of the client to know if the specific strategies espoused by the program trainings and materials were being put into practice in the classroom. We also performed teacher interviews, student focus groups, and student work collections, but all of these relay classroom strategies in a secondary manner. Direct observations combined with a plan to identify and record the strategies in question seemed the most legitimate way to record educational technique at the moment of occurrence. From this starting point, our team decided that the classroom observations would focus on answering the following five main questions:

1) What types of classes are using the programs?

2) To what degree are teachers who are using the programs implementing the program materials in the observed classes?

3) To what degree are the observed teachers who are using the programs successfully implementing instructional strategies advanced by the materials?

4) What are the differences in organizational and teaching strategies, if any, between classes observed that use the programs and those classes observed that don’t?

5) What immediate classroom outcomes can be perceived in terms of student and teacher engagement and behavior, and are there differences between the outcomes observed in classes that use the programs and those that do not?

In particular, our team wished to be able to clearly detect and demonstrate the existence or non-existence of the desired educational strategies and outcomes that happen within the classroom at the moment of instruction, and then give some meaningful and reportable measurement to those events.

Data Organization

Data Sources

The data used for this paper are the classroom observations of 66 English and mathematics classes in grades six through ten. Observation of the classes took place during the late winter and spring of 2003. Each observation was conducted by a member of our team, all of whom were trained in the use of the protocol. Observations lasted a minimum of 40 minutes,



though where the classes were longer the observers typically remained for the entire class. In both control and treatment cases the teachers were aware ahead of time that an observer would attend the class, and they knew that the purpose of the trip would be to study the program in question. Prior to the observation, the observer typically asked a few brief factual questions about the level, name, and materials of the class. After the observation, the observer asked the teacher if he or she wanted to elaborate on anything that occurred during the observation, spending a few minutes talking with the teacher if the teacher wished.

Of the 66 classes, 41 are treatment classes and 25 are control classes. Of the 41 treatment classes, 23 are English classes and 18 are mathematics classes. These treatment classes each used either the English or mathematics curriculum package being studied. Of the 25 control classes, 13 are English classes and 12 are mathematics classes. The control classes did not use the curriculum packages. The following chart summarizes these numbers:

Table 1: Numbers of Observations

English Observations

MathematicsObservations

Total Observations

Treatment Observations (using curriculum) 23 18 41

Control Observations (not using curriculum) 13 12 25

Total Observations (by subject) 36 30 66

We sought to match the characteristics of the control classes as closely as possible to the treatment classes, minus the curriculum package. In most cases we were able to use control classes in the same grade and school as the treatment classes. In those cases, all teachers observed typically had been trained in the alternative curriculum, but those teaching the control classes had chosen to use it and those teaching the treatment classes had not chosen to use the curriculum. In all other respects, the control and treatment classes were similar in size, demographics, level, and curriculum not related to the alternative packages, since all teachers covered the material required by the departments and districts (the alternative curriculum packages did handle much of what was required, but where they did not the teachers were still bound to cover that material).

Once all classes were observed and coded, the codes were compiled into a data set. In the end, the final data set contains a total of 551 records, where each record represents a discrete activity within one of the 66 classroom observations. Each of the 551 records possesses 122 coded variables, providing the detail desired for the subsequent descriptive analysis. The particulars of the activity designations, the codes and coding, and the analysis will be discussed in the sections that follow.



Activity Units

The basic unit of division for these classroom observation forms, and consequently the basic unit of analysis, is a classroom activity (Foorman, 2003). The start and end point of activities were recorded at the minute level. For this study, a change in activity is defined as a point at which the focus of the students, organization of the class, or nature of the work shifts. Examples of the three types of activity markers are given below. Where one of these markers change, that would signal the start of a new activity.

Table 2: Examples of Activity Markers Marker Types Activity Examples

Students engaged in whole class passive listening

Students engaged in whole class interactive discussion

Students engaged in individual reading or work Student Focus

Students engaged in interactive pair or group work

Teacher-led whole class organization

Student-led whole class organization

Small group or cooperative pair organization Class Organization

Independent work organization

Textbook/workbook written exercises

Assigned or student-selected reading

Original writing assignments Nature of Work

Visual presentation

All of these descriptions above are determined by what the majority of the class is doing for the majority of the task. So, in the case of student focus, if a student gives a known response short answer to a teacher's question, but the teacher is primarily lecturing, then the student focus remains whole class passive listening and a change in activity would not be signaled via student focus, even though the student focus may have shifted momentarily away from the teacher to the student who answered the question. If, however, the teacher ends a primarily lecture-based presentation and begins an extended discussion with the class that involves multiple responses and opinions from multiple students, then the focus has shifted to a whole class interactive discussion and a new activity would be coded. Class organization pertains more closely to physical arrangement of the class, be it grouped at stations, at desks facing a teacher, or in small groups, and the signal in a change of activity via this marker is often associated with a change in physical positioning of the students and class furniture, though the change in grouping may also be more psychological than physical (like the shift from many individual desks to one whole class). Lastly, the marker label nature of work is a change in the work itself or a change in the treatment of the work. Again, the general scope of the assignment is considered; so, if the assignment involves alternating between answering short questions aloud then writing the



answers, that would be one activity. If, on the other hand, the focus shifts from answering questions for a time to then writing for a time, that would signal a new activity.

Typically, in our observations, two or three of these signals occurred simultaneously at the change point from one activity to another. For example, if the students were writing in their journals and the teacher asked them to look up at the board in order to discuss the journal topic, that would be a change in activity signaled by the shift in student focus from their notebooks to the board, a change in room organization from individual work to a whole class activity, and a change in the nature of the work from writing to answering questions out loud and listening.

In the end, 551 separate activities were recorded across the 66 observations, leading to the 551 records in the data set mentioned earlier. Two of the 66 classes represent the minimum with one activity each for the entire class, while two other classes represent the maximum with 19 activities each. The rounded mean of all classes yields 8 activities, and 50% of all observations had between 4 and 10 activities while 79% of all observations had between 3 and 13 activities. Ultimately, these 551 activities all received codes for 122 variables apiece.

Procedure and Reliability

Each researcher was equipped with a classroom observation data capture instrument for each observation. In the first few minutes of the class, the observer would record the descriptive information in the first section of the observation form, the Demographic Information Capture Sheet (see Appendix A). This information was typically put together via a mixture of direct observation, on-the-spot inquiry, and follow-up with the teacher. Next, the observer turned to the Narrative Log, which occupied the majority of the observer's time (see Appendix B). Then, after class, we made every effort to take a few moments to talk briefly with the teacher. This time was used to clear up any technical questions about the level and name of the class, to answer any general questions the teacher may have had, and to allow the teacher to discuss anything that he or she may have wished to clarify about what we saw in class that day.

Lastly, the observers went back through each activity and did the coding. This was done at the earliest available time after the observation; typically, that time was in the evening of the day of the observation. In the end, some of the codes used in the categories measured highly quantifiable information. Examples of such data include the beginning and ending times of the class, the number of students, the physical materials and texts used during the class, and the physical groupings of the students throughout the period of observation. Other codes, however, called more upon the experience and training of the observer to make consistent judgment calls on what he or she was witnessing, both within and across classes, sites, and observers. The Instructional Strategies codes and the Engagement/Action codes represent sets of codes requiring the degrees of observational judgment on the part of the person watching the class in progress. Our strategies to boost inter- and intra-coder reliability included pre-visit training for the five observers, immediate team feedback during visits, and data verification by multiple team members after visits for codings identified as potentially troublesome by the observer. The training also focused on identifying shifts in activities.



Observation Instrument

Class and Demographic Information

The first sheet of the observation protocol is designed to capture the general descriptive and demographic information about the class (see Appendix A). This is information that describes the class for the duration of the entire class period or longer, and thus is not tied into the activity structure as is the rest of the information recorded in the protocol. For each observation, some of the information was learned prior to the visit, such as district, school, teacher, and program. Some of the information was extracted through a few pre-observation questions to the teacher. Such information might include class level, texts, and general student grade levels.

The remainder of the data was captured via the researcher's observations; this included the start and end times of the observation, the number and gender of students, and the general racial mix of students. The purpose of collecting the student demographic data in this instance was simply to have general descriptions of the classes. Short of asking each student to self-identify him or herself racially, there would not have been a reliable way to accurately count race at each observation. As such, this piece of data was treated in our report on a class and not a student level, i.e. heterogeneous versus homogenous racial mix. This same rationale applied for identifying gender mix, in that although there is typically less unreliability in identifying gender by sight than race, it is still not 100% reliable as a piece of individual level data when not corroborated by questioning or official records. Still, we deemed the judgment of the observer to be sufficient to make determinations at the broad level that we sought the data, i.e. balanced versus unbalanced gender mix.

Narrative Log

The narrative log is the second section of the observation protocol and it mainly provides a place for the observer to put down a running log of what he or she witnesses and hears as it occurs (see Appendix B). As with many other classroom observation protocols, the narrative log in this case was in fact the part of the observation form that the researchers spent the most time on during the actual observation. The observers were careful to record observed events and details only for the log, without added interpretation prior to the focus called upon later by the coding (Erickson, 1986; Spradley, 1980). Within that frame, however, they attempted to record as much as they could throughout the duration of the observation.

In this case, however, the narrative log served a second critical function. In a column to the side of the log the observers noted the clock time at each shift in activity. The determination of a shift in activity was made using the three criteria described earlier, and these criteria appear in the narrative as the researchers described the particulars of each activity. With the recording of the activity shifts and their times, the log in the end provides a total count of activities in the observation, a description of each activity, and the duration in minutes of each activity. With these three important pieces of information, the activities could be turned over to the final section of the protocol--the codes.



Coding Structure

The idea of developing codes arose from the client's wish to look for certain specific instructional strategies and student actions that the program is designed to foster. With the activity based approach in place, we felt we could count strategies used within a given activity and arrive at a fairly realistic picture of events. For example, if a reading activity lasted for six minutes, and the teacher used two of the strategies under study, then that could be coded as a six minute stretch of class time during which these two strategies appeared in relation to the task described. In fact, once that rationale was established for the observation format, it made sense to expand the codes beyond the specific strategies presented by the client to include other items within the realm of interest to the client. For example, student engagement could enter the coding system, as could teacher involvement and non-program strategies.

Ultimately, three general areas of coding emerged: Class Organization, Engagement/ Action, and Instructional Strategies (one set for math, one for English). In addition, some sub-categories were also developed, for example Materials Used and Racial Differences (Taylor & Pearson, 2000). In total, 101 codes were developed within 19 categories under the three general areas above. All of these are described next.

Organization Codes

The Class Organization codes were designed to allow for a coded description of the typical organizations observed in any given classroom. These codes focus on the way that the teacher physically organizes the activity and the way in which the class interacts within this organization. Table 3 lists these codes.

Table 3: Organization Codes Name Code Description Activity Organization 1 AO1 Teacher-led whole class activity Activity Organization 2 AO2 Student-led whole class activity Activity Organization 3 AO3 Small group or pair cooperative/collaborative activity Activity Organization 4 AO4 Independent activity Activity Organization 5 AO5 Other (please describe) Student Focus 1 SF1 Whole class passive (e.g., watching a movie, listening to a

reading, listening to a lecture) Student Focus 2 SF2 Whole class interactive (e.g., responding to questions,

offering suggestions) Student Focus 3 SF3 Student or group presentation to whole class Student Focus 4 SF4 Individual reading or work Student Focus 5 SF5 Pair work Student Focus 6 SF6 Interactive group work of 3-6 on a single project Student Focus 7 SF7 Other student focus

The Activity Organization (AO) codes include teacher-led whole class activity, student-led whole class activity, small group or pair cooperative/collaborative activity, independent



activity, and other, with the observer filling in other noteworthy details. The Student Focus (SF) codes include whole class passive, whole class interactive, and pair work, among others. There is a slight overlap between the two areas in that they both deal with the organizing principles of the activity. The distinction between the two is that the Activity Organization codes deal more with the physical distribution of the class, whereas the Student Focus codes deal with the organizing principles behind the personal interactions in the class. One example of how these two codes interact would be in an activity where the teacher is at the head of the class discussing a novel. That is a teacher led activity directed towards the whole class, so it would receive an AO1 code under Activity Organization. The next question is to determine how the students are interacting within that organization. If the teacher has set up a dynamic where the students are participating in the discussion, then under Student Focus the activity would receive a SF2 (whole class interactive). On the other hand, if the teacher is lecturing about the novel with little or no student participation, then the activity is whole class passive and would receive a SF1. The point is that within a single physical organizing principle different interpersonal dynamics can arise and this interplay can be addressed with these two sets of codes. Finally, it should be noted that a shift in either one of these code categories indicates a shift in activity, as these categories are two of the activity change markers; therefore, an activity will receive only one of each of these codes.

Engagement/Action Codes

The next set of codes that we developed revolve around levels of engagement. In a sense, these are immediate outcomes of, in part, the instructional strategies examined. These codes fall into two general categories, those for students and those for teachers (Kim, Briggs, & Vaughn, 2003). The student relevant codes are listed below.

Table 4: Student Engagement Codes Name Code Description Student Behavior 1 SB1 No significant classroom disruptions Student Behavior 2 SB2 Small number of classroom disruptions from multiple sources Student Behavior 3 SB3 Small number of students persistently and significantly

disruptive Student Behavior 4 SB4 Large number of students significantly disruptive throughout

activity Student Behavior 5 SB5 Other behavior issues Student Engagement 1 SE1 0-20% approximate percentage of students exhibiting on-task

behavior for majority of activity Student Engagement 2 SE2 21-40% approximate percentage of students exhibiting on-

task behavior for majority of activity Student Engagement 3 SE3 41-60% approximate percentage of students exhibiting on-




task behavior for majority of activity



The Student Behavior codes focus on outward displays of disruption, while the Student Engagement codes attempt to gauge the level of engagement of the class in terms of the numbers of students that are generally on task. Both the Student Behavior and Student Engagement codes were designed to characterize an entire activity, and as such only one would be given per activity. The engagement codes require more attention from the observer to try and examine the students in the class and determine how many are on task. The 20% ranges were established to account for the potential difficulties in doing this, combined with the needs of the analysis in that this range was deemed accurate enough for our purposes. Potential difficulties would include individual work assignments such as problem-solving from a textbook, in which case a quiet student writing does not always mean a student who is working on the particular assignment. In this case the observer, as with the teacher, must rely on apparent behaviors and close observation in as much as the observer is able to see what the student is doing. The observer is at one disadvantage to the teacher in that the observer is not mobile and so determination of individual student work becomes less reliable the further the student is sitting from the observer. On the other hand, it is not the experience of this team that students typically attempt to disguise off-task behavior from the observer, whereas they may do so for the teacher.

The scale was split in the last two codes (SE5 and SE6) into ranges of 10% instead of 20% to account for the fact that an activity in which almost all students were on task should be able to be coded (SE6) and distinguished versus a potentially more common event in which a great majority of students stayed on task (SE5). Taken together, the purpose of these codes is to give a sense of how the students are reacting to the work, environment, teaching strategies, and other stimuli present in the classroom at the moment. These codes do not establish causality, obviously, but over the series of several collections they may establish relationships between types of student behavior and levels of engagement and certain activities and between types of student behavior and levels of engagement and the presence of certain instructional strategies.

There is also a set of codes dealing with teacher action. These codes are included under the sub-heading of engagement/action codes in that they describe the behavior and engagement of the teacher towards the students. These codes are broken into Teacher Activities codes and Teacher Time Related to Instructional Activity codes and are listed below.



Table 5: Teacher Action Codes Name Code Description Teacher Activities 1 TA1 Presenting information Teacher Activities 2 TA2 Leading student work Teacher Activities 3 TA3 Supporting student work Teacher Activities 4 TA4 Providing feedback for students Teacher Activities 5 TA5 Evaluating progress Teacher Activities 6 TA6 Non-instruction class duties Teacher Activities 7 TA7 Non-instruction non-class activities Teacher Activities 8 TA8 Other (please describe) Teacher Time Related to Instructional Activity 1 TT1 0-20% approximate percentage of teacher's time related to

the instructional activity Teacher Time Related to Instructional Activity 2 TT2 21-40% approximate percentage of teacher's time related to




the instructional activity Teacher Time Related to Instructional Activity 6 TT6 91-100% approximate percentage of teacher's time related

to the instructional activity

The Teacher Activities (TA) codes describe the actions of the teachers in terms of instructional approach while the Teacher Time (TT) codes describe the percentage of time devoted to doing the actions described. Similar to the Student Engagement codes, the Teacher Time codes are designed to be assigned once per activity. The Teacher Time codes are not meant as a measure of effectiveness, in that one can easily imagine scenarios in which the teacher devotes all of his or her time to the activity but the students learn little (for example, where a teacher lectures extensively to a largely inattentive class) and vice versa (for example, where the students individually write an original analytical long response and the teacher chooses to engage in another task at that point). Instead, this category, as with Teacher Activities, is meant to be merely descriptive and provide data to establish patterns of teacher behavior with respect to given activities. Also, the codes help to establish amount of time spent on instructional strategies per activity when combined with the instructional strategy codes below.

Lastly, it should be noted that any number of Teacher Activities codes may be checked by the observer within one activity. Since activities are basically defined in terms of the actions of the students, teachers may shift through many different actions within one activity. For example, in an activity that requires students to work on a poster project in groups, the teacher may alternate between moving around the classroom and providing feedback for students (TA4) and sitting at his or her desk and grading a quiz (TA6, non-instruction class duties).

The client was interested in how different types of students received the materials. The Racial/Gender differences codes were created to account for any differences perceived in the



treatment of different types of students or the perception of the materials by different types of students. If used, the codes always required additional notes of explanation by the coder. These codes are listed below.

Table 6: Racial/Gender Differences Codes Name Code Description Racial Differences 1 RD1 No perceived racial/ethnic differences Racial Differences 2 RD2 One or more groups treated differently Racial Differences 3 RD3 One or more groups respond differently Racial Differences 4 RD4 Other racial/ethnic difference observed Gender Differences 1 GD1 No perceived gender differences Gender Differences 2 GD2 Genders treated differently Gender Differences 3 GD3 Genders respond differently Gender Differences 4 GD4 Other gender difference observed

Instructional Strategies Codes

The Instructional Strategies codes were based on the teaching methods and learning modes desired by the designers of the curriculum being studied. These codes were different between English and math classes, and any activity could display multiple strategies or even none of the strategies. For English, these strategies included such actions as “students respond to texts,” “students use mnemonic devices to memorize grammar rules, definitions, etc.,” and “students develop and present texts.” The complete list of English categories and codes are listed in Table 7 below. For mathematics, the strategies included such activities as “students use appropriate terminology in speech and writing,” “students identify and describe situations in which mathematics is useful,” and “students attempt multiple methods or discuss alternative types or methods.” The complete list of math categories and codes are listed in Table 8 below. In addition, we added a set of “lower order” codes to each of the sets of English and math content codes. These identified rote and procedural work and allowed us to account for activities that lacked the instructional strategies promoted by the curriculum material being studied. Counting the lower order codes, there were in the end 16 instructional strategy codes for English and 22 for mathematics.



Table 7: English Instructional Strategies Codes Name Code Description Making Meaning from Texts 1 MM1 Students respond to texts Making Meaning from Texts 2 MM2 Students interpret and analyze texts Making Meaning from Texts 3 MM3 Students put texts into context

Making Meaning from Texts 4 MM4 Students reflect on and evaluate processes used to make meaning from texts

Making Meaning from Texts 5 MM5 Students work with others Creating and Presenting Texts 1 CP1 Students use their own/other voices Creating and Presenting Texts 2 CP2 Students develop and present texts Creating and Presenting Texts 3 CP3 Students demonstrate technical command of language

Creating and Presenting Texts 4 CP4 Students reflect on and evaluate how texts are created and presented

Creating and Presenting Texts 5 CP5 Students work with others Practice and Drills 1 PD1 Students engage in textbook/workbook written exercises

Practice and Drills 2 PD2 Students participate in oral drills to memorize grammar rules, definitions, etc.

Practice and Drills 3 PD3 Students use mnemonic devices to memorize terms, procedures, etc.

Practice and Drills 4 PD4 Students are engaged in writing practice activities, such as drafting or revising written work

Practice and Drills 5 PD5 Students are engage in reading practice activities, such as assigned or student-selected text

Practice and Drills 6 PD6 Other drill or practice activity



Table 8: Math Instructional Strategies Codes Name Code Description

Problem Solving 1 PS1

Students learn, investigate, and practice mathematical concepts in problem solving contexts that require application, evaluation, or analysis

Problem Solving 2 PS2 Students apply mathematics to “real world” situations

Problem Solving 3 PS3 Students attempt multiple strategies; discuss alternative strategies or solutions

Problem Solving 4 PS4

Students discuss or write about the problem solving process globally or for a specific problem; set goals for improving the process

Reasoning and Proof 1 RP1

Students discuss or write about the necessities of reasoning and proof; ask questions related to reasoning and proof for a specific problem

Reasoning and Proof 2 RP2 Students discuss or write about mathematical possibilities; make a plan to explore possibilities

Reasoning and Proof 3 RP3 Students present, discuss, or write about arguments and proofs; seek and cite support or refutation in resources

Reasoning and Proof 4 RP4 Students attempt multiple methods; discuss alternative types or methods

Communication 1 CM1 Students discuss or write mathematical explanations and questions

Communication 2 CM2 Students clearly and articulately discuss or write about mathematics

Communication 3 CM3 Students consider, question, and criticize others’ mathematics in discussion or writing

Communication 4 CM4 Student use appropriate terminology in speech and writing

Connections 1 CN1

Students describe how a previously learned mathematical concept relates to a new situation; apply a learned concept to a new problem

Connections 2 CN2

Students describe the relationship among mathematical concepts; apply multiple mathematical concepts to solve a multi-step problem

Connections 3 CN3 Students identify and describe situations in which mathematics is useful; apply mathematics in these situations

Representation 1 RE1 Students use numbers, symbols, pictures, or models in note taking and problem solving

Representation 2 RE2

Students choose and use appropriate representations for different situations; interpret and explain numbers, symbols, pictures, or models

Representation 3 RE3 Students express information from across content areas with numbers, symbols, pictures, or models

Lower Order 1 LO1 Students listen passively Lower Order 2 LO2 Students read or copy instructional material or take notes

Lower Order 3 LO3 Students practice skills and techniques in classwork or homework

Lower Order 4 LO4 Students sit during attendance, copy homework assignments, or read from a novel



These codes listed above are the strategies that are at the core of the materials studied in that they are desired learning modes within a classroom conducted with the curriculum materials by a teacher trained in the techniques. These codes are different between English and math classes, and any activity can display multiple strategies or even none of the strategies. For these codes it is worthwhile to note that in terms of our coding we did not code the intention of the teacher with respect to these strategies, but rather the observed actions of the students. For example, if an English teacher asks for an interpretation of a poetic passage, one could assume that the teacher’s intention is that students would conduct some analysis of the text (MM3). But if no students respond and the activity turns into the teacher telling the students the meaning of the poem, then that activity would not receive a MM3 coding, despite the intentions of the teacher. On the other hand, not all students have to engage in the strategy for the activity to receive that code. In the previous example, if at least one thoughtful comment was delivered by a student, then the activity would receive that MM3 code (the low student participation in this example would be handled by the Student Engagement codes). Lastly, AIR added a set of “lower order” codes to each of the sets of ELA and math content codes (PD codes for English and LO codes for math). These are codes that identify rote work and allow us to account for activities that do not engage the content strategies in the client's materials.

Materials Used Codes

The Materials Used codes allow for an accounting of all materials used during each activity. The Materials Used codes for both English and math are listed below in Table 9 and Table 10, respectively. We attempted to include all typical items used during an English or math class, though the coders could of course write in other materials not listed.

Table 9: English Materials Used Name Code Description (English) Materials Used 1 MU1 Program materials (English) Materials Used 2 MU2 Computers (English) Materials Used 3 MU3 Fiction (English) Materials Used 4 MU4 Nonfiction (English) Materials Used 5 MU5 Poetry (English) Materials Used 6 MU6 Drama (English) Materials Used 7 MU7 Film (English) Materials Used 8 MU8 Word/vocabulary materials (non-program) (English) Materials Used 9 MU9 Worksheets/workbooks (non-program) (English) Materials Used 10 MU10 Other (please describe)



Table 10: Math Materials Used Name Code Description (Math) Materials Used 1 MU1 Program texts (Math) Materials Used 2 MU2 Textbooks (Math) Materials Used 3 MU3 Worksheets/workbooks (Math) Materials Used 4 MU4 Calculators (Math) Materials Used 5 MU5 Tools (ruler, compass, etc.) (Math) Materials Used 6 MU6 Computers (Math) Materials Used 7 MU7 Manipulatives (Math) Materials Used 8 MU8 Other (please describe)

Results

The next two sections give selected results in order to provide examples of the use of this form to date. While not nearly as comprehensive as what the client received due to the different scope of this paper, the results presented will still provide a general overview of the trends we found in our observations and our treatment of the data. A summary of the general results follows the selected results discussions.

Selected Class Description and Behaviors Results

Table 11 shows the mean number of activities for four types of classes: the English classes using the English curriculum package, the control English classes, the math classes using the math curriculum package, and the control math classes. In the tables that follow, all of the results are shown using these four categories. Throughout these tables the English and math curriculum packages being studied are referred to as "English Program" and "Math Program," respectively.

Table 11: Mean Number of Activities and Duration of Activities in Minutes

English Program Classes

English Com-parisons

Math Program Classes

Math Com-parisons

Total Program Classes

Total Com-parisons Total

Mean Number of Activities 7 5 12 9 9 7 8 Mean Duration of Activities in Minutes 8.6 13.0 5.5 6.5 6.9 8.8 7.5 Mean Duration of Classes in Minutes 78 75 73 79 76 77 76

The next two tables show examples of results in categories where the differences between the treatment and control classes were not large. First, in terms of the physical organization of activities, we found that almost all activities observed fell into one of three



categories: whole class activities led by the teacher, small group or pair work, and individual work, with the largest difference being the greater use of small group work in the math classes using the program materials.

Table 12: Percentages of Activity Organizations

Activity Organization




Math Com-parisons



Teacher-led whole class activity 48% 40% 50% 51% 49% 45% 48% Student-led whole class activity 4% 9% 4% 2% 4% 6% 4% Small group or pair cooperative activity 15% 25% 31% 18% 22% 21% 22% Independent activity 27% 22% 14% 23% 21% 22% 21% Other organization 6% 4% 1% 6% 4% 5% 4%

Related to the organization of the class, we also undertook to show the nature of the students' overall interaction pattern per activity, where as one would expect based on the first table the largest difference is in the higher percentage of interactive group work for the classes using the math program materials.

Table 13: Percentages of Student Focus

Student Focus




Math Com-parisons



Whole class passive 18% 19% 8% 22% 14% 20% 16% Whole class interactive 28% 21% 44% 36% 35% 28% 33% Student or group presentation to whole class 4% 8% 5% 3% 4% 6% 5% Individual reading or work 30% 23% 7% 17% 20% 21% 20% Pair work 2% 2% 1% 5% 1% 3% 2% Interactive group work of 3-6 on a single project 13% 22% 34% 11% 23% 17% 20% Other student focus 5% 4% 3% 5% 4% 5% 4%

The next two tables deal more with the reactions of students, and in these tables the differences are more marked. The actions coded below address the short term behavior and



apparent engagement of the participants in the class. In a sense, the actions coded below could be considered short-term classroom outcomes. In other words, one would expect that if the classroom organization, learning materials, and teaching strategies are engaging and effective, that the students and teacher would exhibit behaviors that reflect their engagement. For our observations we attempted to codify this in terms of the students in two ways. First, we made a measurement of general student behavior for each activity. The scale ranged from no significant classroom disruptions, to a small number of classroom disruptions (those kind that can be solved by a few words from the teacher), to a small number of students persistently and significantly disruptive (wherein the teacher has to use more time and sharper measures if the actions of one or a few students are to be addressed), to a large number of students significantly disruptive throughout the activity. The results are displayed below.

Table 14: Student Behavior – Percentage of Time Observed




Math Com-parisons



No significant classroom disruptions 70% 47% 84% 60% 76% 53% 68% Small number of disruptions from multiple sources 24% 48% 15% 20% 20% 35% 26% Small number of students persistently disruptive 5% 0% 1% 7% 3% 3% 3% Large number of students significantly disruptive 0% 4% 0% 13% 0% 8% 3%

Beyond simple acts of disruption, we wished to attempt to measure whether or not those students who were not disruptive were engaged in the work or not. For this we created the Student Engagement set of codes, which is essentially a measurement of the percentage of students that appeared to be on-task for the majority of the duration of the activity. The table below shows the results. Just as in the table above, there seem to be clear differences between the treatment and control classes, with both the English and math classes that use the curriculum packages faring better in terms of student behavior and apparent engagement by substantial margins.



Table 15: Student Engagement – Percentage of Time Observed




Math Com-parisons



91-100% students on-task for majority of activity 21% 4% 61% 48% 39% 25% 34% 81-90% students on-task for majority of activity 42% 39% 25% 2% 34% 22% 30% 61-80% students on-task for majority of activity 19% 29% 13% 13% 16% 21% 18% 41-60% students on-task for majority of activity 12% 10% 0% 29% 6% 19% 11% 21-40% students on-task for majority of activity 7% 17% 0% 4% 4% 11% 7% 0-20% students on-task for majority of activity 0% 0% 0% 4% 0% 2% 1%

Next, we turn to a sampling of the results for the instructional strategy codes.

Selected Instructional Strategies Results

The strategies coded during classroom observations are those instructional strategies that the client's materials and training are intended to promote. They are strategies that focus on fostering complex and creatively analytic thought processes on the part of students, and as such represent what might typically be considered a higher order of classroom interaction and learning. The following tables list the percentage of activities and percentage of class time the activities represent for which various strategies were employed. These codes cover the strategies of learning that are key to the programs as well as sets of “lower order” codes created by AIR in order to be able to code other, non-program learning strategies employed in the classroom. The first three tables show some of the results of the English class observations, and the next three tables show some of the results of the math class observations.

The Making Meaning codes include strategies that students might employ to take mental ownership of written material (Table 16). The Creating and Presenting Texts strategies pertain to encouraging students to generate their own written materials (Table 17). Practice and Drills refers to activities that require more rote work that typically would not require creative or involved responses or actions on the part of the students (Table 18). It is in this last table that the most differences are seen between treatment and control classes, in that the control classes spent a larger percentage of class time in various lower order work modes.



Table 16: Percentages of Making Meaning from Texts Use English Program Classes

English Comparisons English Total

Making Meaning from Texts Strategies

% of Activities in which Code is Used

% of Obser-vation Time Activities Represent





Students respond to texts 28% 27% 16% 18% 25% 24% Students interpret and analyze texts 30% 35% 28% 27% 29% 32% Students put texts into context 21% 21% 30% 37% 23% 27% Students reflect on and evaluate processes to make meaning 2% 5% 10% 10% 4% 7% Students work with others 7% 12% 5% 10% 6% 11%

Table 17: Percentages of Creating and Presenting Texts Use English Program Classes


Creating and Presenting Texts Strategies







Students use their own voices 13% 17% 10% 12% 12% 15% Students develop and present texts 7% 11% 7% 5% 7% 9% Students demonstrate technical command of language 3% 3% 7% 4% 4% 3% Students reflect on and evaluate how texts are created 7% 4% 18% 13% 10% 7% Students work with others 7% 12% 3% 2% 6% 8%



Table 18: Percentages of Practice and Drills Use English Program Classes


Practice and Drills Strategies







Students engage in textbooks/ workbook written exercises 2% 4% 7% 11% 3% 7% Students participate in oral drills to memorize rules/definitions 1% 2% 2% 3% 1% 2% Students use mnemonic devices to memorize terms/procedures 0% 0% 0% 0% 0% 0% Students engage in writing practice activities 5% 8% 11% 15% 7% 10% Students engage in reading practice activities 5% 10% 7% 21% 5% 14% Other drill or practice activity 3% 3% 3% 8% 3% 5%

The math categories displayed below consist of Problem Solving (Table 19), Communication (Table 20), and a set of Lower Order codes for routine work (Table 21). In all three tables substantial differences can be seen between the treatment and control sets of classes, in that the class using the math curriculum being studied had far higher percentages of activities and class time in which the more challenging instructional strategies are used, and lower percentages of the lower order work modes.

Table 19: Percentages of Problem Solving Use Math Program Classes Math Comparisons Math Total

Problem Solving Strategies







Students learn, investigate, and practice math concepts 19% 30% 6% 5% 15% 20% Students apply mathematics to ”real world” situations 13% 13% 2% 2% 9% 9% Students attempt multiple strategies 5% 8% 2% 1% 4% 5% Students discuss or write about problem solving process 2% 5% 0% 0% 1% 3%



Table 20: Percentages of Communication Use Math Program Classes Math Comparisons Math Total

Communication Strategies







Students discuss or write math explanations and questions 30% 38% 27% 27% 29% 34% Students clearly and articulately discuss or write about math 24% 33% 11% 10% 20% 24% Students consider, question and criticize others’ math 6% 11% 1% 1% 4% 7% Students use appropriate terminology 4% 5% 3% 6% 3% 6%

Table 21: Percentages of Lower Order Use Math Program Classes Math Comparisons Math Total

Lower Order Strategies







Students listen passively 19% 15% 29% 22% 22% 17% Students read or copy instructional material 10% 6% 23% 19% 14% 11% Students practice skills in classwork or homework 13% 17% 27% 35% 18% 24% Students sit or read non-math materials or copy homework 13% 15% 22% 34% 16% 22%

Summary of Findings

The tables above, together with the remainder of the observation data, produced a number of findings. The data revealed that both the treatment English and treatment mathematics teachers incorporated more instructional activities into their classes than did teachers in the comparison group. Moreover, the nature of these instructional activities were different between the treatment and comparison classes, although this is more true for mathematics than English. The mathematics classes showed greatly increased uses of certain



types of groupings, more interactive assignments, greater frequencies of higher order strategies, and more varied materials used. The English classes showed a different picture, in which the materials weren’t used as much and the results were less dramatic in the instructional strategy areas but still high in the student outcome areas. Most strikingly, treatment classes for both English and mathematics appear to have had much higher levels of student engagement and teacher time devoted to instruction and lower levels of disruptive student behavior.

Conclusion

Benefits of Design

We believe that this observation form represents a useful tool in general for researchers examining curriculum and school reform. The benefits of this observation form stem from the desire to produce easily assimilated and quantifiable output from the observations while still addressing the complexities of the classroom experience. In this case, the activity as an analysis unit provides the flexibility necessary when examining the interactions within a classroom while still giving a frame that can be used to construct the simple descriptive charts given above.

To date, we see the following benefits:

• A frame that reflects the natural division of action in a classroom

• The creation of units of analysis from the activity structure

• A coding system that provides comprehensive flexibility

• The ability to records actions and responses in a manner that can be used for meaningful reporting

• Narrative and categorical capabilities

Future Adaptations

We also see possibilities for expansion and improvement on this beginning. Depending on the needs of the study at hand, here are some of the adaptations we've considered to date:

Correlations at the individual teacher level. Based on the needs of the project described in this paper we focused strictly on the correlations of program use versus non-use with respect to the desired behaviors and strategies. With this instrument, though, the possibility clearly exists to examine correlations at other grouping levels, i.e. the teacher or school level, in order to ferret out other factors of possible influence based on differences of implementation between teachers or schools, particularly where a larger sample size is used.



Coding with a rating scale for quality. The coding system we used for this project is at the yes or no level in that we marked whether or not a certain behavior or strategy appeared within an activity. Depending on the project, though, one may want to look at an even finer grain of distinction by employing a scale that describes the quality of the event, for example on a 1 to 5 coded scale, and not just whether or not the event occurred.

Coding with a rating scale for duration. In the project discussed here, we described all timeframes at the level of activity, with the explicit understanding that coded events within the activity did not necessarily occur for the duration of the activity time. One may choose, however, to be more detailed about the elapsed time of coded events. One possible way to do this would be to code using a scale for approximate percentage of activity time. This scale would give a range that would tell the percentage of the activity during which the coded activity occurred.

Coding with a rating scale for participation numbers. We used the Student Engagement codes to relate how many students stayed on-task for the activity, but observers may wish to be more explicit about the numbers of students staying on-task for every instructional strategy or other event listed in the activity.

Parsing activities at the level of seconds. While probably not needed for most applications, one could choose to record duration of activities down to the second, as opposed to the minute.

Utilizing a qualitative software package. Many of the changes above involve looking at finer and more levels of distinction. Though not a change in the observation instrument itself, the use of a qualitative program could facilitate the manipulation and analysis of the data where more layers of complexity are added.

Certainly this list is not exhaustive. We feel that the framework we've laid out, however, allows for these and a variety of other adaptations without a loss of its basic form and usefulness. With time, future projects, and the insight of others, we look forward to expanding, adapting, and improving this instrument as a means towards channeling the complex dynamics of the classroom into useful and meaningful stories told by data.


References

Edmonds, M., & Briggs, K.L. (2003). The instructional content emphasis instrument: Observations of reading instruction. In S. Vaughn & K. Briggs (Eds.), Reading in the classroom: Systems for the observation of teaching and learning (pp. 31-52). Baltimore: Paul H. Brookes Publishing.

Erickson, F. (1986). Qualitative methods in research on teaching. In M.C. Wittrock (Ed.), Handbook of research on teaching (3rd ed.). New York: MacMillan Publishing.

Evertson, C.M., & Green, J.L. (1986). Observation as inquiry and method. In M.C. Wittrock (Ed.), Handbook of research on teaching (3rd ed.). New York: MacMillan Publishing.

Foorman, B.R., & Schatschneider, C. (2003). Measurement of teaching practices during reading/language arts instruction and its relationship to student achievement. In S. Vaughn & K. Briggs (Eds.), Reading in the classroom: Systems for the observation of teaching and learning (pp. 83-109). Baltimore: Paul H. Brookes Publishing.

Kim, A., Briggs, K.L., & Vaughn, S. (2003). The classroom climate scale: Observing during reading instruction. In S. Vaughn & K. Briggs (Eds.), Reading in the classroom: Systems for the observation of teaching and learning (pp. 83-109). Baltimore: Paul H. Brookes Publishing.

Shavelson, R.J., Webb, N.M., & Burstein, L. (1986). Measurement of teaching. In M.C. Wittrock (Ed.), Handbook of research on teaching (3rd ed.). New York: MacMillan Publishing.

Spradley, J.P. (1980). Participant observation. New York: Holt, Rinehart and Winston.

Taylor, B.M., & Pearson, D.P. (2000). The CIERA school change classroom observation scheme. Retrieved March 26, 2004, from University of Minnesota, Curriculum & Instruction Web site: http://education.umn.edu/CI/taylor/Files/CIERAScheme.pdf

24 American Institutes for Research

Appendix A. Demographic Information Capture Sheet

District: Number Students:

School: Number Female:

Teacher: Number Male:

Observer: Grade Mix:

Obs Date: Grade Level:

Program: Ability Mix:

Level: American Indian:

Class Name: Asian:

Class Subject: Black:

Class Level: Hispanic:

Class Length (Mins.): White:

Periods per Week: Other:

Obs Start: Obs End: Texts:

Handouts:

Other Artifacts:


Appendix B. Narrative Log Sheet

Time/Act. # Observation Narrative Page # _____


Appendix C. Organization and Engagement Coding Sheet

Org. Codes 1 2 3 4 Code 5 6 7 8 CodeAO1 Teacher-led whole class AO1 AO1AO2 Student-led whole class AO2 AO2AO3 Small group or pair cooperative AO3 AO3AO4 Independent activity AO4 AO4A

ctiv

ity

Org

aniz

atio

n

AO5 Other (please describe) AO5 AO5TA1 Presenting information TA1 TA1TA2 Leading student work TA2 TA2TA3 Supporting student work TA3 TA3TA4 Providing feedback for students TA4 TA4TA5 Evaluating progress TA5 TA5TA6 Non-instruction class duties TA6 TA6TA7 Non-instruction non-class activities TA7 TA7T

each

er A

ctiv

ities

TA8 Other (please describe) TA8 TA8SF1 Whole class passive SF1 SF1 SF2 Whole class interactive SF2 SF2 SF3 Student or group presentation SF3 SF3 SF4 Individual reading or work SF4 SF4 SF5 Pair work SF5 SF5 SF6 Interactive group work on a project SF6 SF6 St

uden

t Foc

us

SF7 Other student focus SF7 SF7 SB1 No significant classroom disruptions SB1 SB1 SB2 Small disruptions from multiple sources SB2 SB2 SB3 Small number persistently disruptive SB3 SB3 SB4 Large number disruptive throughout SB4 SB4 St

uden

t B

ehav

ior

SB5 Other behavior issues SB5 SB5 SE1 0-20% exhibiting on-task behavior SE1 SE1 SE2 21-40% exhibiting on-task behavior SE2 SE2 SE3 41-60% exhibiting on-task behavior SE3 SE3 SE4 61-80% exhibiting on-task behavior SE4 SE4 SE5 81-90% exhibiting on-task behavior SE5 SE5 St

uden

t E

ngag

emen

t

SE6 91-100% exhibiting on-task behavior SE6 SE6 TT1 0-20% teacher's time related to activity TT1 TT1 TT2 21-40% teacher's time related to activity TT2 TT2 TT3 41-60% teacher's time related to activity TT3 TT3 TT4 61-80% teacher's time related to activity TT4 TT4 TT5 81-90% teacher's time related to activity TT5 TT5

Tea

cher

Tim

e

TT6 91-100% teacher's time related to activity TT6 TT6 RD1 No perceived racial/ethnic differences RD1 RD1RD2 One or more groups treated differently RD2 RD2RD3 One or more groups respond differently RD3 RD3R

acia

l D

iffs.

RD4 Other racial/ethnic difference observed RD4 RD4GD1 No perceived gender differences GD1 GD1GD2 Genders treated differently GD2 GD2GD3 Genders respond differently GD3 GD3Gen

der

Diff

s.

GD4 Other gender difference observed GD4 GD4


Appendix D. English Instructional Strategies Coding Sheet

English Codes 1 2 3 4 Code 5 6 7 8 Code

MM1 Students respond to texts MM1 MM1

MM2 Students interpret and analyze texts MM2 MM2

MM3 Students put texts into context MM3 MM3

MM4 Students reflect on and evaluate processes used to make meaning from texts MM4 MM4

Mak

ing

Mea

ning

from

T

exts

MM5 Students work with others MM5 MM5

CP1 Students use their own/other voices CP1 CP1

CP2 Students develop and present texts CP2 CP2

CP3 Students demonstrate technical command of language CP3 CP3

CP4 Students reflect on and evaluate how texts are created and presented CP4 CP4

Cre

atin

g an

d Pr

esen

ting

Tex

ts

CP5 Students work with others CP5 CP5

PD1 Students engage in textbook/workbook written exercises PD1 PD1

PD2 Students participate in oral drills to memorize grammar rules, definitions, etc. PD2 PD2

PD3 Students use mnemonic devices to memorize terms, procedures, etc. PD3 PD3

PD4 Engaged in writing practice activities, such as drafting or revising written work PD4 PD4

PD5 Students are engage in reading practice activities, such as assigned or student-selected text PD5 PD5 Pr

actic

e an

d D

rills

PD6 Other drill or practice activity PD6 PD6

MU1 Program materials MU1 MU1

MU2 Computers MU2 MU2

MU3 Fiction MU3 MU3

MU4 Nonfiction MU4 MU4

MU5 Poetry MU5 MU5

MU6 Drama MU6 MU6

MU7 Film MU7 MU7

MU8 Word/vocabulary materials (non-program) MU8 MU8

MU9 Worksheets/workbooks (non-program) MU9 MU9

Mat

eria

ls U

sed

MU10 Other (please describe) MU10 MU10


Appendix E. Math Instructional Strategies Coding Sheet

Math Codes 1 2 3 4 Code 5 6 7 8 CodePS1 Students learn and practice math concepts in problem

solving contexts requiring application or analysis PS1 PS1

PS2 Students apply mathematics to “real world” situations PS2 PS2

PS3 Students attempt multiple strategies; discuss alternative strategies or solutions PS3 PS3

Prob

lem

Sol

ving

PS4 Students discuss or write about the problem solving process globally or for a specific problem; set goals PS4 PS4

RP1 Students discuss or write about the necessities of reasoning and proof; ask questions related to reasoning and proof for a specific problem

RP1

RP1

RP2 Students discuss or write about mathematical possibilities; make a plan to explore possibilities RP2 RP2

RP3 Students present, discuss, or write about arguments and proofs; seek and cite support or refutation RP3 RP3

Rea

soni

ng a

nd

Proo

f

RP4 Students attempt multiple methods; discuss alternative types or methods RP4 RP4

CM1 Students discuss or write mathematical explanations and questions CM1 CM1

CM2 Students clearly and articulately discuss or write about mathematics CM2 CM2

CM3 Students consider, question, and criticize others’ mathematics in discussion or writing CM3 CM3

Com

mun

icat

ion

CM4 Students use appropriate terminology in speech and writing CM4 CM4

CN1 Students describe how a previously learned mathematical concept relates to a new situation CN1 CN1

CN2 Students describe the relationship among math concepts; apply multiple math concepts CN2 CN2

Con

nect

ions

CN3 Students identify and describe situations in which math is useful; apply math in these situations CN3 CN3

RE1 Students use numbers, symbols, pictures, or models in note taking and problem solving RE1 RE1

RE2 Choose appropriate representations for situations; interpret and explain numbers, symbols, pictures RE2 RE2

Rep

rese

n-ta

tion

RE3 Students express information from across content areas with numbers, symbols, pictures, or models RE3 RE3

LO1 Students listen passively LO1 LO1

LO2 Students read or copy instructional material or take notes LO2 LO2

LO3 Students practice skills and techniques in classwork or homework LO3 LO3

Low

er O

rder

LO4 Students sit during attendance, copy homework assignments, or read from a novel LO4 LO4

MU1 Program texts MU1 MU1MU2 Textbooks MU2 MU2MU3 Worksheets/workbooks MU3 MU3MU4 Calculators MU4 MU4MU5 Tools (ruler, compass, etc.) MU5 MU5MU6 Computers MU6 MU6MU7 Manipulatives MU7 MU7

Mat

eria

ls U

sed

MU8 Other (please describe) MU8 MU8


Appendix E. Mathematics Strategies Coding Sheet


researching education as it happens using classroom ... · pdf fileusing classroom...

Documents