© THE EVALUATION OF TEACHING EFFECTIVENESS AND TEACHER EDUCATION

N . L . Gage

шгшшшЕ mm

ПЕР seminar paper: (5 J

THE EVALUATION OF TEACHING EFFECTIVENESS AND TEACHER EDUCATION

N . L . Gage

A contribution to the HEP Seminar on "The evaluation of the qualitative aspects of education" 30 September - 4 October 1974

INTERNATIONAL INSTITUTE FOR EDUCATIONAL PLANNING

(established by Unesco) 7-9, rue Eugène-Delacroix, 75016 Paris

0 Unesco 1975

The authors are responsible for the choice and the presentation of the facts contained in these papers and for the opinions expressed therein, which are not necessarily those of the Institute and do not engage the responsibility of Unesco.

CONTENTS

THE OBJECTIVES OF PROJECTS IN THE AREA OF TEACHING

COMPONENTS OF PROGRAMMES FOR THE EDUCATION OF TEACHERS

THE RESEARCH BASIS FOR THE EVALUATION OF TEACHING EFFECTIVENESS AND TEACHER EDUCATION

CONCLUSION

- 1 -

THE EVALUATION OF TEACHING EFFECTIVENESS AND

TEACHER EDUCATION

The topics considered in this paper are significant because of the

importance of teachers (here used to denote part-time teachers and vocational

education instructors as well as other kinds) in the educational process.

This paper deals first with the character of objectives in projects con­

cerned with teaching effectiveness and teacher education. Then it deals

with the five major categories of subject matter and objectives that enter

into teacher education. Finally, it deals with the general character of

research in these areas and its significance for future work in the evalua­

tion of teacher effectiveness and teacher education.

THE OBJECTIVES OF PROJECTS IN THE

AREA OF TEACHING

The objectives of the projects concerning teaching fall into

the following main categories :

1. Increasing teacher effectiveness in general and in an

unspecified way ;

2. Increasing the knowledge and skills of teachers ;

3. Improving the instructional practices of teachers ;

4. Improving the effectiveness of teachers as indicated by the

achievement of their pupils.

Objectives in the first category - increasing teacher effectiveness

in general - can be illustrated by the following kinds of phrases, quoted

from the abstracts of the objectives of projects in various countries;

"Train instructors needed to expand and improve primary teacher

training throughout the country."

"Improvement and strengthening of teacher training."

Objectives referring to attempts to increase the knowledge and

skills of teachers can be illustrated by the following:

"The training college expansion is to prepare more teachers for

science teaching in primary schools."

"The training would also be designed to prepare teachers to become

village leaders and would therefore include courses in community

development techniques, school guidance and administration."

- 2 -

"Teacher-training programmes in a proposed teacher-training

college which would be equipped to offer courses such as

industrial arts, agriculture, general science and home

economics."

Objectives that refer to the improvement of instructional

practices of teachers can be illustrated by the following :

"... staffing arrangements and teaching methods envisaged offer

resaonable assurance of achieving objectives sought by the

schools."

"To assist and upgrade teachers and keep them abreast of curriculum

development, classroom instruction will be supplemented by ETV. "

The basic idea that the effectiveness of teachers ought to be

reflected in the achievement of their pupils is taken into account in

statements of objectives that refer to such achievement as an objective

to be attained through improved training of teachers'. Such objectives are

indicated in the following :

"The quality of primary school output improved by raising teachers*

standards should provide a better secondary school input."

"Other modifications have been designed to improve the training

programmes and to extend their impact."

"This (government teacher-training college) would contribute to an

improvement of the instructional programme and the quality of the

output from primary schools."

In all their generality and brevity, these statements of objectives

in the area of teaching must be considerably supplemented before they can

further achievement of the goals set forth. These statements provide general

and ultimate goals that need to be explicated in the form of much more

specific kinds of teacher knowledge, understanding, decision-making ability,

and skills in specific kinds of behaviour and performance.

COMPONENTS OF PROGRAMMES FOR THE

EDUCATION OF TEACHERS

A teacher education programme is often regarded as including five

kinds of components :

1. The general education of teachers ;

2. The teacher's education in the knowledge of the subject he is

going to teach ;

- 3 -

j5. The foundations of education - social, psychological, philoso­

phical, and historical ;

4. The theory of curriculum and instruction in the field in which

one is going to teach ;

5. Ability to perform the decision-making and technical skills of teaching.

The general education of teachers

The general education of teachers is the same as that of any citizen

who has received a higher education. It provides him with the knowledge

and understanding of the major domains of human knowledge in the natural

sciences, both physical and biological, the social sciences, and the

humanities. It is often regarded as valuable in itself and as something

that infuses human activity in any field with greater wisdon. Just as

physicists and artists, physicians and lawyers, housewives, cobblers and

farmers, ought to have a certain general education, so should teachers.

The teacher's knowledge of his subject matter

The teacher's knowledge of his subject matter is generally assumed

to be essential to his ability to teach that subject. At least, laymen

tend to make this assumption. The importance of the teacher's knowledge of

the subject matter may vary considerably at different grade levels and in

different subject matters. It also may depend heavily on the kinds of

instructional methods that are used, being more important perhaps when

the lecture method is relied upon and less important when programmed

instruction or independent study is involved. In any case, this aspect of

teacher-education programmes must be taken into account in describing and

evaluating projects in teacher education.

Thus, in some investigations the teachers' own understanding of

what they teach has been investigated. One researcher, for example,

investigated teachers' understanding of arithmetic processes and concepts.

Some teachers displayed such ignorance as to throw doubt on their ability

to produce understanding on the part of their pupils. Other investigations of

of teachers' knowledge of their own subject have revealed significant variance

among physics teachers and among teachers of secondary-school mathematics in

their knowledge of the subject they were teaching. Although most educators

would incline to hypothesize that the more the teacher knows and understands

about the subject, the better his effects on his students will be ; this

hypothesis has not been supported by the dozen or so correlational studies

conducted so far. But these studies have compared more and less knowledgeable

- 4 -

teachers. No one has compared teachers who have no knowledge of the subject

with those who do. Hence the basic assumption that teachers need knowledge of

their subject remains unchallenged.

The foundations of education

The foundations of education are the fields of knowledge that provide

the professional teacher and educator with a grasp of the sociological,

psychological, philosophical, and historical concepts, principles, and

issues that have characterized man's concern with education over the centuries.

The sociological foundations of education deal with such matters as the

structure and functions of large and small social entities, ranging from

whole societies to small groups. The psychological foundations of education

deal with the nature of learning and the learner - his cognitive and social

development, his motivation as a state of the moment and as an enduring trait

the learning process, and the nature of achievement and its measurement and

evaluation. The philosophical foundations of education deal with relationship

between curriculum and values, with the theory of knowledge underlying

the curriculum, and with the proper relationship of education to the

larger society and the good life. The historical foundations of education

draw upon human experience down through the ages in acculturating the

young so that they become useful, productive, and integrated members of

the society into which they are born.

Many teachers at the beginning of their careers in the field of

education have low regard for these matters, looking upon them as useless

in their day-to-day interaction with their pupils. But as their experience

accumulates and they begin to appreciate the problems with which they

must deal, they become motivated to acquire a better understanding of the

origins and nature of those problems. Then they become interested in

understanding these 'foundations1 of the enterprise of education. Delaying

the study of these subjects until students have become somewhat experienced

as teachers might make these fields receive higher regard from students

in education.

Theory of curriculum and instruction

The field of curriculum and instruction theory in any given subject-

matter field is one that deals with the decisions about what should be

taught and how it should be taught. The factors that should enter into

making-decisions about what should be taught are studied here. Similarly

the psychological factors - concepts and principles from the psychology

- 5 -

of learning - are studied in the effort to improve the efficiency of

teaching. Here, the teacher should learn about the ways in which charac­

teristics of the student (his abilities and motives) should interact with

the character of the educational objectives (cognitive, social-emotional,

or psychomotor) and the various kinds of teaching strategies or styles

that are available (the lecture method, the discussion method, role-playing,

individual study, self-directed learning, humanistic methods, etc.).

Studies have shown that most teachers persistently use a single method

of instruction, the classroom recitation, regardless of the objectives

being sought, the characteristics of their students, or the superior

suitability of other methods of instruction. Teachers need to know about

alternative methods, to understand their advantages and limitations, to

have some basis for choosing among methods according to their suitability

in various situations, and to know how to put them into effect once the

decision to use such methods has been made. Then teachers will not fall

into the rut that has been revealed in observational studies of classroom

teaching made since the early 1900s.

The technical skills of teaching

Finally, we come to that part of the teacher-education programme

that is intended to provide the teacher with knowledge of 'how' to

perform various technical skills and decision-making processes in the

interactive phases of teaching. It is in these phases that the teacher

meets his students face to face. Traditionally, 'student teaching', as it is

called in England is used as the main avenue whereby a beginning teacher

acquires these skills.

Microteaching and minicourses. During the last decade, some new approaches

have appeared -'microteaching1 being especially noteworthy. Here, the

beginning teacher has an opportunity to acquire these skills in scaled-down

situations, working with a small number of pupils (say 5) for a relatively

short period (say 10 minutes), on a restricted part of the teaching process

(say 'making an assignment1 or 'giving an explanation').

The microteaching approach has been widely adopted in the USA

and has also been used to some extent in several other countries, including

Scotland, Sweden, Australia, the Federal Republic of Germany, Israel, and

some African countries. It has also provided the basis for 'minicourses',

which are transportable, self-administrable, self-contained packages of

teacher-training products (including films, videotapes, manuals, tests,

- 6 -

and printed instructions) designed to improve specific technical skills on

the part of teachers. Among the skills for which minicourses have been

developed are those related to the discussion method of teaching, techniques

of developing language skills, effective questioning in a classroom

discussion, tutoring in mathematics, and organizing the kindergarten for

independent learning and small-group instruction.

Thus far, minicourses have been demonstrated to be effective

primarily in the sense of changing teacher behaviour rather than in

the more important sense of improving student performance, behaviour,

or achievement. But, of course, effect on teacher behaviour is a prere­

quisite of effectiveness in improving student achievement. An impressive

three-year follow-up has demonstrated the effectiveness of a minicourse

as a vehicle for changing teacher behaviour. It was found that videotape

recording of each of 24 experimental teachers made before, immediately

after, four months after, and 39 months after training, revealed that

the teachers were significantly above their precourse level on 8 of the 10

behaviours scored even J59 months after their training. Thus, in improving

ability to redirect questions, prompt, clarify, avoid repeating questions,

avoid repeating pupil answers, elicit longer pupil responses, reduce

frequency of one-word pupil responses, ask higher-order questions (those

requiring thinking), and reduce teacher talk - in improving all these

kinds of teacher performance - the minicourse was found to be effective.

Teacher-training products. Many teacher-training products have been

developed in addition to the relatively expensive and elaborate minicourses,

which cost approximately $1,200 per course in their present form. A recent

collation and analysis brought together more than 600 such teacher-training

products in a computerized catalogue. Each of the products was described

in many different ways, such as the subject matter and grade levels, if any,

for which it was specific, the kinds of teacher behaviour that it was

intended to influence, the kinds of outcomes in student achievement and

behaviour that the teacher behaviour was in turn intended to improve, the

kinds of training situations in which the products were to be used, and

the kinds of facilities needed. Information on the source, availability,

and cost of the product was also furnished.

This computerized catalogue should be available for use in selecting

specific teacher-training products that appear to be appropriate for

the improvement of teacher education in countries other than the U.S.A.,

- 7 -

once appropriate translations and modifications of the products have been

made. The products range in complexity, from simple manuals, simulation

games, film-strips, and programmed instructional textbooks, to more elabo­

rate and expensive devices such as minicourses or complete modularized

programmes suitable for use as entire courses at the University or college

level.

It cannot yet be claimed that the teacher-training product approach

to the improvement of the performance aspects of teacher education has

been proven successful in the sense of raising the level of student achieve­

ment of various kinds of objectives. But the approach appears to be

promising enough to merit serious consideration in the development of

teacher-education programmes in all nations, including the less developed

ones.

The foregoing description of major types of components of teacher-

education programmes goes well beyond the level of specificity to be found

usually in teacher education projects. It would be desirable to have

specifications at an even more detailed level if projects on teaching and

teacher education are to be evaluated. Otherwise, obviously, it is impossible

to determine which aspect of teacher education the project is concerned with.

It is also clear that, if the evaluation of a project is to be at all

adequate, the level of specificity must go considerably beyond that illus­

trated in the foregoing description of major components of teacher-education

programmes. The level of specificity illustrated above is necessary not

merely to facilitate evaluation but, even more, to assist the developers

of projects in formulating their own plans and objectives - that is, in

deciding what they want to do.

THE RESEARCH BASIS FOR THE EVALUATION OF TEACHING

EFFECTIVENESS AND TEACHER EDUCATION

In general, the evaluation of teacher effectiveness and teacher

education follows the same logic and principles as the evaluation of

educational activities or enterprises of any kind. But teacher effecti­

veness and teacher education are involved with specific kinds of educational

variables and activities and refer to relationships between variables of

specific kinds. These are indicated in the diagram presented in Figure 1.

In evaluating teacher effectiveness, one studies the relationships

between the teachers1 behaviours and characteristics, on the one hand,

and the resulting student behaviours and characteristics on the other.

- 8 -

The teacher behaviours may take such forms as giving lectures of certain

kinds, conducting recitations, giving assignments, forming discussion

groups, or testing with greater or less frequency. The teacher's charac­

teristics may consist of his age, sex, amount of experience, or knowledge

of his subject. The student's behaviour and characteristics are those

with which the educational objectives are concerned. Special emphasis must

be placed on determining that the relationship between teacher variables

and student achievement is causal and not merely correlational.

Evaluation of teacher education

Teacher-education procedures and programmes

Evaluation of teacher

effectiveness

r~7 Teacher behaviours

and characteristics

-7

Student behaviours and

characteristics

Figure 1. Types of variables and their relationships involved in the evaluation of teacher effectiveness and teacher education

In evaluating teacher education, one seeks to determine whether a

kind of teacher-education procedure - e.g., courses of various kinds,

selection criteria of various kinds, microteaching, student teaching,

role playing, film viewing or listening to lectures - affects the kinds

of teacher behaviours or characteristics that have been set up as the

objectives of the teacher-education programme.

Research on teacher effectiveness

What should be the objectives of the teacher-education programme ?

In general, these objectives should be the kinds of teacher behaviours

and characteristics that have been found to have desirable effects on

students, i.e. that promote students1 achievement of educational objec­

tives. The nature of such teacher behaviours and characteristics has,

of course, been the subject of philosophising, speculation and debate

for centuries. In recent decades, however, a strong beginning has been

made at providing answers based on scientific method, i.e. on empirical

studies of relationships between teaching behaviours and student achievement.

A generation of research workers has, since the 1950's, begun to amass more

or less consistent findings. Two recent reviews of such findings (Teaching

behaviours and student achievement, I97I5 The study of teaching, 1974) have

- 9 -

behaviour such as 'criticism' or 'acceptance of student ideas' or 'enthusiasm'

to exhibit a positive (or a negative) relationship to student achievement

(adjusted for initial student ability or achievement). Tables 1 and 2 show,

for illustrative purposes, the results of collations of studies of two of

these kinds of teacher behaviour. Table 1 shows that Teacher Acceptance of

Student Ideas has correlated positively with adjusted student achievement in

eight out of nine independent studies. Table 2 shows that Teacher Criticism

and Disapproval has correlated negatively with adjusted student achievement in

Ik studies and positively in only three studies.

Results with this high degree of consistency are at least as good

as the average to be found in the behavioural sciences. Though the studies

yielding these results may have methodological flaws, and though they are

correlational studies that do not permit causal inferences, they provide

a basis for hope that we have begun to identify teacher behaviours that

make a difference.

If teachers were trained in several of these kinds of behaviour that

have been found consistently correlated with student achievement, it is

likely that their effectiveness would be improved. As students experienced

the efforts of such improved teachers over several courses and years, it is

likely that the relationship of teacher variables to achievement would come

closer in magnitude to that of home background variables.

Research on Teacher Education

But now let me turn to the second kind of relationship — that between

teacher education procedures and methods, on the one hand, and teacher

competencies, on the other. How can we know when a teacher education method

is producing a teacher competency, or a desirable kind of teacher behaviour,

or the use of good models or methods of teaching, or "events that support

learning processes"? In short, how can we evaluate the instructional strategies

that are used in teacher education programmes?

'Knowing That' and 'Knowing How'. My answer is based in part on a distinction

between two kinds of competencies. This is the distinction between 'knowing that'

and'knowing how'. The first kind of knowledge, 'knowing that', refers to the

ability to state factual propositions, such as knowing that Skinner coined the

term 'operant' or that teacher education is a controversial subject. 'Knowing

how', on the other hand refers to skills or operations, and the distinguishing

feature of such skills is that they imply learning through practice. One could

not know how to swim or speak French unless one had at some time practised

swimming or tried to speak French.

- 10 -

and the distinguishing feature of such skills is that they imply learning through practice. One could not know how to swim of speak French unless one had at some time practised swimming or tried to speak French.

Table 1. Collation of Studies of Relationships between Teacher Acceptance of Student Ideas and Student Achievement

Source (1)

D-B, R D-B, R D-B, R D-B, R D-B, R D-B D-B, R D-B, R R

Author

Flanders (1970), 2nd grade Flanders (1970), 4th grade Flanders (1970), 6th grade Flanders (1970), 7th grade Flanders (1970), 8th grade Hughes (197З) Perkins (1965) Soar (I966) Wright and Nuthall (1970)

Result

-.45 +.19 +.30 +.40 +.19 + + + .05 +.17

(1) D-B : Dunkln, Michael and Biddle, Bruce. The Study of Teaching New York: Holt, Rinehart and Winston, 1974.

R : Rosenshine, Barak. Teaching Behaviours and Student Achievement; London, England: National Foundation for Educational Research in England and Wales, 1971.

- 11 -

Table 2. Teacher Criticism and Disapproval in Relation to Student Achievement : Collation of studies by Rosenshine (1971) and Dunkin and Biddle (197̂ -)

Source Author Result

R D-B, R D-B D-B, R D-B, R D-B, R D-B, R D-B, R D-B, R D-B, R D-B, R R R D-B, R D-B D-B,R R R R

(1) This coefficient was erroneously recorded as +,48 by Rosenshine in his Table 2.1, but correctly as negative on his p. 59.

Anthony (1967) Cook (1967) Felsenthal (1967) Flanders (1970) 2nd grade Flanders (1970) 4th grade Flanders (1970) 6th grade Flanders (1970) 7th grade Flanders (1970) 8th grade Harris and Serwer (1966) Harris, et al. (1966) Hunter (1968) Marsh (1956) Perkins (1965) Soar (1966) Soar, et al. (1971) Spaulding (1965) Wallen (1966) 1st grade V/allen (1966) 5̂ d grade Wright and Nuthall (1970)

-.48 (1) -.55 7

+,18 -.5^ -.52 -.50 -Л5 +.16 -.26 -.61 +,05 --,19

9

-.25 -.25 -.22 -.50

- 12 -

My purpose in referring to these distinctions is to delimit the

kinds of outcomes or objectives of teacher education with which I shall deal.

In some conceptions, all outcomes are embraced. Such conceptions include the

teacher's general or liberal education, the teacher's knowledge of the subject

he will teach, the teacher's knowledge about curriculum and instruction, the

teacher's knowledge about the social, historical, and philosophical foundations

of education, and finally, the teacher's knowledge of how to teach.

Most of these kinds of knowledge belong in the 'knowledge that' category.

Omitted from this discussion is any consideration of ways of helping teachers

to acquire such knowledge and ways of evaluating techniques of giving teachers

such knowledge. The reason for restricting discussion in this way is twofold:

First, ways of inculcating and evaluating such knowledge of this kind, we have

done about as well or as poorly as the rest of higher education. Second, many

of these kinds of knowledge are provided by parts of the university outside of

schools or colleges of education, and professional educators can impose new

approaches outside their own domains only at the risk of arousing considerable

academic opposition. It is better to try to establish the value of teacher

education in those domains for which professional educators are primarily

responsible. And, of these, the one that provides the greatest room for

improvement and the greatest opportunity is the realm of 'knowledge how' —

the realm of skills and habits in teaching. This is the realm which includes

what the teacher does in making decisions about how to teach and how he then

uses the skills necessary to carry out those decisions. 'Knowing that' is

illustrated by the student's ability to state and even explain Archimedes'

and Newton's laws. 'Knowing how' is illustrated by the student's swimming.

Although swimming is explained by Archimedes' and Newton's laws, knowing

those laws is one thing, while knowing how to swim is another. The two kinds

of knowledge are independent of one another. Similarly, knowing the laws

of learning and being able to expound operant conditioning or cognitive

theory is one thing, but event though those theories deal with what makes

certain kinds of teaching possible, knowledge of those theories is not

sufficient to enable one to teach.

It is this realization that has led teacher educators to include student

teaching, or some other kind of practice in their programmes. Teacher

educators may not have been aware of the distinction between knowing that

and knowing how, or the many subtle elaborations of that distinction that

- 13 -

have been formulated, but they have acted upon that distinction. It may be

that distinction which most clearly lays the basis for the whole concern in

the United States that teacher education be performance based. Teachers

should know how to teach, not merely to know that certain things about

teaching are true. This realization has also led to all the attempts of

the last decade to improve the effects of practice for prospective teachers —

attemps in the form of microteaching, minicourses, and the hundreds of

teacher training products that have been developed in recent years.

How to Evaluate Teacher Training. How should we evaluate the strategies

that we develop to improve the degree to which teachers behave desirably?

In dealing with this topic, we must set up hypotheses about desirable ways

of teaching. Furthermore, we must set up hypotheses about strategies for

getting teachers to behave in these desirable ways. Both of these kinds

of hypotheses have frequently been made in the developmental work of the

last few years.

How should strategies for inculcating teaching competencies be

evaluated? To a considerable degree, the broad outlines of answers to

these questions are already available. In general, we can evaluate such

products or strategies by using them as independent variables, or treatments,

in experiments or quasi-experiments. Teachers or prospective teachers given

training with the procedure or product should be compared with teachers or

prospective teachers not given such training. Designs for such experiments

have been clearly set forth and have recently been treated with specific

reference to the problem of evaluating teacher training products. The

latter exposition focused particularly on three quasi-experimental designs

because of the presumed difficulty, in research with teachers, of obtaining

control groups of teachers. These three quasi experimental designs are the

time-series design, the equivalent time-samples design, and the post-test-only

control group design.

These matters of design are well known. And the latter three designs

have been well illustrated in their application to the evaluation of an actual

teacher training product namely a Teaching for Mastery self-instructional programme.

- 14 -

CONCLUSION

In less developed countries, where projects have been aimed at improving

teaching effectiveness.and teacher education, it is not premature to use the

outcomes of this recent work. Educators can exploit the instruments - such

as teacher-training products, classroom interaction observation techniques,

and microteaching - both for the evaluation of teaching and for its subsequent

improvement. The use of these instruments almost always raises the level of

teachers' awareness and understanding of their own methods, behaviour

and effects.