process-product research oh teaching: ten years later

385

PROCESS-PRODUCT RESEARCH ON TEACHING: TEN YEARS LATER*

James W. Garrison Department of Curriculum and Instruction

Virginia Polytechnic Institute and State University C.J.B. Macmillan

Department of Educational Foundations and Policy Studies Florida State University

Nathaniel Gage is the perhaps the most important researcher on teaching in the latter half of this century. His work blazed the trail for almost all of the significant research on teaching that followed. Our own work would have been impossible without the clarity and insight that he brought to research on teaching. Somewhere down the pathway Gage cleared for us our own thinking about research on teaching divergedfrom his. We do not, however, wish to appear unappreciative. Our criticisms ten years ago and now are the result of taking Gage’s work seriously and should be read as a high form of praise.’ Without his example there would be little to look back on.

It was in the context of developing our own program of scholarship and research on teaching that we felt it necessary to comment on Gage’s work. It seemed to us then, as it still does, that those who seriously think they have anything new to offer the field of theory and research on teaching must begin by walking a way down the path Gage cleared. If the nascent investigator cannot establish any prospects for finding something new, then there is no good reason to follow her or him away from the well-worn path of prior wisdom. This reasoning led us to write “A Philosophical Critique of Process-Product Research on Teaching” in 1984; that essay later became a chapter of our book A Logical Theory of Teaching: Erotetics and Intentionality.2

In 1989 Gage and Needels published a detailed response to the critics of process- product research on teaching3 In that article, they responded to many issues we had

This essay was written independently of the preceding one. Garrison and Macmillan did not see Gage’s essay, nor did Gage see theirs. 1. Our discussion on this occasion is primarily arepriseof our own work completed between 1983 and 1988. This period begins with the publication of C.J.B. Macmillan and James W. Garrison, ”An Erotetic Concept of Teaching,” Educational Theory 33, (1983): 157-66, and concludes with the appearance of our book A Logical Theory of Teaching: Erotetics and Intentionality (Dordrecht: Kluwer Academic Publishers, 1988). A good deal of our individual and joint work in the years since our hook appeared has been an attempt to defend, extend, and apply our erotetic (questions and answers) theory of teaching. 2. James W. Garrison and C.J.B. Macmillan, “A Philosophical Critique of Process-Product Research on Teaching,” Educational Theory34, no. 3 (Summer 1984): 255-74 and Macmillan and Garrison, A Logical Theory of Teaching. 3. N.L. Gage and Margaret C. Needels, ”Process-Product Research on Teaching: A Review of Criticisms,” The Elementary School1ournal89, no. 3 11989): 253-300. This article will be referred to as PPR in the text withpagenumbersforallsubsequentcitations. Wewillfrarnemuchof thisessayasaresponse to that work, leaving it to the reader to decide how much water has flowed over the dam in the intervening years.

EDUCATIONAL THEORY / Fall 1994 / Volume 44 / Number 4 0 1994 Board of Trustees / University of Illinois

386 E D U C A T I O N A L T H E O R Y FALL 1994 1 VOLUME 44 / NUMRER 4

raised in our earlier work. In this essay, we will review three of them. They are: (i) the inability of process-product research to deal withintentionality; (ii) the atheoretical posture of process-product research; and (iii) normative issues. Gage and Needels follow Aristotle in distinguishing between essences and accidents. They write, "The distinction here is crucial. The essence of process-product research is the search for relationsbetween process and product variables. Other features of such research are incidental" [PPR, p. 254).

Gage has defended this "essence" since his classic article of 1963 in the first Handbook of Research on Teaching.? But because this defense employs reductive strategies that ultimately limit the scope and substance of research on teaching, it eventually begins to block progress along the very path that Gage pioneered. It would be better, we want to suggest, to allow "incidentals" like intentionality, beliefs, and knowledge to become essential in an expanded version of research on teaching. We want to stand on Gage's shoulders and see further if we can. Indeed, we believe this is exactly what students of his work have already d0ne.j

DEALING WITH INTENTIONALITY

Our approach begins with what we believe is the essence of teaching. Teaching, like loving (somebody) or shooting (say a basketball) is an intentional act. We intend to teach somebody something, we mean to love our lover, we intend to make the game winning shot. We shoot, and miss. We love, and lose. We teach, but children do not learn. Tragedy and comedy are about the vicissitudes of human intentions. Think about your athletic life. Think about your love life. Think about teaching. Now, think about what it will take to understand the vicissitudes of human intentions. Finally, think about the essence of process-product research.

If intentionality is the essence of teaching (assuming that teaching does have an essence), it seems to us that theories of teaching and research on teaching must display that essence at the very core of their methodology and logic. Now many philosophers and most scientists shy away from intentionality in all its guises; the presumed danger of taking an intentional approach to teaching (or anything else being studied empirically) is that intentions seem mysterious, beyond empirical verification, and (for some) irrelevant to the description and assessment of human conduct. Under thm assumption, any dealing with intentions is doomed to fuzziness, to a lack of scientific rigor. But we think that this is a misunderstanding of

4. N. L. Gage, "Paradigms for ResearchonTeaching," in Handbook of Research on Teachinged. N. L. Gage (Chicago: Rand-McNally, 1963).

5. For aparticularly poignant example, see Lee S. Shulman, "Paradigms and Research Programs in the Study of Teaching: A Contemporary Perspective," in Handbook of Research on Teaching, ed. M.C. Wittrock (New York: Macmillan, 1986).

~

JAMES W. GARRISON is Professor in the Department of Curriculum andhtruction, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061-0313. His primary area of scholarship is philosophy of education.

C.J.B. MACMILLAN is Professor in the Department of Educational Foundations and Policy Studics, 306 Stone Building, Tallahassee, FL 32306-4070. His primary areas of scholarship are philosophy of education and social foundations of education.

GARRISON AND MACMILLAN Process-Product Research on Teaching 387

intentionality; in an attempt to show how research on teaching can be rigorous and intentional, we proposed our ”erotetic” analysis and definition of teaching.

The definition reads: “It is theintention of teaching acts to answer the questions that the auditor (student) epistemologically ought to ask, given his or her intellec- tual predicaments with regard to the subject matter.”6 We will not defend this formulation here beyond making a few passing comments to avoid some of the more common misreadings. By providing an object for teaching other than students’ learning, we allow for teaching, even excellent teaching, to miss, just as all-star basketball players frequently miss the game-winning shot. Our definition empha- sizes the relation between student, teacher, and subject matter without centering the interaction on any one of them. The students need not actually ask the question. It is simply that given their epistemic state (for example, background knowledge) they ought to ask the question, and could. Here we will only be concerned with the fact that intentionality seems part of the “essence” of teaching. While the process- product research program can stretch to accommodate intentionality “inciden- tally,” it would be better to make it central in all consideration of teaching.

In our original critiques we borrowed from the work of Larry Laudan because he allowed us to draw two important distinctions.’ First, Laudan’s “research tradition’, model emphasized comparison between competingmodels of scientific research. For Laudan, science is a problem-solving enterprise. Assessment of competing theories and traditions considered not only the number and importance of problems the tradition could solve, but also the rate at which a tradition is solving its problems. Such an approach makes it rational to pursue a tradition with a high rate of solution even when it may not yet be rational to accept that tradition because it does not yet have the same probIem-solving power as a competing theory. In the middle 1980s process-product research no doubt had the greatest problem solving capacity of any of its scientific competitors, but we thought that the program had stalled and that it was time to compare it to alternatives.

Although there have been criticisms of process-product research that stress different views of the nature of science and of empirical research, we have tried to tread a line that does not rule out any approach as unscientific; yet we have tried to argue for the inclusion of such elements as values and intentionality in the traditioaB

6 . Macmillan and Garrison, A Logical Theory of Teaching, 32.

7. See Larry Laudan, Progress and Its Problems (Berkeley: University of California Press, 19 771. The preface is especially helpful. There has been considerable advance in the philosophy of science in the intervening years, but nothing to eviscerate the basic point about progrcss and the rate of progress of scientific research traditions. See also Garrison and Macmillan, “A Philosophical Critique,” 255-74; and C.J.B. Macmillan and James W. Garrison, ”Using the ’New Philosophy of Science’ in Criticizing Current Research Traditions in Education,” Educational Reseazcher 13, no. 10 [December 19841: 15-21.

8. John H. Chambers is perhaps the most trcnchant critic of this sort. See his ”Unacceptable Notions of Science Held by Process-Product Researchers,” in Philosophy of Education 1989: Proceedings of the Forty- Fifth Annual Meeting ofthe Philosophy ofEducation Society, ed. Ralph Page [Bloomington, 111.: Philosophy of Education Society, 1990),81-95. But see also Macmillan’s response in the same publication ipp. 96-100). Chambers has considerably modified and expanded his critique in Empiricist Reseazch on Teaching A Philosophical and Practical Critique ofits Scientific Pretensions (Dordrecht: Kluwer Academic Publishers, 19921.

388 E D U C A T I O N A L T H E O R Y FALL 1994 VOLUME 44 1 NUMBER 4

The second factor that attracted us to Laudan was his emphasizing that the ability of a research tradition to solve its conceptual problems is as important as solving its empirical problems. The conceptual problems of process-product research were what attracted most of our attention. Gage and Needels in turn respond to most of our comments under a subtitle, ”Criticisms of Conceptualization. ” They begin with the issue of disregardmg teachers’ intentions, a good beginning, although it ignores many other important elements of the intentional activities of teaching. Their discussion begins with the following passage from our earlier essay:

The process-product tradition explicitly ignores the intentions of teachers and learners in its investigations., . .Intentions cannotbe investigatedby theusuallow-inference methods embraced by this tradition .... The failure of process-product research to come to grips with the essential intentionality of teaching is its greatest conceptual shortcoming.. ..Without some wider context, behaviors are as meaningless as physical movements of chess pieces (PPPR, p. 25.5).

We would like to reaffirm this claim. Commenting on this passage Gage and Needels assert that “This criticism

focuses on two aspects of teaching: [a) the teacher’s intentions, that is, the teacher’s own conception of the purposes of the teacher’s behavior, and [b) the teacher’s observed behavior in the class-room” (PPR, p. 255 ). From an intentionalistic perspective something has already gone wrong. Intentional acts of teaching involve students, subject matter, and teachers. But here the intentional acts are centered entirely on the teachers. It is as if the acts of loving could be interpreted without reference to the beloved. The additional consideration of subject matter in teaching only complicates an already complicated context: the other side of intentionality [that caught up in “intentional objects” - things known, believed, understood, and so forth) adds its own confusions and difficulties.g We will not press this issue further here, but its importance should not be underplayed.

Gage and Needels attempt to finesse the problem of meaning and context in the following way: “The low-inference process variables used in some process-product research are not no-inference variables. They do entail some inference as to the meaning of the behaviors” [PPI?, p. 256). This seems an odd locution. Inference usually involves passing from one proposition to another as from premises to conclusion in logical deduction, or from statistical sample data to generalization in inductive argument. In the present case it would seem to mean passing from observed phenomena to the existence of a variable. In itself this is not a problem. However, the determination of meaning is generally taken to involve interpretation of observations or perceptions, but the “essence” of process-product research provides no such guidance for interpretation. This is a serious problem because no inquiry, scientific or other, can occur without identifiable criteria for the entities whose existence it postulates. Such criteria are central to interpretation, and any attempt to bypass this aspect of science ignores the central role of higher-level theory in determining how to “read” the phenomena under investigation. All too often, educational researchers miss this point, seeing the quest for theory being little more than a quest for an explanation that can be postponed until the “facts” are in.

9 , This was stressed in Chapter I of A Logical Theory of Teaching


Without an interpretive theory or research methodology that specifically ad- dresses intentions, Gage and Needels are forced to go elsewhere. Their basic idea is to triangulate among researchers’ interpretations of the meaning of teachers’ behaviors:

The customary checks on observer reliability determine how well two or more observers agree in their interpretations of the same teacher behavior. Such checks ascertainthe degree to which inferences of intentions from behavior are as wildly susceptible to error as the critics seem to assume. Observer-occasion agreement often yields reliability coefficients equal to .80 or better.. ..Such reliability does not, of course, guarantee validity, or correctness in inferring intentions. But it does suggest that validity is attainable. The validity of observations has often been sought through training observers thoroughly. The training has continued until the observers‘ codings of videotaped classroom processes agreed closely with those of expert observers (PDR, p. 257).

The problem of identifiability remains, it has only been pushed one step down. The questions now become “How do we identify expert observers?” and “In what will the training consist?” We are in a potentially vicious regress. What stops the regress?

Our fear is that what stops the infinite interpretive regress is the ”subjectively reasonable theories” of the experts.’O These subjective theories are never themselves tested, if indeed they are even articulated. Expert researchers use interpretive theories that we believe should be stated and placed at risk of refutation. As matters stand the “expert” researchers’ opinions seem to be adjudicated only by validity and reliability statistics without much regard for the meaning of their disagreements. Each one’s opinions appear irrefutable in principle. One would expect an appeal to ethnographic research methods at this point, since intentions are central to that methodology, but that would expand the research program beyond its stated essence. In responding to some criticisms by Frederick Erickson, a prominent educational ethnographer,’I Gage and Needels write:

As we noted above, the teacher and student behaviors observed by process-product researchers are always interpreted, even in the case of low-inference variables. Accordmgly, there is no difference in principle between the process variables of the process-product researcher and the meaning-perspectives of the interpretive researcher (PPPK, p. 292).

Put boldly, there is a huge difference between the interpretive researcher and the process-product researcher. In the first place, intentions are essential to the ethnographic researcher’s methodology, peripheral (at best) to the process-product methodology. Second, the ethnographic researcher has a method, and perhaps a theory, for interpreting observed action of a kind not possessed by expert encoders. Finally, and fatally, the ethnographic researcher is self-consciously engaged in high-inference interpretive research. Interpreting intentions is a high-inference activity. Just thmk about the complex motives in your love life, or why it is so difficult to prove “point- shaving” scandals in basketball.

10. See Gary Fenstermacher, “ APhilosophical Consideration of Recent Research onTeacher Effectiveness,” in Review of Research in Education 6, ed. Lee S. Shulman (Itaska, Ill.: F.E. Peacock, 19781, 170-76, for a discussion of teachers’ subjectively reasonable beliefs. The notion deserves to be extended to the discussion of researchers’ approaches to their own material. 11. See Frederick Erickson, “Qualitative Methods in Research on Teaching,” in Wittrock, Handbook Of Research On Teaching, 119-61.

390 E D U C A T I O N A L T H E O R Y FALL 1994 / VOLUME 44 / NUMBER 4

Wemay further note that theintentional behaviors of students and teachers with respect to subject matter must eventually be operationalized in a quantitative manner in order to take the argument places represented by the variables inprocess- product research. The essence of teaching, intentionality, cannot be represented in the essence of process-product research unless what is meant by “variables” is immensely expanded in ways that preclude measurement. Our problem is this: Rather than constantly seeking to reduce everything to fit the essence of process- product research, why not let process-product research mutate into a more refined methodology?

THE ATHEORETICAL POSTURE OF PRODUCT-PROCESS RESEARCH

As already indicated, we believe that no explicit theory guides process-product research. We have already seen the deleterious effects of not having a clear interpretive theory on the essential research tasks of determining meaning, context, and identifiability. In this section we will examine some other consequences.12

Gage and Needels borrow from R.E. Snow the idea that there are six levels of theory: formative hypotheses, elementisms, descriptive theories and taxonomies, conceptual theories and constructs, and axiomatic or broken axiomatic theories. l3

They discuss the progress of process-product research in regard to all except descriptive theories and taxonomies. Gage and Needels here use Snow’s hierarchy to support the scientific legitimacy of process-product research; but there are problems with Snow’s conceptualization as they use it, and their own examples may show what those problems are.

The first level is that of “formative hypotheses.” ‘Trocess-product researchers,’, say Gage and Needels, “create and test formative hypotheses whenever they choose a process variable as worthy of investigation” (PPR, p. 262). But why this is a level of theory is unclear. Hypotheses can come from anywhere and be about anything. Suppose someone were to hypothesize that Tarot cards work because ethereal spirits take up residence in them. At this level of “theory” nothing is disqualified. Do we really want to call that a level of theorizing? Gage and Needels assert, “Process- product research has produced many such formative hypotheses” [PPR, p. 262). Indeed. We have already shown what we think is wrong with ad hoc hypothesis formulation. In a mature science hypotheses are deduced [inferred) from laws, principles, and axioms, and then tested. In emerging sciences researchers’ subjectively reasonable webs of belief would at least be systematically stated so that they could be experimentally tested. Failing the test should endanger the entire theory or web of belief. But because of the way process-product research hypotheses are formulated, a failure to establish a connection could always be explained away by ad

12. Our most pointed comments about the atheoretical nature of process-product research occur in a paper that Gage and Needels do not cite and may not be aware of. See James W. Garrison and C.J.B. Macmillan, “Teaching Research to Teaching Practice: A Plea for Theory,” fournal of Research and Development in Education 20, no. 4 (Summer 1987): 38-43. The following passages depend upon that earlier work.

13. R. E. Snow, “Theory Construction for Research on Teaching,” in Second Handbook of Research on Teaching, ed. R.M.W. Travers (Chicago: Rand McNally, 19731. Cited in Gage and Needcls, “Process- Product Research,” 262ff.

GARRJSON AND MACMILLAN Process-Product Research on Teaching 39 1

hoc hypotheses; in any event nothing beyond the hypothesized variables and their connection would seem to be refutable. Not even the vague webs of belief of the researchers who proposed them need be placed at risk.

The second level is called “Elementisms.” These consist of “logical or empirical attempts to reduce the definition of variables and relations between them to the most elementary units possible.. ..Factor analytic subdivision of complex constructs similarly seeks primary units for theoretical work.”14 In discussing this level, Gage and Needels point out how they believe process-product research fits:

Process-product research has developed many such elementisms. Lists of technical skills of teaching are an example. The widely used categorization of process variables in teaching as structuring, soliciting, responding, and reacting also exemplifies elementisms.. . .Such elementisms constitute a kind of theory that seems to go unrecognized by those who regard proccss-product research as atheoretical (PPR, p. 263).

It is clear that no one would count just any old category scheme or list of elements as a theory, especially a scientific theory. A mere list of elementisms is no more a theory of teaching than the following is a theory of thermodynamics:

The rays of the sun; Fiery meteors; A flame; Animals: Horse dung; Nec Bureae frigus adurit [Not the cold wind of the North]I5

As we indicated some years ago, “Gage’s view of science (as exemplified in The Scientific Basis o f the Art of Teaching) is excessively Baconianandat best dubious.”16 In an earlier article we were imoressed by R C. Anderson’s observation that

knowledge is not a “basket of facts” .... Facts in isolation are meaninglcss; they do not interpret themselves. Facts are also promiscuous. They will consort with any number of interpretations and misinterpretations. Any fact may be used to support or disprove any theory that is at a more general level than the fact itself - depending of course, on the interpretation we give it.’’

Without some unifying theory, or at least a web of beliefs, the preceding Baconian list is just a “basket of facts.” So too is a list of teacher-process student-product correlations.

What could be excluded as theories by a criterion such as elementisms? And where I d process-product research get its categories? Our guess is that such ”elementisms” depend on the dubious process of hypothesis formation already discussed. This may be low level “theorizing,” but it is only useful when hypotheses are deduced from the categories, lists, and elements for testing. Nothing can be proven until something can be refuted. Gage and Needels ignore Snow’s level of descriptive theories and taxonomies. The discussion of lists and categorization

14. Snow, “Theory Construction for Research on Teaching,” 85. 15. This list was offered by Francis Bacon. See The New Organon And Related Writings, cd. Fulton H. Anderson [Indianapolis: The Bobbs-Merrill Company, 1Y60). Cited in James W. Garrison, “The Impossibil- i ty of Atheoretical Educational Science,” The lournu2 of Educutionul Thought 22, no. 1 (1988): 24.

16. See Garrison and Macmillan, ”Teaching Research to Teaching Practice.”

17. Cited in Garrison and Macmillan, ”Teaching Research to Teaching Practice,” 41.

392 E D U C A T I O N A L T H E O R Y FALL 1994 / VOLUME 44 1 NUMBER 4

processes at the level of “elementism” may well cover the descriptive and taxonomic level as well. Perhaps that is why they do not discuss this level.

The next level is that of “conceptual theories and constructs.” These “take the form of the hypothetical constructs that intervene between observed stimuli and responses” ( P P R , p. 264). Gage and Needels are eager to point out that process- product research may be extended to included cognitive mediational processes such as those described by cognitive psychologists. These processes intervene between teaching processes and pupils’ products. They see this move as “conceptual integration in the best tradition of scientific parsimony” ( P P R , p. 264). They note that we had criticized this move as being an extension of an old paradigm.18 They complain that it “has been regarded as a kind of ’co-opting.”’ The implication is that incorporating full attention to cognitive mediational processes into process-product research is somehow specious” (PPR, p. 264). We stand by that criticism. We were not and are not questioning the value of conceptual integration and scientific parsimony. What concerned us then was that

Gageseemsunaware.. .that the type of [cognitivemediational] entityproposed ... may bedifferent in kind from the variables standardly considered in the tradition; the methods of investigation appropriate for measurable variables of behavior achievement are not obviously adequate for considering ”mediating processes” -and Gage does not provide us with an argument for seeing them as if they were similar enough for variable-analysis t~eatment . ’~

What we did not say then that we would say now is that even if this were worked out, the result would not be low-inference variables or theories. No one actually sees a “cognitive schema” or “script,” nor do we infer their existence from ”stimuli and responses” as Gage and Needels put it. Like Z-particles in quantum physics the existence of such entities is inferred, and the inference of unseen entities, though exceedingly valuable and often fruitful in scientific investigation, is a highly theoretical high-inference task. Such hgh-inferences ordinarily call for higher level theorizing than that provided by “elementisms.” Said differently, high-inference entities (or variables) are ordinarily deduced from [or are an integral part of) even higher level theories. The situation is complicated even further when we realize that many of the most important mediating “variables” in research on teaching are themselves highly complex intentional and interpretive structures.

The highest level of Snow‘s hierarchy is axiomatic or broken-axiomatic theory. According to Gage and Needels this level of theorizing consists of “a set of theorems derived from a set of postulates relating undefined terms” [PPR, p. 264}. Newtonian physics is among the examples they provide. Gage and Needels conclude, “Such theory has not been achieved in behavioral and social science .... Such theory is too stringent as a present day criterion for evaluating theory production in process- product research on teaching’‘ ( P P R , p. 264). We agree with this conclusion. We would, however, remind the reader that a great deal of human behavior is puIposefu1 and goal directed. That is, it is intentional. Insofar as human behavior is intentional it may turn out not to lend itself to the construction of quasi-geometricalhypothetico- deductive theories. In any event, any theory constructed at this level would have to

18. The criticism had been made in Garrison and Macmillan, “A Philosophical Critique,” 270.

19. Ibid.. 269.


include axioms that contained intentional predicates and rules of inference that preserved these intentional properties. So far as we know the erotetic theory of teaching is the only theory of teaching that even approaches this level of theorizing. Although it is not axiomatic the logic of questions and answers does permit the natural deduction of testable conclusions.

Gage and Needels claim theoretical explanatory success for process-product research by showing how so many findings come back to “academic learning time” ALT. Here is their conclusion:

In short, many findings of process-product research can be explained by their connection with ALT. That theoretical construct, in turn, derives from the importance of time in cognitive processes. Learning, remembering, thinking, and problem solving take time (PPR, p. 265).

In support of this conclusion, they argue that the enhancement of ALT explains the effectiveness of many classroom management practices supported by process- product research:

These practices keep teachers and students from wasting time, and hence reducing ALT, by reducing students’ (a) disruptive behavior, (b] standing in line to have seatwork checked by the teacher, (c] inability to participate in recitations because brighter students are preempting too many opportunities, Id] experiencing frustration and anxiety arising from overly difficult tasks ,... and so on (PPR, p. 265).

The problem with this claim is that ALT itself does not explain academic achievement - although, as Gage and Needels point out, the two are highly correlated. If we want to understand that achievement, we need to look at what constitutes it, including all its intentional aspects. And this is just what is missing from process- product research, except when it takes cognitive mediations to be central. Even then, the correlations sought do not do the explainirgZ0

The failure of process-product research to come to grips with the intentionality of teaching is especially evident in its principle of causation. Gage and Needels argue that process-product research has contributed to improving teaching. They accu- rately observe, “Improvement implies causation - to change teaching in such a way as to cause greater achievement of desired ends” (PPR, p. 269). We agree, but talk of “desired ends” fairly cries out for an intentional theory of teaching. Instead, we are told,

In science, one prime way to determine whether a relationship is causal is to conduct an experiment, that is, an investigation in which the “independent”’ variable is manipulated and thesubsequent valuesof another, or “dependent,”variable are measured. ... If sourcesof variation in the dependent variable other than the independent variable have been controlled, the independent variable is considered to have caused the change in the dependent variable (PPR, p. 269).

This is an odd statement because Gage and Needels are careful elsewhere in their paper to note that correlation does not imply causation. But our worry is that this is fundamentally the wrong kind of causation to be talking about in the context of an intentional activity like teaching. In our book we sought to extendDonald Davidson’s idea of reasons as causes to teaching. The idea was to develop an idea of logical

20. In A Logical Theory of Teaching, we provided one example of how a finding of process-product research might be explained erotetically: the “inverted-U” correlation between time-on-task and academic achievement. See pages 208-9.

394 E D U C A T I O N A L T H E O R Y FALL 1YY4 / VOLUME 44 / NUMBER 4

causation compatible with storytelling and narration as well as with qualitative and ethnographic research. We suspect that process-product research cannot capture the narrative of teaching at all - a serious shortcoming, for, as Jerome Bruner says, ”Narrative deals with the vicissitudes of human intentions.”21 It is fruitful, we believe, to view all forms of causal explanations as forms of narration. But that is another argument.

As must be clear, we have no great confidence in Snow’s delineation of levels of theorizing, but we have accepted them for the purposes of examining Needels and Gage’s response. So what may we conclude from this examination? First, and foremost, we observe that the essence of process-product research cannot deal with the intentionality of teaching and therefore must go outside of its own limits to cope with the essence of its subject matter. The result is a series of reductive and, yes, co- opting, strategies. Low-inference techniques will not handle the high-level inference required to deal with intentionality. These difficulties are only compounded by relying on the subjectively reasonable or personal theories of expert coders and researchers, especially since such theories are rarely stated or explicitly tested.

Bolstering the bottom level “theorizing” of process-product research with Baconian lists and category systems does not seem to us to advance the research program. The inability to deal with intentionality at the level of mediating variables fails completely because the inference to ”hidden” variables must be high-level. Moreover the inferred “hidden” variables are often intentional. How these intentional constructs are to be quantified is unclear. We suspect that in the overwhelm- ing number of cases the quantification will be almost uselessly reductive - like trying to measure love by the number of kisses given per unit time, and love’s causes by an analysis of variables. We are puzzled as to why Gage would not be pleased to have the essence of his research program taken up into ever-expanding research programs on teaching, as it has, rather than attempting to reduce emergent research programs to the essence of process-product methodology.

NORMATIVE ISSUES Gage and Needels liken the successes of process-product research to that of

medical research. They specifically discuss the success of aspirin, discovered in 1890 but not given a theoretical basis until 1970. They observe that “Macmillan and Garrison, however, objected to such a comparison on the grounds that the medical experiments had theoretical bases, while, in their opinion, these experiments on teaching do not” (PPR, p. 282). To see what we meant in our original critique of Gage’s comparison we need to clarify the context in which our comments were originally made.

Gage declared early on that “I want to study teaching for the purpose of improving Improvement may not be the essence of process-product

21. Jerome Bruner, Actua l Minds, Possible Worlds (Cambridge: Harvard University Press, 19861, chap. 4, ”Two Modes of Thought.”

22. N.L. Gage, “Research on Cognitive Aspects of Teaching,” in The W a y Teaching Is (Washington, D.C.: Association for Supervision and Curriculum Development and NEA Center for the Study of Instruction, 19661.


research, but it provides the practical justification of such research for purposes of funding and implementation in teaching practice. Our concern in 1984 arose from Gary Fenstermacher’s observation that “the school situation is made up of persons who act intentionally within a social Our concern was not merely with the absence of theory. Rather, we found that the problem was that “Process-product research is conducted in the language of analysis of variables but its application is in the intentional language of action and belief. The possibility of misinterpretation

Elsewhere we noted that facts are subject to multiple interpretation dependent on the given interpretive context.25

Our concern was that whereas medical researchers and medical practitioners share a similar set of biochemical background theories with which to interpret the facts of research in practice, the theories, such as they are, of the researchers on teaching may be largely incommensurable with the subjectively reasonable theories of the teacher practitioners. Simply turning research facts over to practitioners is likely to lead to objective facts being subjectively interpreted and applied in idiosyncratic and inconsistent ways. This concern is made more serious by the fact that Gage insists that process-product research is to serve as the scientific basis of the art of teaching.26 Gage acknowledges that because of complex interaction effects, relationships between more than three or four variables, ”the usefulness of what science can give the teacher begins to weaken .... At this point, the teacher as artist must step in and make clinical, or artistic judgments about the best way to teach.”27

Yet as we noted: These results [such as time on task] are used in [teachers’] subjectively reasonable theories.. .our contention is that the rcsearchers should attend to the development of objectively reasonable theories. Statistics do not tell the story, and when classroom happenings begin to belie the statistics, that is, when teachers find that process-product findings do not give guidance, they fall back on pre-research intuitive theories. But that is where they need help.*”

The statistics do not tell the story of the vicissitudes of human intentions that teachers need in order to understand the unfollng drama within their classrooms.

One reason that we became concerned with the philosophical underpinnings of process-product research at the time that we did was that by the early 1980s legislation was beginning be introduced to hold teachers accountable to the results of process-product research. In Florida this became The Florida Performance Mea- surement System and various sets of “essential teaching competencies” for teach- e r ~ . ~ ~ In Virginia, the legislative mandate took the form of the Beginning Teacher

23. Fenstcrmachcr, “A Philosophical Consideration of Rccent Research.”

24. Garrison and Macmillan, “Using the ’New Philosophy of Science,”’ 18.

25. Macmillan and Garrison, A Logical Theory of Teaching,

26. See Gage, The Scientific Basis of the Art of Teaching (New York Teachers College Press, year). See also Gage and Needels, “Process-Product Research on Teaching,” 289.

27. Gage, The Scientific Basis of the Art of Teaching, 20.

28. Garrison and Macrnillan, “Using the ’New Philosophy of Science,”’ 18.

29. Our criticisms went in this direction, too. See, for example, C.J.B. Macmillan and Shirley Pendlebury, “The Florida Performance Measurement System: A Consideration,” Teachers College Record 87, no. 1 (1985): 67-78.

396 E D U C A T I O N A L T H E O R Y FALL 1YY4 VOLUME 44 / NUMBER 4

Assistance Program.3o Both of these programs involved little more than behavioral check-off lists. They are typical instances of research first being done by researchers and subsequently being used on teachers as an accountability device. Our concern with normativity was not, as Gage and Needels wrote in 1989, that “education cannot be studied empirically with scientific methods because its phenomena, such as learning, change with human purposes” (PPR, p. 260). Rather our concern was with the conversion of the results of scientific research into practice.

In our original work we considered arguments that social sciences are inherently policy sciences, and that social scientists couldnot hide behind the cloak of scientific detachment and impartiality. Even so simple a notion as a “problem” has overtones that deny such qualities. As Kenneth Strike writes, “Situations do not become problems unless we approach them with values which specify what properties these situations ought to have.”3’ Gage himself seems to move in this direction with his ‘ I teacher should” statements. For example, “Teachers should have a system of rules that allow pupils to attend to their personal and procedural needs without having to check with the teacher.”32 Prima facie, this seems like a good idea, but the sense in which it is an inference from an analysis of the interactions of three or at most four variables, as Gage suggests it is, is not entirely clear. We suspect that there is a subjectively reasonable theory floating around here somewhere and we would like to know what it is, so that it might be scientifically tested.

The normativity of process-product research provided the practical justification for an expensive research program. Sooner or later policy makers were bound to cach it in. We believe that when research programs make explicit normative claims and are prepared to issue normative performance statements that are readily converted to rules of policy, then moral issues may be properly introduced into the evaluation of the research program. Our primary concern is that process-product research has had relatively limited results for its expense and has tended to be used to disempower rather than empower classroom teachers.

We will conclude by going to a philosophy of science that may help to clarify our position. Imre Lakatos observed, “We have to remember the conventionalists‘ methodological discovery that no experimental result can ever kill a theory [or research program]: any theory can be saved from counterinstances either by some auxiliary hypothesis or by a suitable reinterpretation of its We believe that Gage is pursuing such a conventionalist strategy, but we are not trying to “kill” process-product research. Rather, we are urging its author to allow it to evolve into

30. The Beginning Teacher Assistance Program was changed from the Beginning Teacher Assessment Program in the last phase of implementation. Jim Garrison was one of the reviewers of the program of the State Department of Education in Virginia.

31. Kenneth A. Strike, “An Epistemology of Practical Research,” Educational Researcher 8, no. 1 (19791: 10.

32. Gage, The Scientific Basis of the Art o f Teaching, 20.

33. Imre Lakatos, ”Falsification and the Methodology of Scientific Research Programmes,” in Criticism and the Growth of Knowledge, ed. Imre Lakatos and Alan Musgrave (Cambridge: Cambridge University Press, 1970), 116.


the host of research programs that have flourished in the healthy environment he provided.

Gage’s conventionalism shows most clearly in his distinction between the essential and incidental features of process-product research. Lakatos wrote:

Allscientificresearchprogrammesmay becharacterizedby their “hardcore”.. ..We mustuseour ingenuity to articulate or even invent “auxiliary hypotheses,” which form a protective belt around this core .... A Research programme is successful if all this leads to a progressive problemshift; unsuccessful if it leads to a degenerating problemshift.34

The core of process-product research is what Gage and Needels call its essence. They conclude:

The essence of process-product research on teaching consists of the search for relations between process and product variables. The accidents are those incidental features of such research that may or may not be present in any specific process-product investigation without altering its process-product character (PDR, p. 2911.

As we have indicated repeatedly, this essence does not come to grips with the essential intentionality of teaching. Indeed, the essence of process-product research seems to be purely formal and devoid of empirical or conceptual content. It can import intentionality or anything else into its protective belt by various ad hoc hypotheses, but the question remains, Does it lead to a progressive problemshift?

The answer, we think, is no. A progressive problem shift for Lakatos means either a theoretically or empirically progressive shift. A theoretically progressive shift occurs “if each new theory has some excess empirical content over its predecessor, that is, if it predicts some novel, hitherto unexpected fact.”35 It seems to us that Gage’s co-opting strategies succeed only in explaining the empirical content of other research in process-product terms without any unexpected novel empirical predictions. Indeed, we are not sure that “theories” at the hypothetical and elemental stages can predict much of anything novel. An empirically progressive problem shift occurs when “some of this excess empirical content is also corrobo- rated, that is, if each new theory leads us to the actual discovery of some new fact.”3h Even where process-product research has shown progressive problem shifts we are not convinced that the rate of progress justifies the expense. In the end these assessments will be made by the field of educational research. We urge caution.

Gage and Needels conclude their paper as follows: “In the long run, educators should insist on process-product research. They need knowledge of the connections between what teachers do and what students learn” (PPR, p. 295). We certainly agree. We only urge Gage and other researchers on teaching to allow this research project to become part of more expansive research programs that continue along the path of inquiry that Gage pioneered. Ultimately this is why future generations of educational researcher will recognize him as the most important researcher on teaching in the last half of the twentieth century.

34. Ibid., 133.

35. Ibid., 118.

36. Ibid.

process-product research oh teaching: ten years later

Documents