STEPHEN P. NORRIS

Synthesis of Researchon Critical ThinkingStudents need more than the ability to be better observers,they must know how to apply everything they already knowand feel, to evaluate their own thinking, and, especially, tochange their behavior as a result of thinking critically.

he field of critical thinking ismore vibrant than ever There ismuch research in progress on

the meaning of critical thinking, on thetransferabilits of critical thinking skillsto a wide range of subject areas. andon methods of teaching critical think-ing. A substantial bodv of knowledgeexists in the area. and it is possible tosuggest with confidence some re-search results to help teachers. In of-fering these suggestions I interpretresearch broadly to include not onlyempirical research, but also philo-sophical and polic- researchCritical Thinking Is aComplex of ManyConsiderations.Thinking critically can be defined asrationally deciding what to do or be-lieve (Blair. 1983. Ennis, 1981: Hitch-cock. 1983). To he rational about suchdecisions requires more than avoidingsome standard list of errors in thinking Being a critical thinker of courseimplies assessing the views of othersand one's own views according toacceptable standards of appraisal Butit implies more than this One mustalso be productive, in the sense ofconceiving of alternative courses ofaction and candidates for belief, be-fore critically appraising which alter-native to cho(xse People must be ableto produce reliable observations.make sound inferences., and offer rea-sonable hypotheses Finallyv, one musthave the dispusition to think produc-tively and criticall- about issues, orelse no amount of skill in doing so willbe helpful

Critical Thinking Is anEducational Ideal.

For many people in education it mightseem like a needless question to askwhv critical thinking is desirable It islike asking wh- education is desirableIt can be argued that both are worth-while in themselves However, justmouthing or blindly concurring withan educational goal provides fragilesupport for it. There needs to be ajustihcation for the teaching of criticalthinking based on grounds that wouldbe considered sound no matter whatthe current trends.

Recent work by philosophers ofeducation begins to provide this need-ed justification (\McPeck, 1981; Siegel,1980, 1984) According to their view,critical thinking is not just anothereducational option. Rather it is an in-dispensable part of education, becausebeing able to think critically is a neces-sarv condition for being educated. andbecause teaching with the spirit ofcritical thinking is the only wa- tosatisfy the moral injunction of respectfor individuals, which must applh tostudents as well as to anyone elseAccording to this reasoning, studentshave a moral right to teaching thatembodies the spirit of critical thinkingand a moral right to be taught how tothink critically Thus. to abide by themoral principle of respect for persons.teachers must recognize "the student'sright to question. to challenge, and todemand reasons and justifications forwhat is being taught (Siegel. 1980. p14)) In addition. there is a responsibilits to teach them to do these things

well, because in the end students mustch(x)se for themselves there is noescaping this truth

Critical Thinking Ability IsNot Widespread.

Mans claims that critical thinking is notwidespread are hased on anecdotalevidence. However, more systematicresearch also suggests that most highschool and college students do notperform extremely well on the kindsof tasks that are used to indicate criti-cal thinking competence. and there isevidence to suggest that adults fare nobetter In addition. there is considerable evidence on the consequences ofpeople failing to subject their behaviorto the standards of critical thought

Ei'idence from perfornance on crit-ical thinking tests The most widelsknown general critical thinking testsare the Cornell Critical Thinking Tests,Levels X and Z (Ennis and Millman.1985) and the Watson Glaser CriticalThinking Appraisal. I:orms A and B(Watson and Glaser, 1980) For theCornell Tests. critical thinking is de-fined as the process of reasonablsdeciding what to believe" (Ennis, Mill-man, and Tomko, 1985). with this rea

Stephen ' N%'orrmi is A.sistant rofe.ssor,Institute for KEldcatbortal Research and I)e-velopmentt. ,tle7noriacl i nitrersin of Nivefoundland, St John s. Ne'tefoundland,Canada

Author s n tc I thank Robert Ennns,.lMicbael Jackson, Richard Paul, and LindaI'hillips-Riggs for teir helpful commentsoni an earlier dra/i aof tbis article

io40 E.1)i (w.i i0%,,v LEM IA KsIM

sonable decision making to be carriedout in accord with certain principles ofthinking (Ennis. 1980) Test items areintended to indicate whether examin-ees have a know ledge of these princi-ples and their application The highestreported median score on Level X is48 out of a possible score of '1, ob-tained by 10th grade historn students.and the lowest is 29, obtained byabove average IQ 8th grade students.On Level Z. median scores reportedfor undergraduate university studentsare 30 out of a possible 52. These testresults suggest that the level of criticalthinking is not extremel high at an!level of schooling, a disturbing resultif it can be confirmed

The W'atson Glaser Tests are de-signed to measure such things asability to recognize assumptions, toevaluate arguments. and to appraiseinferences Results for high school stu-dents indicate median scores of be-tween 41 and g4 out of a possible 80,with an increase in scores occurringwith grade level Median scores forcollege students range from 52 to 60.The students wxho do the best on thetests are enrolled in M13A and medicalprograms. n ith median scores of 66and 68 respecmtielh.

If we niote thit the lproblems posedon these tests are the sorts of prob-lems wCe w-ould like everoine to heable to solve xvell. then the resultsdemand somc attention from educa-to rs Bs and la-ge. median scores arelo.w, indic;ating that at least half thestudent piopulati()n cannot coinsistentlvthink criticallx about the problems onthile tests This conclusiotn is in concertx ith recent findings in mx researchusing a test of a single aspect of criticalthinking-the abilinty to appraise ohb-servations High school students'scores on the test averaged -49 percent.and ranged from less than 2 percent to.4 percent. with 9() percent of studentsscoring less than 65 percent (Norrisand King. 1984) The test has about a6th grade reading level, seemed tocapture students' interest and dili-gence. and was X; ell understood bhmost, leaving poor critical thinkingahilitV as 1a ter pla;sible explanationof loxw scores

F vidlenc /rfiY'm psl'cho/ogica/ -researdc l'sycchohOgic.i[ l research onthinking do)es n(it UisLiall deal directlxwvith criticll thlikinig S,)me studiesfoicus (n errors of adufilts thinking.others ex.millne the thinking of expertsin particular fields to discover how;A

they approach problems differentlyfrom novices; while others examinehow qualirt of thinking bears on socialrelations such as obedience to othersand authoritv over others. Each ofthese sortns of research is relevant toeducation. It is helpful to know theerrors in reasoning that persist intoadulthood so that preventive measuresmight be taken in schools. It is alsouseful to know how experts think,since this can provide guidance forinstruction in good thinking Finally, itis crucial to knox- how critical think-ing. or lack of it. affects our socialrelations

One of the most extensive reportsof studies conducted on the qualilt ofadult thinking focused on aspects ofthe inferential abilirv of adults andindicated that there are systematic ten-dencies to err on some of the simplestjudgments of even-dav affairs (Nisbertand Ross. 1980). One such situationinvolves the determination of whetheror not txo things are associated Con-sider thile diagraml in Figure 1 Thenumbers indicate the number of casesin which a disease wnas present orabsent. Thus. the present/present cellindicates that in 20 cases the symptomand the disease w ere present together:the cell to its right indicates that on 10cK'casions the disease was absent whenthe symptom was present. and so on Acommon error. among man! others. isto conclude that tile symptom and thedisease are related because more peo-pie wxho have the symptom have thedisease than xwlho have the symptomand do not have the disease Thisthinking is erroneous The main proh-lem is failure to recognize that all theinformation must he considered to-gether to arrive at a legitimate conclu-sion.

Similar errors are made hb peoplewho conclude that running is had foryou because people have died of heartfailure wxhile running. that smokingdoes not cause cancer because the-knox mlan! people wxho smoke xwhohaIve not contracted calncer. and thatrural people are more hospitable than

cint people because the rural peoplethey have met are hospitable. One aimof critical thinking instruction is toimprove thinking about matters suchas these.

A famous piece of research on theeffects of quality of reasoning on socialrelations was conducted bv StanleyMilgram at Yale Ulniversity more thantwo decades ago (Milgram. 1963). Theexperiment studied the degree towhich people will allow their commit-ment to obev someone in authority tooverride other competing moral prin-ciples Contrar to all predictions. obe-dience to authorint led to frighteningand telling results

The subjects were studied separate-lI. Each was ordered to administerelectric shocks to a learner wheneverthe learner failed to perform correctly.The subject administered the shocksbh pushing a series of switches on anelaboratelh designed panel. Theswitches were clearlv labeled withvoltage readings ranging from 1S to450 volts and x ith descriptions slightshock, moderate shock. strong shock.vern strong shock. intense shock. ex-treme intense shock. danger: severeshock. and just the letters XXXI on thelast two switches The subject was toldto push the next higher switch eachtime the learner failed to respondcorrectl- The learner was in a sepa-rate room visible to the subjectthrough a wvindox-. and communicatedanswers to the subject hb pushingbuttons

The learner -;las an actor and aconfiederate of the experimenter. andno real shocks x-ere administered Thelearner respl)nded to the vari(ous"shocxks" in standard -xavs When thesubject administered the 300-voltshock. the learner (xxho xwa-s hound tohis chair) pounded on the wall of theroonm so that the subhiect could hearFrom this point on tile learner gave nomore responses to the subject's ques-tions The experimenter asked thesubject to continue and to treat noresponse as an incorrect one. If thesubject hesitated, the experimenter

M\At 19]q8S t

Figure 1. Association Between Symptom and Dhiseae.

Disease A

Present AbsentSymptom X Present 20 10Symptom X Absent 80 40

gave an order to continue. Despite"profuse sweating, trembling, and stut-tering" only 14 of the 40 subjectsdefied the experimenter's order andrefused to continue the experiment tothe end The remaining 26 subjectscontinued until the maximum shockof 450 volts was administered.

What do the results mean? The sub-jects, all adults, knew from childhoodthat it is wrong to hurt other personsagainst their wills. Yet, the majority ofthe subjects violated this principle onthe command of someone who had noway to enforce his commands, and noway to punish those who disobeyed.From their expressions and words,many subjects clearly knew that theywere acting immorally, vet they con-tinued with the experiment. The re-sults point to a breakdown betweencritical thought and action, a link thatinstruction in critical thinking is in-tended to forge.

Critical Thinking Is ExtremelySensitive to Context.This is true for two reasons. First, theinferences and appraisals of infer-ences that a person can justify makingdepend on the background assump-tions, level of sophistication, and con-cept of the task. Inferences that do notagree with those sanctioned by a testor with those a teacher might make donot necessarily indicate a critical think-ing deficiency There are other possi-ble sources of the disagreement.Therefore, assessment of critical think-ing competence must take into ac-count the context in which the think-ing is done.

This is nevet more apparent than inattempting to assess people's ability tomake and appraise inferences. Ennis(1984) cites an example from the Wat-son-Glaser Test that requires the ex-aminee to make certain political as-sumptions in order to choose thecorrect answer. Making one set ofassumptions leads the examinee tochoose the keved answer, while mak-ing another set would mean selectionof a response that would be markedincorrect. In Ennis' view, it is unfair forpolitical beliefs to influence scores ona critical thinking test because suchbeliefs are "value judgments aboutwhich there is possible [reasonable]disagreement and which are not con-stitutive of critical thinking." So, ininterpreting scores on the test, it isimportant to understand students'background assumptions because theyhelp determine the context in which

students reason and justify the conclu-sions students reach.

Second, critical thinking is sensitiveto context because context can dra-matically affect the quality of one'sperformance This is a highly con-firmed result in the area of deductivelogical reasoning (Evans, 1982). De-ductive logical reasoning is based onthe form of the reasoning rather thanon its content. Simply put, the ques-tion of whether or not a conclusionfollows from some reasons is an-swered in deductive logic by examin-ing the structure of the reasoning. Ifthe structure is of a deductively validform, then the conclusion follows.This decision about structure is madeindependently of the content of thereasons and the conclusion. Despitethis, people reason better deductivelywhen dealing with thematic contexts,with contexts that relate to their per-sonal experience, and when they donot have presumptions about the truthof the conclusion. In addition, deduc-tive reasoning performance is loweredin contexts involving threats andpromises. There is reason to think thatcontext will also affect critical thinkingperformance (McPeck, 1981; Norris,1985).Assessments of CriticalThinking Should SeekExplicit Indications ofPeople's Reasons forTheir Conclusions.Explicit indications of people's reason-ing are required in order to differentiate between deficiencies in thinkingand differences in background beliefsand assumptions between the examin-er and the examinee Most criticalthinking tests do not provide informa-tion about what the examinee is think-ing. That is, they provide only theconclusions to thinking processes, notthe processes themselves. This is par-ticularly troublesome when test scoreshave direct implications for individ-uals, but can be alleviated in part byseeking reasons for answers on suchstandardized objective tests. Anothertechnique is to use essay instead ofobjective tests Essay tests are harderto grade, but they do lead to a moreprofound insight into the thinkingprocesses the examinee used in arriv-ing at solutions. The Ennis-Weir Criti-cal Thinking Essay Test (Ennis andWeir, 1985) is one to consider in thisregard.

In our research (Norris and King,1984) we have addressed this problem

by using protocols of students' think-ing in the design of a critical thinkingtest on appraising observations (Nor-ris and King, 1983). Test questions areput in the context of stories. One ofthe stories describes a traffic accidentat an intersection, and people whowere either involved in the accident orwho were bystanders report on whatthey observed Examinees are to judgethe believability of the reports. Suchfactors as the observers' conflict ofinterest, their expertise, and theiremotional condition are relevant fac-tors in making the evaluations. Weasked a sample of high school studentsto think aloud as they worked throughthe questions. The protocols assistedin adjusting the test until good andpoor thinking were associated, by andlarge, with keyed and unkeved an-swers. The following example illus-trates one such adjustment

The introduction to an earlier ver-sion of the test provided a list of thenames of all the characters and whatthey were doing at the time of theaccident. Since there were severalcharacters in the story with differentroles, we thought that providing thenames in a single list would helpstudents keep them straight However,this raised unexpected problems. Forthe first six items many students re-ferred to the introduction for evidenceto support their choice of answersWhile this is a legitimate thinking strat-egy, it contributed to uncontrolled in-fluences on students' responses andthence to unjustified interpretations ofthe qualitv of their thinking

For example, in item I two peoplewho were in a car when the accidentoccurred, but who were not involvedin the accident, reported on the num-ber of cars at the intersection Martine,the driver, reported that there werethree cars. Pierre, a passenger, wasreading a map and trying to decidewhich way to go. ie said there werefive cars at the intersection. The kevedanswer is that Martine's statement ismore believable Good critical think-ing would lead to this response be-cause Martine, who was driving, wouldtend to be more alert to the number ofcars than would Pierre.

However, several students chose thecorrect answer by referring to theintroduction and counting the numberof cars mentioned there The introduc-tion did not say how many cars were atthe intersection, but does mention

E)oCA' ro\Ai Li:\I )ER.islIP

"From their expressions and words, many subjects clearly knew that theywere acting immorally, yet they continued with the experiment Theresults point to a breakdown between critical thought and action, a linkthat instruction in critical thinking is intended to forge."

three cars that wxere involved in theaccident Thus, a student who was notthinking critically would assume thatthe number of cars at the intersectionequalled the number of cars in theaccident, whereas the critical thinkerwould realize the fallac' of this rea-soning. Thus, the noncritical thinkerwould be rewarded with getting theitem correct through an unsoundthinking process Having obtained rec-ords of the students' thinking, we wereable to more accurately interpret theirchoice of answer and to make suitablemodifications to the test Without ex-plicit indications of examinees rea-sons for their conclusions, the testwould not likely have been modifiedin this way. and thus would have con-tinued to yield inaccurate indicationsof level of critical thinking

Readily Identifiable Errors inThinking May Be Indicative ofThinking Errors at a DeeperLevel.Errors in thinking are often describedin terms of fallacies committed orprinciples of good thinking violatedHowever. addressing these deficien-cies directly and even correctingthem, may not be a complete solutionto the problem ,of poor thinking Inaddition to what are often called cog-niti*e or nonexecut, e thinking skills,sound thinking also requires the use

of imetacognitiue or exectaite skillsCognitive skills are those directly usedin carrning out some task and are theones on which instruction typicallyconcentrates. Recent studies haveshown that in addition to these cogni-tive skills. good thinking involves theuse of such metacognitive skills asplanning. monitoring, and revising theprogress of the cognitive skills. In thearea of reading comprehension. forexample. Brown ( 19-8) has found thatthe sound use of metacognitive skillsmarks an important difference be-tween better and worse readers

Some of the more important infor-mation on this topic derives from re-search on the thinking of experts inparticular areas compared to novicesThe assumption is that experts shouldbe critical thinkers (at least in theirown fields) and that studxving how theythink ought to provide insights intohow w*e can make novices into criticalthinkers Txwo features become imme-diatel- apparent -when experts arestudied closely: the!- possess far moreinformation than novices and haveautomated man!- of the sequences in aproblem solution. thex- are thus capa-ble of arriving at a correct solution infar less time than the novices Supportfor this belief lies in the research onexpert and novice performance insolving physics problems (Larkin andothers, 1980)

The possession of more informationand more automated problem-solvingtechniques is, however, only part ofwhat distinguishes the expert from thenovice. Another difference involvesthe heuristics (problem-solving meth-ods) employed by both groups. Tvpi-cally. novices solve problems by work-ing backward from the unknownsolution to the facts that are given inthe statement of the problem. Workingbackward in this way is usuallythought to be a sophisticated strategy.Experts are more discriminating intheir approach. When problems seemamenable to relatively straightforwardsolution. experts work forward fromthe given facts without any particu-lar planning, except to generate asmuch information about the problemsituation as is possible with the factsprovided. Their thinking is that thesolution will turn up among this infor-mation. Working-backward strategiesare employed bv experts only formore difficult problems.

In addition to this initial decisionregarding the direction in which towork. expert phvsicisLt spend time atthe beginning of a problem decidingon the appropriateness of other fea-tures of their approach. For example.they decide whether a qualitative or aquantitative approach is better. wheth-er or not to employ a pictorial repre-sentation of the situation. and whichphysical principles seem most relevantto the problem. In short. the expertphysicist approaches a problem byfirst making decisions about the over-all strategy to he used before gettingdown to the actual process of solvingthe problem. The novice. on the otherhand. gets immediatelx to work at theproblem-solving process. The time theexpert spends in initial planning paysoff in the end.

Our current research supports thefinding that the initial stage of prob-lem solution is most crucial. The bet-ter thinkers on our observation testconcentrate initially on identifying thecorrect problem thev are to solve.Poorer thinkers usuallyv fail to identifythe correct problem. may simply re-peat details of the item as their re-sponse to the problem, and often be-come embroiled in irrelevant detailsof the story line. which lead them ontangents away from the real problemthey are to solve Typically, they do allof this xwithout any apparent recogni-tion of the fact that they are goingastray.

'lxx t989 -43

I

NIAN 1985 43

The Critical Spirit Is asImportant as Skill In CriticalThinking.No matter what level of critical think-ing skill a person possesses, it is of nopractical benefit unless the person isdisposed to use these skills when theyare appropriate (Sternberg, 1983); thatis, unless the person has the criticalspirit. This spirit has three require-ments. The first is to employ criticalthinking skills in reasoning about situ-ations encountered in the world. Thesecond requirement is that criticalthinking be turned upon itself, that is,to think critically about one's ownthinking (Paul, 1982). Without this,critical thinking becomes mere criti-cism instead of an honest and opensearch for truth. To avoid this result,teachers must explain the value of thecritical spirit and display it in theirdealings with students Finally, there

a

must be a disposition to act in accordwith the dictates of critical thought.Having the correct belief or knowingthe right thing to do is not sufficient, asthe Milgram experiment startlingly il-lustrated.

Critical Thinking Skills Are NoSubstitute for Experience,Common Sense, and SoundKnowledge of Subject Matter.A set of critical thinking skills, howev-er well developed, cannot compensatefor lack of knowledge in the area inquestion. The application of criticalthinking principles involves a compe-tence over and above knowledge ofthe principles themselves (Ennis,1980; Norris, 1984). Successful appli-cation requires, among other things, aknowledge of the subject matter, expe-rience in the area in question, andgood judgment. This realization can

lead to the conclusion that criticalthinking is best taught within the tradi-tional subject areas rather than as aseparate subject (McPeck, 1981), butnobody really knows which approachis better (Norris, 1985). There is goodreason to believe, however, that prin-ciples of critical thinking taught with-out any view to their application toreal world problems will not be bene-ficial

There Is Little DetailedKnowledge About theEffectiveness of TeachingCritical Thinking.Research on the effectiveness of criti-cal thinking instruction almost invari-ably uses indicators of effectivenessthat are insensitive to fine details. Theresearch typically concludes that in-struction is effective (Annis and Annis.1979; Frank, 1969; Moll and Allen,1982; Ross and Semb, 1981; Wolf andothers. 1968; Wright, 1977: Yeazell.1981) In these studies. and ones likethem, classes of students experience atreatment designed to improve someaspect of their thinking ability Thetreatments usually consist of a unit ofwork extending over a few weeks or aslong as a vear. and are based on theintuitions of the researchers aboutwhat ought to he effective instructionin thinking. The criterion for determining whether or not the treatmenthas a positive effect is often one of thegeneral critical thinking tests mentioned earlier. or a test designed spe-cifically for the studv

Manv of the studies do not usecontrol groups, so special care mustbe taken when interpreting their resuits. Regardless of anv specific limita-tions on the research design, however,two issues emerge. There is little, ifany. evidence on the long-term impactof instruction in critical thinking, de-spite the fact that the vision of suchimpact is central to the justification ofcritical thinking instruction In addi-tion, while the conclusion of the studi-ies is usually that instruction leads tobetter critical thinkers, we do notlearn what specificallv makes thesestudents better thinkers and in whatspecific ways they can still improveAre thev better thinkers because theyhave acquired a greater knowledge ofprinciples of thinking, such as thoseproposed by Ennis, or because theytend to monitor more skillfully theprogress of their own thinking, orbecause they have more completely

44 .,sw\'.i jA-,AI)l H'.T lip

Highlights from Research on Critical Thinking* Cidtial thi is a comIex of many considerations. It requires

individuals to assess their own and others' views, to seek alternatives,make inferences, and to have the disposition to think critically.

* C 11 is an educKonal Ideal. It is not an educationaloption. Students have a moral right to be taught how to think critically.

* Calid hhi_ ably is not widespread. Most students do notscore well on tests that measure ability to recognize assumptions,envaluate arguments, and appraise inferences. Adults, as well, frequent-ly make simple judgmental errors on simple problems.

* Cdial I is sensislve to conlext. Students' backgroundknowledge and assumptions can strongly affect their ability to makeoorrect inferences. Inferences are more likely to be correct when the

-context relates to the individual's personal experience and whenperformance is not associated with threats or promises.

* Tacers sheuld look for the reasoning behind students' conclu-19m1. Coming up with a correct answer may not be the result of critical

thinkling. Essay tests are more likely to reveal the student's thoughtprocesses than are objective tests. And the tests themselves must beevaluated critically to make sure they require critical thinking skills.

*Sple errn s may signal errors in thinking at a deeper level. Intrying to solve complex problems, for example, students may err notjust by making a miscalculation, but by using an incorrerct approach tothe problem. They should be encouraged to take time before solving aproblem to decide how to go about finding the solution.

* Hmaw a cilial sp is as hqmorant as thinking criticaly. Thecritical spirit requires one to think critically about all aspects of life, tothink critically about one's own thinking, and to act on the basis ofwhat one has considered when using critical thinking skills.

* To Odl cadly, one must hawe knowledge. Critical thinkingcannot occur in a vacuum; it requires individuals to apply what theyknow about the subject matter as well as their common sense andexperience.

MWe do not know a great deal about the effects of teaching criticalAhY . Critical thinking programs may teach students to be betterthinkers, but more detailed knowledge is required before.we willknow specifically how students improve and how they remain defi-cient.

44 Eli (,Mr()N.J L.F/\)ERSIIP

adopted Siegel's critical spirit? Theproblem is similar to one encounteredwith some studies of the power ofcertain science curriculums to teachscientific thinking processes. Whilemany of these studies conclude thatthe programs are effective in, for in-stance, teaching students to be betterobservers (Ayres, 1969; Somers andLagdamen, 1975; Wideen, 1975), theydo not specify (because they neverwere designed to find) the detailedwavs in which students have and havenot improved (Norris, 1984). If diag-nosis and remediation of specific flawsin reasoning are goals of critical think-ing instruction, then more fine-grained information on the effects ofparticular teaching strategies will haveto be sought

The critical thinking field is on themove. Educators are willing to supportthe production of new teaching andtesting materials and to introduce criti-cal thinking instruction in schools andcolleges Much work remains to bedone, but care is needed so as not towaste time and resources reinventingthe wheel Although there remainmany differences of opinion about thenature of critical thinking and how it isbest taught, there are suitable teachingand testing materials available for thepractitioner wishing to get a start.While adaptation for local use is oftendesirable, local districts need not plancritical thinking instruction from theground up Sound foundations havebeen laid by a number of scholars Themain requirement is to think criticallyabout the selection of critical thinkingmaterials.-]

Refirenc es

Annis. I. F. and Annis, D BI "The Impactof Philosoiphy- n Students' Critical Think-ing AbiliRn- Contelmporart Iducational1'Pcdologv - (19'9): 219-226

Ayres. I I) "Evaluation of the I se ofScience A Process Approach with Pre-School Age Children Science Education53 (1969): 329-334

Blair, I A Presentation at the third C TProject Speaker/'Workshop Series, Califor-nia State Universitn. Sacramento, 1983

Brown, A i. 'Knowing When, Where,and Hlow to Remember A Problem ofMetacogniticin IIn Adtances in Instruc-tional Ps.vc17olog' (Vol 11 Edited byhv R

Glaser. Hillsdale, NJ.: Lawrence Erlbaum,1978.

Ennis, R I11. "A Conception of RationalThinking." In Philosophy of Education1979 Edited byvJ. R. Coombs. Normal, III.:Philosophy of Education Society, 1980.

Ennis, R. I '"Rational Thinking and Edu-cational Practice." In Philosoph, of Educa-tion (80th vearbook of the National Societyfor the Study of Education, Vol. 1). EditedbhyJ F Soltis. Chicago: The National .Socienfor the Studs' of Education. 1981.

Ennis, R II "Problems in Testing Infor-mal Logic/Critical Thinking/Reasoning Abili-nt Informal Logic 6. 1 (1984): 3-9

Ennis, R Ii, and Millman, J. "CornellCritical Thinking Tests. Levels X and Z."Pacific Grove, Calif: Midwest Publications.1985

Ennis, R. I. Millman. J; and Tomko. TN M.anualfor the Cornell Critical Thmnk-ing Tests, Leels X and Z Pacific Grove.Calif: Midwest Publications, 1985

Ennis. R. II, and \W'eir. E. "The Ennis-Weir Critical Thinking Essa! Test." PacificGrove. Calif Midwest Publications, 1985

Evans. J St B T The Pgtdchologl' ofDleductie Reasoning London: Routledge& Kegan Paul. 1982.

Frank. A D 'Teaching High SchxoolSpeech to Improve Critical Thinking Abili-tn Txe .Speecl/ T(acwle- 18 (1969) 29--302

I litchcock. 1) Critical Thinking A Guideto Fiaalating Information Toronto Me-tlhuen. 1983

larkin. I. McDermott. 1. Simon. D P:and Simon. 11 A. "Expen and Novice Per-formance in Sol-ing Phy-sics ProblemsScience 208 (1980): 133S 1342

McPeck., Critical Tbinking and Fdutcation Oxford Martin Robertson. 1981

NMilgram. S 'Behavioral Study of Ohedi-ence" Journal of Abormnal and Sox-ial/Iclc/oloogy ' ( 1963): 3-1-3-8

Moll.i. N .and Allen. R D "DevelopingCritical Thinking Skills in Biolog "lour--,al ofCfollege Sctuece Teadcing 12 ( 1982):95-98

Nisbett. R. and Ross. 1. Ihman Inlfer-en'ce Strawegis. and 9?ortcornikngs of .o-cial ludgment Englewood Cliffs. NIj Pren-tice-lall. 1980(

Norris. S P 'D)ehning ObsenrationalCompetence" .Science Eduication 08(198-)H 129-1.42

Norris, S P 'The Choice of StandardConditions in Defining Critical ThinkingCompetence" Fducational Tnwon' 35(1985) 9--10(

Norris, S P. and King. R "Test on Ap-praising Ohservations St John's New-foundland Institute for Educational Re-search and Development, Memorialtniversitv of Newfoundland. 1983

Norris. S I'. and King. R Tnx Design of

a 0Citical Thinbg Test on _ApprisngtObsenations. St. John's, Newfoundland:Institute for Educational Research and Development, Memorial University of New-foundland, 1984.

Norris, S. P., and King, R. "ObservationalAbility: Determining and Extending ItsPresence" Informal Logic 6,3 (1984).

Paul, R 'Teaching Critical Thinking inthe 'Strong' Sense: A Focus on Self-Deception. World Views, and a DialecticalMode of Analysis '" nformal ogic Neusler-ter 4, 2 (1982): 2-7.

Peters, R S. "Aims of Education-A Con-ceptual Inquiry'" In The Phibsply, of Edu-cation Edited by R S. Peters. London:Oxford Universitn Press. 19'3.

Ross. G. A. and Semb, G "PhilosophyCan Teach Critical Thinking Skills." Teach-ing Philosopat 4 (1981): 111-122.

Siegel, H. "On the Distortion of theHistonr of Science in Science Education."Science Education 63 (1979): 111-118.

Siegel, H "Critical Thinking as an Educa-tional Ideal" Educational Forum 45, 1(1980): '-23.

Siegel. H. "Critical Thinking, Philosophy,and the Informal Logic Movement." Paperpresented at the Second International Con-ference on Critical Thinking and Educa-tional Reform. Sonoma State Universirt,Sonoma. California. 1984.

Somers. R L.. and Lagdamen. J. M 'TheEffect of Modem Elementanr Science Cur-riculum on the Abilint of Filipino Childrento Obsenre, Compare, and Classify Geo-metric Figures" journal of Researc inScience Teaching 12 (19'S): 297-303.

Stemberg. R J "Criteria for IntellectualSkills Training." Educatonal Researcber12, 2 (1983): 6-12, 26

Watson. G, and Glaser. E M. "'Watson-Glaser Critical Thinking Appraisal. Forms Aand B." Cleveland. Ohio: The PsychologicalCorporation. 1980

Wideen. M F "Comparison of StudentOutcomes for Science-A Process Ap-proach and Traditional Science Teachingfor Third, Fourth. Fifth, and Sixth GradeClasses: A Project Evaluation ' Journal ofResearch in Science Teaching 12 (19'5):31-39

Wolf. W: King. M L: and Huck, C S."Teaching Critical Reading to ElementarySchool Children " Reading Research Quar-ter/l' 3 (1968): 435-498

Wright, D P. -Instruction in CriticalThinking: A Three-Part Investigation." Pa-per presented at the annual meeting of theAmerican Educational Research Associa-tion. New York. 19 " (ERIC DocumentReproduction Senice No. ED 138 ~18.)

Yeazell, M I "A Repon on the First Yearof the Upshur County. West Virginia, Phi-losophv for Children Project." Thiking 3(1981): 12-14 -

MA, 1985

Copyright 1985 by the Association for Supervision and Curriculum Development. All rights reserved.