EE Toolbox-Workshop Resource Manual

TteEnvirenmeilalEducation

EvaluatingEnvironmentalEducationMaterialsLeeann TourtillottDepartmentof Enuironmenul Studies

and Planning

Sonoma State U niuersity

Peggy BrittNational Consortiumfor EnuironmentalEducation andTraining

Uniuersity of Michigan

This project has been funded by theUnited States Environmental ProtectionAgency under assistance agreement numberEPA-G-NT901935-0i-0 to the Universityof Michigan School of Natural Resourcesand Environment.

Lr

Not ono Consoil !m for

En!ifonm-.ntol Ed!.oto.

NCEEI

School of NaturalResources and Environment

University of MichiganAnn Arbor, MI 48109.1115

373 998-6726

Workshop Resource ManualManaging Editors:

Martha C. MonroeDavid CappaertPeggy Britt

tsBN 1.884782 10 8

The contents of this documentdo not necessarily reflect theviews and po icies of theUnited States Env ronr,entaProtect on Agency, nor doesmention of trade names orcommercial products const -

tute endorsement or recommendation for use.

@ Copyr ght 1994 by theRegents of ihe Universityof IVlichigan

Educators may photocopythese materia s for thenon.commerciaL purposeof educational advancement.

Prlnted on paper contarn ng

100% recyc ed fiber/2o%post.consumer f iber usingsoy based nk.

Production Coord nator:Joan WolfDesigned by lvlargaret Re

Typeset n Sabon,News Gothic, and Frankl n

Goth c Number 2

Contents

lntroduction .......2

O Evaluation Basics

AFewTerms ........4AFewKeyConcepts ...,.....5Comparing Directvs. lndirect Evaluation. . . . . . 5

O Evaluation CriteriaEnvironmental Education Goals and Objectives . 7Pedagogy .....8PresentationandDesign ......9ABitAboutBias... .........9

O Developing Your Evaluation Strategy

Step 1. Quick Review of Materials . . . . 11

Step2. PrepareYourEvaluationTool . ..... .. 72

Step3. CompileResults .....14Step4. EvaluatetheEvaluation ...... 15

O e Look at Formal Evaluation Projects

TheExpertReview. ........77TheGroupReview. .........77ThePre-Tes7Post-TestReview . ...., 18

Approach A: Provide Evaluated Materials . . . . 20

Approach B: DevelopCriteria ....... 20

ApproachC: Doa Mini Evaluation.. . .. ..... 2l

O What to Do in a'Workshop

O ActivitiesandMasters .....23

@ Resources. ........44

Steve and Anita Take Stock

"I was neryous as we planrred this workshop but the teachers really seemed to learn a lof!"exclaimed Steve, a resource speciali*. He and Anita, a district environmentaleducation coordinator, were meeting fo discuss their recent biodiversity workshop.'

"They seemed to have a good time too, but let's see how they evaluated us,," Anitaproposed as she began looking through the surveys teachers filled out at the endof the workshop. Anita pointed to a few of the completed evaluation$,'noting,"several teachers did say they were overwhelmed with the volume of matetials."

Looking over her shoulder at the evaluation forms; Steve said, l'I wonder if we mightbe wasting some trees. Can teachers'rcally use all those materials? Some of themtook so much stuff, they had to make two trips fo fheir car!"

"I remember when I was a teacheq" Anita recalled. *Most of the resources, I got at work-shops turned out to be dust collectors. I just didnt have time to sort through allthat stuff to figure out which were worth using. I sti}l have that problem!"

She and Steve decided it would be bemer to provide teachers with only the best resources.They agreed it made sense to evaluate the resources they provide, iust as they .

always evaluate the success of their workshops.

"It seems like a big task," said Steve. *I've read some articles in:theJournal of,dwiron-mental Edwcation (JEE), and all of them stressed being very systernatic aboutevaluations, developing a rating scale, and setting up a eommittee of teachers.It seems like we should get someone at the university to do it."

"I dont think we need to be that technical absut it for our own personal use,l' rypliedAnita. "I'm sure we can combine our knowledge of triodiversity and educationwith some of the information from thoqe JEE atticles and come ug with anevaluarion strategy that works{or us and our workshop participants.'1

"I don't know, Anita. Do you thrnk we'll be able to predict teachers'needsll' Steve asked.

Anita thought for a n.loment; 'o,I have an idea," she said, !'I$flhat if we build sorne time' '

into our workshop for teach€rq tc leam about a couple of evaluation scales.Then, using those scales a& examples, they can select or create their own criteria,scales, and evaluation methods.r'

"Great idea!" Steve added. "Yre can figure it out and then pass those skills on to teachersand they can improve on our work. That way they can apply evaluation to alltheir EE materials." Anita and Steve agreed to,do some research on their ovrr1.

They planned to meet again to evaluatitheir materials and begin developinga workshop session on evaluation. ,

6vaiuatlrg Environmental Educatian Materials

Introduction

NE oF THE REASoNS TEACHERS ATTEND INSERVICE PRoGRAMS iS tofind new instructional resources. As inservice providers, we respondto this need by distributing instructional materials and handouts.Walk into any environmental education (EE) teacher workshop and

you will see the walls covered with posters and tables piled high withsupplemental resources containing content background, activities, and newteachingtips. EEconferences have rooms full of exhibits displayingnewactiviryguides and other instructional materials.

Often teachers have no trouble finding EE materials, rather formany it is a daunting task to choose from all that are available. Teachers oftenhave neither the time nor the confidence to sort through and choose the bestinstructional materials for incorporating EE into their curriculum. You can helpteachers increase the quality of their lessons by providing simple and effectiveways to evaluate EE instructional materials. Furthermore, evaltation canfamiliaize teachers with environmental education concepts. Teachers canmake beffer choices to meet any number of objectives and individual needs,such as:

- meet state and local learning objectives

- address environmental education goals

- convey acclJrate information

- include relevant context for various socio-economic groups

- relate age-appropriate concepts

- adapt to different teaching sryles

- accommodate the different learning styles of their students

- select interesting and easy-to-use materials

- fit individual constraints, such as time or money.

All educators can gain the skills and knowledge to evaluate environmentaleducation materials.

'We will provide you with background information,simpletools, relevantexamples, andactivities. Youcanusetheseto evaluatethematerials you present in workshops and to lead activities that build teachers'evaluation skills.

Workshop Resource Manual

This unit focuses on evaluation of written environmental educationinstructional material, most specifically activity guides. However, the criteriaand techniques discussed here can be adapted for use with many types ofinstructional material such as video, sofrware, and texts.

The sections in this unit focus on the following steps:

Eualuation BasicsBackground information puts the evaluationprocess in a larger context, helping you tobetter understand evaluation procedures,guidelines, and criteria. Included in Section1 are basic terminology and key concepts.A more complete glossary of terms is pro-vided on page44.

Eualwation CriteiaThe heart of any evaluation effort is thecriteria used to rate the instructional mate-rials. There is no single answer to whatmakes a good activity guide. However, thereare several aspects that every EE activityguide should include. Section 2 providesa framework for developing criteria andsuggests specific criteria relevant to EE.Use this information to generate a listof criteria that meets your specific needs.

Deueloping Your Eualwation Strategy

Just as no single set of criteria will meetall evaluation needs, no one evaluationstrategy applies to all EE activity guides.Selecting a strategy can be a confusingtask. This section (pages 11-15)demysti-fies evaluation strategies by providing bothguidelines for modifying existing tools andstep-by-step procedures for developingyour own.

A Look at Formal Eualuation Projects

Just because an activity guide has been for-mally evaluated and approved by a partic-ular organization does not guarantee itwill satisfy your needs. The guidelines andreal-life examples in Section 4 can helpyou evaluate methodologies. You can alsouse this section to understand some ofyour options if you are interested in devel-oping a more formal evaluation project.

Most inskuctional m*terialsmay be useful even if they donot rate well on all criteria. ltis best to focus on strengilrs,weaknesses, aild usBfulnessin your evaluations.

'What To Do in aWorkshop

Teachers are more likely to evaluate materi-als if they receive hands-on experience inyour workshops. Section 5 provides ideas,tips, and sample agendas to help you incor-porate evaluation topics into your work-shops. Several strategies are provided tohelp you choose an approach that willwork best in your workshops.

'Workshop Actiuities and Sample ToolsSection 6 complements the previoussection by matching the workshop ideaswith specific activities. Each activity isdesigned so you can use or modify it foryour particular workshop.

ResowrcesThis document may leave you wantingmore information, ideas, and details. TheResource Section provides a bibliographyof evaluation projects and additionalreadings.

Evaluating Envtronmental Education Matertals

Evaluation Basics

IKE ANy orHER DISCIrLINE, evaluation has unique terms and con-cepts. This section presents some of the basic tenets of evaluation andenvironmental education. Understanding this background will helpprepare you to adapt evaluation criteria and methods (described inII and III) without compromising effectiveness.sectrons

A Few Terms

Terms such as "curriculumr" "instructionalmaterial," and "activity guide" are oftenused interchangeably. Perhaps they havevery specific meanings for educators inyour area. To avoid confusion in this docu-ment, we use these terms very specifically.

A curricwlum is a complete course of studycarried out over the semester or entire schoolyear to meet a specific set of learning objec-tives. The learning objectives may be estab-lished by teachers, school districts, stateguidelines, or national standards.

Instructional materiak are resources thatteachers use to address the curriculum.Such resources may include texts, activityguides, videos, and other materials designedspecifically for classroom use, as well asnewspaper articles, movies, comic books,and other sources of information on envi-ronmental concepts or issues.

Actiuity guides are one type of writteninstructional material that often containbackground information, student activities,and suggested lesson plans. Many activityguides cover specific learning objectivesand are designed as supplements, form aunit, or make up a course.

Eualwation of instructional materials isthe systematic application of criteria tojudge whether the materials will meet theuser's goals.

These distinctions are intended to make onepoint very clear: most instructional mate-rials should not be evaluated as ifthey will make up an entire curriculum.As part of a larger picture, they may beuseful even if they do not rate well on allcriteria. Hence, this unit focuses on identi-fying the strengths, weakness, and useful-ness of written EE instructional materialsfor the specific needs of teachers.

Another set of terms is used to describemethodology. To help you meet yourgoals of developing a simple and effectivestrategy, the following sections outlinean informal approach with the followingcharacteristics:

Atmchair Reuieu: F;rrlpllasis is on anindirect evaluation of materials (beforeyou use them) to select those that best meetyour needs.

Teacher usefulness: These straregiesinclude criteria based on goals of EE andhow well the materials meet specific needsof teachers (vs. the goals of the authors).

Limited scope: The criteria and strategiesin this unit are best suited to evaluateEE activity guides. Some criteria may notbe appropriate if you are evaluating othertypes of instructional resources.

Single or Comparatiue: Crjteria canbe applied to one actiyity guide or usedto compare several guides.

Wo r kshop Rev:u rce M a nu a I

A Few Key ConceptsYou may have an intuitive sense of whatworks, or know which type of activities areusually popular with students. Yet, if askedto provide an "evaluation," you may notknow how to support your inruition.Is it enough to know whether activitiesexcite students? Is helping them learn oneor two concepts sufficient? You may findit difficult to define all of the factors thatmake an activity work with students.

One way to evaluate EE activity guidesis to think about your goals as a teachereducator. Your role is to motivate teachersand help them develop a good knowledgebase, skills, and a desire to bring EE totheir classrooms. In measuring the successof your workshop, you may gauge whetherthey became more motivated, knowledge-able, skilled, and active environmentaleducators. Likewise, evaluating EE activityguides considers many of the same typesof elements: reactions, learning, behavior,and results.l

Reaction Eualwation measures the users'immediate positive or negarive responseto the learning experience. This is themost common level of evaluation. An arm-chair review may predict how students willrespond to the material.

Knowledge Eualuation measures whetherusing the activities with students increasestheir knowledge and awareness of environ-mental issues and influences attitudes.In an armchair evaluation you can assess

the quality of the content and if teachingmethods meet your students needs.

Behauior Eualuation measures whetherthe learning experience alters studentbehaviors. Your armchair review coulddetermine whether the materials providethe knowledge and skills the learners needto be able to engage in new behaviors.

Results Eualwation measures whether thebehavioral changes have an overall impacton the environment or students' lives.

It isvery hardto measure whether an impacton students' lives is due to the instructionalmaterials or any of a million other poten-tial factors. In fact, evaluating for results isoutside the scope of most informal evalua-tions and will not be discussed in this unit.Let's explore how you may achieve reac-tion, knowledge, and behavior evaluationin an armchair setting.

Comparing Directvs. lndirect Evaluation

A direct evaluation of EE activity guides iseasy to envision. Teachers can gauge studentresponse and use a variety of assessmenttools to find out whether students learned.In the context of an informal armchairreview, you can only gauge your responsesto the material. However, you can still adddepth to your evaluations by focusing onthe likelihood that the activity guide willelicit a positive response, help studentsIearn, and change behaviors.

Direct evaluation usually yields a stron-ger certainty, but it is difficult to evaluatemore than a few activities.'$7ith an indirectreview youcan critique many activity guidesin one sitting. The rwo scenarios in thebox on the next page illustrate how youmight indirectly predict which guides willhe useful.

€tllll:1el,.. lll:i1i1.

r<|lurliii;llIli,iiilia'iiiijiYl;l;lll:ia0lliiii6rrl

i(er,,:illl!lllrlaiiii:lrri:ll'i--l1t:-l"j .\-',1

$ielXl*liiI;rtrr;r,rrr,rat..,iieut:,iii4u,,.

-.i\ .t-\ -

::ll,alll'r.iiili*l:'riiii$lr,l

rrtY..r,lli&,,:rll0lll,::rla, :'iiffititiiprt:tiilrlli$ll\.iii11iS,,,,i9lll:,illi

ll;arr11g;,,,,,t,'.,,

ir,::1le,,,rr\?,,,,1i:lF 1.. _,. |

",riii:ii,bilr'"i]l!illir!rl;:i

"1"2,ll:,rlgr,,rrll9lr,,rrrlll:lra;i@tu':iiiiiiut:iitrt:i

];-616ll&lll'r

ltgrlr.li3r,,,,,liii9ll,lnil

'11- 'r ^--/-I- --r

.y.-.:,.* .

illti!riiiiietu::,iiiicl'. @))):.:..::::.9)::*::.9t :

i... rry-rl!,):.:!l

'l*- - t" t,

,i]iii:ll:ltt6irrl'1s;rrr:.iiirStlriiltlliiii!i:i1,Stul

al3'rrrai'rriri:lll:rrir;;l

llr;tYll:l:tiili:l};iiiii'l

ffiiiiilillll,, :!elrll,.i:llrl,:{ilr:

tllll ilai!U:.,i1f)1r..\llgl

1,,,.:19:r:ll:!rr.iril!:l

lllgrrr*:rrra3:rrriii

L,:lll!9llr;lli:llli,, r:i.

,lt-: I -

lrrr,llcllrliliqlll,,l :r'

"*\ -l *

:lllllllrriia:lrliii:rrrlri{9u.iliiii?1,.4:l,lt:*iir'

ilrllrlrlrriii6ll:*a::'

tilrl6rlii;llxlt\-, r':f e - ; ---:

\Yiiiirrriiiid! j:iiiie)ri,in]r,l-'iilelrr:8tr iil

;ll::iieiulii$l:ii@!G",:.'i,16li iiilgrl-,

, '*r:t -':

.6l3...8.rr,,Xi:;r,.li!{llitull'l'iiAr:ii{

;\:i \-l1)"",::

iri rtelrli6illlidlllll*lii,:xiiiaut'tiiiidt:i<ltrl

,rrr?lt*rlrXrer*lr,al

lrrreL,rYrrr*el6qil|l'iiiiaut::iiiiat, i

.-i - . \"n2*/ s f.::lir;iiir:irl'}]l, iiiir@rrr.:t 6-itg) 1 -l

?Qllr,.::rlglllrA:li;li

:i9;ii:qut::trru:rt,alri;i|:a,r,,:r

1:;U:,-iiiiquu. illliiri.l

i llgraiiiSir;iir$?lr;

-1: *!. q "

:ii4,r,,.,€u!,19]:rr,?*t"ti€ul'iiGut.(i3lriklllY:6llll':hll:l: .f { /

iGlirriirgrlr..,irlrCDrrr..l

Eva luati n g Env i ro n menta I Education M ate r t a I s

| /\'allr?irrallr'r:rlllr,r:ll'll'ii:

t- ( -*" .--

x.rrr:iiir9rt.., l rlrit:,,, :i:i

-- 1 \ "..) I

iiiiiirr,..:30rr'iriii9,

:' t a -\6l:l":riiibtilllliiJr:liri

itrlurlirrrtArr.(lrtrrl

.tiltllrir;r,l|rytr,,,:11llr,lrt;lrylr,:,,ii!l3llll,,::iri

irrl&irrr*lllii{rrrrlliiG

ii&rrlr..iiiiilg]]]]'::iii]l,.;

rrcll,,,llll{rrl',iiIrrr 1f ""/ \'

.rsr,,,..iiel:, 1;lllr.!,,-..,19r'. i,iia:,tr:iiilj

illr9:ia,rrr;rllf :rrrlljl:;:

,:.1iru,liillrrliila

1'\,\.l:\:|Ulr. (alrlrrrd9L, \ :.'ii]rlrilrrili'iti:::;

iirr''ri:iall[:]lt:ii:':;.',f ll{.'irri..{rlllllir,{llll;Sr9:i::,iiila1;ril

'iiiiiili'r'!lrl-:ii€1,,.1$

S!l:,.,lliliSl,":rillilll, ;.t- . /r;'iiiorrr)iiiisii j{rrlll

;f lilrd: rrrerllgllll.ilil9lli:lrl.irilVl, irml: .!r.\tlllt,..,{Atl:r]a;iillll::1111t91."ri:i:;ll"'irii

5ii,,riid:rt-rgrrrrr:'i111S' illll.liiii!:lrrrl.ri:

...i!;llirrrr'rtil risrrt.r:{ta'iii,i!iiiii:,1:iii!:,rrl

hiillglli,.iiOlllllr.iiirll;

:!,,llor1iiiii)r::':l?

;4ll lllll6ir9,rlJlolll

iiillgll]a,liA]'.ili9rr.

;lliSlliiAlrlrr..rrgrrlr.'iiiiarrrrl?.il9rrt,til;lrlraiallll!lrr,,,,lltlrrr,,l,lr:.ai:r.ll1&,..,ll1lrl

;':ill$;l:rllrrLrr,!t,r

i llll:lrr,,.,iliglllr",illl1lll

;i:glrlrlilldllllll i iiiolllll

,.* i. : ', ,r i , / -,I "/ \ -

€llrla9lllr'::iiliiiliiili-r- r t' ,

;llY::i;:lii:"1!1,,::Yll,,ll,:tillll?t:.ittr4trt,,,it!

illl9ll;tlil$lll".r.irii, I'

:iiii$ iiiiialiiiililrl^iii;(:llrrrrr.,irrrlglrllr' iiilllllf

l,r\,19!rrr.iriililll'::iiilil

1i 2 \-'''1@lr iil6lll'l::riia:

' s r-\

iiiiiarlrl{ryrl,,siiYlllr:,ll;lllrf l]ll::rilll9].it;:i

rll,;.iii{l1:,1<riiGi:.,,t1@

Xil9llXrerr:irrl*-'"\ -:a\

.irrt?rr,iir9uul,:li:gru,.:lllO,rl,e;:llllauuli

lrll:,,rl1itllllrisru'llittiettrqii*rr,,,irtrt::erri.

ii;lrlsll:Giii{!lll::.ti;9i

? /r\ -

'..iiibr,rr...,ii6irritll,rr i

;il?llrAilill.lliLitrlS.i;tl

tllrtr.ltgt:.a@uu-.}

Examples of Dircct Evaluation and lndhect Review

Layton and Miguel have each received a new activity guide on recycling.Layton, a high school teacher, decides to evaluate it by using it with hisclass. Miguel, a resource specialist with a local governmental agency, doesan armchair evaluation before using it irr his next workshop.

Layton's Direct Evaluation

Reaction: Layton noted students'general level of enthusiasm, atten-tron span, how many questionsstudents asked and how hard it wasto encourage discussion.

Knowledge: Layton started his classsession by having students wnte ashort essay about the impact of solidwaste in their lives. He comparedthe level of knowledge in the essaysto test $ores from a post.lesson quiz.

Behavior: Students asked permissionto set up aclassroom recycfing project.Layton noied whether the unit pro"vided them with the motrvation andskills to accomplish this task.

Miguelb lndir*t Evaluation

Reaction: Miguel listed the attributesof fun learning experiences. Hereviewed the activity guide for suchfactors as providing a variety oflearning situations, relevancy tostudents in his area, and hands-onactivities.

Knowledge: Miguel reviewed theactivity guide to determine if keyconcepts were present, how wellthey were addressed from a varietyof perspectives, and whether prob"lems and solutions were presentedso students may identify appropri-ate roles for themselves.

Behavior: Miguel looked to seewhether activities presented infor-mation on a broad range of feasibleactisn strategies and included activ.ities directed at building skills stu-dents would need to engage in suchactions.

Wo rkshop Resou rce Ma nu a I

RrrERrA ARE THE HEART of any evaluation effort. Now that you have

a sense of what an armchair evaluation is, it's time to start defining

There are different categories of criteria you may consider for yourevaluation. Educators tend to focus on the quality and accuracy of content.Learning sryles and presentation of the material are also important consider-

ations. Many EE evaluation efforts measure against the goals of environmentaleducation Often teachers use state or local education objectives to define the

content goals. This section provides a framework for developing criteria based

on three overlapping aspects: environmental education goals, pedagogy, and

materials design. Together, they can create a well-rounded, thoughtful evalu-

ation. After you identify relevant criteria, Section III will help you place themin a usable format.

As you read this section, remember that the focus is on EE activiryguides; there will be categories or specific criteria that do not apply to othertypes of instructional materials. For example, you would not judge a text onclassification of trees against criteria for involving students in action strategies.

Evaluation Criteria

Environmental EducationGoals and ObiectivesThe ultimate goal of environmental educa-tion is to establish an environmentally lit-erate and involved citizenry.2 To meet thisgoal, it is important that students not onlylearn environmental concepts but also beable to define environmental issues withina social context, understand multiple per-spectives on an issue, and understand theirroles in both environmental problems andsolutions. (For a more in-depth discussionon the definition and goals of EE, refer to"Defining Environmental Education" inthe Worksh op Resour ce Manual.)

Obiectivesof Envircnmental Education

Awareness

Knowledge

Attitudes

Skills

Participation

from the Tbilisi Declatation, Unesco Canfercnce, 1977 3

Eva lu ati n g Env i ronmenta I Educatio n Mater ta I s

Enuironmental Ar,uareness and Knowle dgeTwo important factors that you can assessin an armchair review are accuracy andbreadth of content. Teachers not familiarwith an environmental topic area mayneed help in developing content specificcriteria. You can start by narrowing thetopic into 5 - 10 key concepts and developcriteria that address these concepts. Youmay wish to team up with environmentalspecialists for this step. Some importantcriteria might be:

- terms are defined

- environmental concepts are setin a social as well as an ecologicalcontext

- ecosystems include individuals,the local community, and the globalcommunity

- environmental issues relateto specific environmental concepts

- information contains multipleperspectives (or bias is declared)

- concepts are relevant to learners' life

- depth of information is relevantfor the intended developmental level.

AttitudesThere's no way around it, environmentalissues and the solutions we choose arevalue-laden. Attitudes about the environment arebased on individual and social value systems.It is important for an EE activity guide tohelp students define and understand theirown values as well as understand and valueother people's perspectives. Students ideallycome away from a lesson feeling positiveabout participating in environmental issues.Considerations of attitudes include:

- activities encourage a sense of personalstake and responsibility

- materials express a hopeful outlook

- materials contain positive role modelsfor discussion and action

- activities encourage students to clarifytheir own values

- environmental topics are covered froma variery of perspectives, and no one per-

spective is predominant.

Skills and P afticip ationStudents are more likely to become activein environmental solutions if they areaware of the range of possible actions, areable to evaluate them for appropriateness,and have the skills necessary to implementchosen action strategies. Successful EEinstructional materials address action strat-egies in several ways:

- environmental problems are presentedwith a range of possible solutions

- the material compels students toconsider the implications of possiblesolutions

- acdvities help students gain newproblem solving skills

- action strategies focus on win-winsolutions, whenever possible

- activities provide students withcritical reasoning and problemsolving experience

- borh individual and community actionstrategies are presented

- acrivities help students gain communica-tion skills, group skills,leadership skills,and other interpersonal skills necessaryto become involved in solutions.

PedagoEly

Pedagogy, or how the material is taught,is an important consideration. Good envi-ronmental education is often said to begood education in that it emphasizes inter-disciplinary, experiential, relevant learningand focuses on preparing students to par-ticipate in their community. Because envi-ronmental education is as much a processas a content area, many pedagogical crite-ria overlap with the environmental educa-tion criteria.

- a variery of cultural perspectivesis included

- content excludes gender, rac% age)and other biases (for more informationon bias, see page 9)

- different learning styles are included(for example, hands-on, student-centered, research, lecture, and othermethods)

Wo r kshop Resou rce Ma nua I

- activities allow students to build fromprevious knowledge

- materials offer srudents rhe opportunityto pracrice critical thinking processes(such as hypothesis making, collectingand organizing dara. analyiing, drawlneconclusions. and problem solvingl

- activities facilitate and encouragecooperative lea rning straregies

- teaching methods are appropriate to theguide's goals

- content pertains to state or local learn-ing objectives

- materials can be readily integrated intoestablished curricula

- appropriate student assessment methodsare included

- learning opportunities are conductedin a variety of settings

- lessons include suggestions for extend-ing or adapting the learning experience

- the materials can be easily adaptedto varied learning environments, suchas large and small classes, mixedlevelclasses, or rural and urban settings.

Presentation and DesignIf a guide has illegible type, few graphics, abinding thar disinregratei, or an ixorbitantprice, teachers may never discover that thecontent is good. Evaluating effectiveness ofmaterial design takes into account such ele-ments as text and illustration quality, andthe guide's organization.

- the goals and objectives of the materialsare clearly stated

- teacher instructions are clear

- guide is easy to use and understand

- handouts and other masters areincluded and are easy to duplicate

- the writing is clear and interesting

- illusrrations are appropriareand engaging

- required activity materials are suppliedor easily obtained.

A Bit About BiasTeachers should address the issue of biaswhen evaluating resources. Each of us hasour own biases developed through ourexperiences. In addition to our personalbia_ses, we are confronted with many formsof bias every day-in textbooks, port.rr,newspapers, and on television. Industrvand environmental groups are most oftenaccused of presenting slanted information,

Frcshkills, Staten lsland is theworld's largest landfill. lt serves7,5 million people and won,t hfilled for another 1O years,Thirty-five similar landfills couldserye the whole United Statesand occupy O.OO5o/o of the landin ihe lower 48 states. lt wouldtake more than Z,OOO cenluriesfor the United Sfutes to he buriedin its own solid waste.

* NYC Department of Sanitation

Althowglt tbis is factwal, one can detectbias.-Wbere 35 enaftfious landfiltsu,ill be place$ tbe cast af truckingwaste froru one stdte ta anothet andthe wisdam of bwyingresourcesfl.t currefit rates arent mentioned-

but almost all forms of informational mediacontain some type of bias. Many are verysubtle and merely a matter of how ,,facts,,

are derermined and presenred. \X/ho paysfor the material often indicates what'slanrit might take. For example, textbooks tendto include the same mat.rirl, edition afteredition, because that is what teachers andcurriculum commiftees expect and there-fore will buy. The bias is tbward statusquo and tradition, determined by sales.Other forms are more obvious, i.rch a,language that portrays leaders, scientists,or active individuals as men, or illustrationsof domestic roles portrayed by women.

l.R' L

?uiitut:5ri;{r!::rl::,,i.:j,,.]

:Jt?,iiilr;Jli$ii:,!:,*(,jtr$dt;i]:{::"(:,it:y:.::::'+,:,etr. \ J:l..z I

*'.::a::.P:::,*,a1y..,4::)ia:::,i;,.a.

I3r',, d,t;t\J,S3;lll.,ll:: ri.a'iili:l

;:11;'j,:i{},,li;],,r$t,,',.('l*i{ii;J,;;;ill E-:"riiliiiarru:.1:g;.:::,:,:t:::aa.;

,u\ t.li;l,'J**i$rii*siill,:rrS::)llllS;riIllllr''idr,i'a

:,,,SiCi,&rdtlt.33r&i',,4.f ".;liii(rl!l:a:il)i:i:'::::@a...,t:::.)"-.": { l

le,iiirrf.:tir}r,iyiia:'iatr,iii;i,,1

ls.l,ii1st9?il::i::,

:arAzrlili1rl,iFrr1;,;;;

i3,,1Sl'ii,il,,i;l:{:re,t&t,i'.1!:':::::a?:a).:a,.{.;

N- r: \t i"'' i

iyi€rr.!tlrriit;,,

I#3e,e,

J:iirtr;,,1,f#

:Jt?,iii3r;J1i..%i

i:,!:,*(,jtr$dt)::{:,"(:,it:Y:.::::'+,:,etr. \ J:l..z I

*'.::a::.p:::,*,a

Yr:1llr?::;ii';

;lll.,lr:: ri.a'iili:l

114'j,:i{}'rr;],,r$t,,',l('l{:i{ii;J,;;iill {1"riili,iiirrl,r':::,:,:t:::Aa.;,u\ t.l

:lliii:,ivaiQ]]Jr,:'(l],,,r: -

;rii*si:::t&:,iliji

TP.ai:&:;,i3,r\i,,,:F.f " .;

,iitl<elrriiilt,rit,

9,'::::irNl:,:{,:.:;a(1a:l,tiiitrid)1.r.,....:.141):::i.;a):.

,iia:'iatr,iii;il,,l

? "* ';r{,et,{llrrAutii,;,jtl,:l!lt:ii!urAl,,ii).1, -- I

ii:!i:,:trii':r,i":::t:a?::).a:,y;.

N.LTl--"'';,.:. t{.:..V..:i*)):.)

'S|r:l1t&t$,.j3',&..:,ta,,S,l

t:Y:.::::'+,:,etr. \ J:l..z I

*'.::a::.p:::,*,a1y..,4::)ia:::,i;,.a.

I3r',, d,t;t\J,S3;lll.,lr:: ri.a'iili:l

;:11;'j,:i{},,li;],,r$t,,',.('l*i{ii;J,;;;ill E-:"riiliiiarrll:.::lr.:::,:,:t:::Aa.;

,u\ t.li;l,'J**i

$rii*siill,:rrS::)llllS;riI llllr''idr,i'a

:','SiCir&riillr,33,r\i,,,:4.f ".;.:rrrr(tl!l:i:ill,ir;

:i:'::::@a...,t:::.)"-.": { l

Slillii.ttl:e?1r.ll3rrriidri,,

ig..rrl(9,:}:l::i::,

4,,:azilit,litrliFlrrliltrt,,.ti)

3,,1Slrial',ilrril:{:rei]*.]ttil!:':::::a?:a).:a,.{.;

N- r: \t;" '' i;S r1tt'-,,s ,,:li,t.t*r,S,l

ffit;s:j;i\:,r!,rrl::iil:v]ra,trrri-.5 t r:

r;:at::ii!::ji;ll:,ilrt:i!u,:ii;:rrt jvua^r,,;j)rrip,@r,,;:U:it:.

,telr{llar-,)-r,icriiqliArt

TLff:te:,.t,i.ir.ira:;1;rr,Brr.,:l;-lI:'riii?air...a'iil::L::,,:rS.,1li'iiiSrir:tEr,,t,l,,,iir,dtl,;!rii1Ial,tS,,;:at.*)::21,,*....tr.rna,rg:illtr..:l

t:':::ri::ii1!l,,riits.,0?,,rr*:r-.\".;

:::?hL;dl;:t

i.l.r' ',t],1lite?ricr,r:illr:il":ii1l'i1;S.!ritr,.i.4i:!],,;lllliid:tliilr^r, jl

i','iBe.,,,i$:;$,,t9,S'1

?:+:"-)

!riadsi$,lli,::l!(l'rrrii,al-'iir;all4:::eaaaab;,,:;

:11:irltiiierldrl

.tJ;S!i,';n

l,itrra,:..r,1;rt,

;{.,::i.,:,:,,:):.

..:?:::a.t ;),..lg:'&:ii:),1

'rllJ(Pl:di:,!:,,.Jjtr{r;:9:,ii:i,-+.rr,reurr&lrur.r

;L;1i{*'1:!a::ail:::,:

rzat&:f/ --

i!:::rllt,,:i',,.\

*i&,J;,il(l3tr'ri!;r,::y,aaa:::a,:&:::a

,u\ }

i,l;,ri:rc,:xa't::t!::e..

P/;,fif::::t,*;4;

Y\4.tr.J,.Yt\4.t t

i:iii(ll!l:i:ill,:.rS::iilrlitrii!

ie,iltiuia:tir):2tll:'1Li;;'? ax i

l:itlSrs,errr';e "iiarrrid,,,;t,arrtAutilt;l

' F - -.i1l,,1*l'.*'.;l:{:re,]i'.Y":::.:::?::).a.k

N- i:t i"'"

,yi€rralllrriit

l'?!?!Q,

. -/'^,*.:E;'I",*\-lr

4 v.)i:rriArrr?rtl',Yil!:.ttaLtt:ai;;;)...ta{.::i..:\):. ;:,1

i,iletaiuat;

, t "-t.<.t - ;'ar,.?rr.j;-la

i rt i.l...ll...:i?!:).:.\...

Elnl]]rg:ill

;:':::ri::iil:jl. ,. t I Ia r - s.i7. F , ":

:. -1"

. "\ -t'I"l-/... :

V.!,,::,.:L::a']),!:.::il:::l:,*,.,;:1,

;!;,$,l:ri,,11r

, .I ,;:i'diirYaa.:ti..i

,--:""t':) " & *i'\' a "ii-, i;;rJ;s*r

!+riidrr,.ir',s..rrrar?ur&,,r,{ul

,,t3',:Y,tSrli?{:''':a.t a.::?a::t o::.,i)..g::'€:airf,)],:

ai:i3:C,a(a:,))dr a \ t

lrr:d,iltratrt)1,.:iilj:lliiieit-.'Pri*:.'.\'r ll, --l?r:e:'lnl,id),iliidrrlprriii;u.,ii!lri!rrral,iii,];;t;

\ --'it.i 'l:'' njt"...i.)' r'"'-"'-\--I n/

i't,,"r.|*, ... iq@r,in,iil*t,iirat,?rr,&iir;j

,€rr::Y,riAlr;t. j: j::r*lr4rrtirdlaX

i..?.,t::l

::ill:)!{:i,J;

;6lrS,rt

!t,,llti.IIll,tdt,4):,ta

.,.:l!t::lti

1:ielqv;Y;ilrllell

drrrdll

;,3:id;l;llt,\];,":dt.:*l*u,,,,tL.i

:e)..))a:.

ie{vi,t,,;ll:;

"iit!'(g

l;.r:*tti..h,;i:iirl,,)trl

:iit,:.ir\w?::L::.1

,,11Y,::t?:

)..,*t.N

iirr*l::;

l:::d;;,riii{,itl

c],iift,tl

t,,,ii1\1!lii9."

(diierrit:?!):t<

:'';Y:rrlrrillrrri(

a::.&::.P

:iil(iwr,i1l'rsiif]i9,(

.:11.::l!.,

lrt,CtrA

,il::;:,:::

\a).tlLi

;},;;i;

[;;,,

):::ta!).na

,jldi

r4la{Xst:iil

r,ilJl;lll,!rt!

!:.ln

liililiin

t:lric

i.rr,illr.ia::l:,,,:i:];llifirl

i,:hl

rre

dlt,j

iii!r:i,l

Eva I u ati n g Env i ron menta I Edu cati on Mater i a I s

It is important, therefore, for educatorsto acknowledge that we do not live in aworld free of bias and to facilitate stu-dents' ability to recognize and address biasin their lives.

Recognizing bias is the first step toapproaching issues (especially value-ladenenvironmental issues) with an open per-spective.'We each have our own bias; com-paring different materials will help identifyit. Activity 5: "'W'hat About Bias" (page

43) is designed to help teachers see bias ininstructional materials and discuss how toovercome it.

Some types of bias relate to:

- race

- culture

- gender

- political views

- economic status

- different stances (e.g. industrialistvs. environmentalist).

Identification of bias is a critical step inevaluating materials. You may not want toeliminate all biased material. Such materialcan help students understand what bias is,that they are affected by bias, and that theywould, therefore, gain from identifyingtheir own biases. Educators can deal withbias in a variety of ways:

- not use the material

- accept the bias as is

- balance the bias with other materials

- point out the bias to students and havethem discuss and analyze it

- help students develop critical thinkingskills to identify bias on their own

- modify materials to reduce bias.

Workshop Resource Manual

Developing Your Evaluation Strategy

ow Do you BEGIN to deal with all these criteria and how do youkeep track of them when you are reviewing activity guides? Indeveloping your strategy you will prioritize criteria, choose ques-

tions that address these criteria, devise away to apply the criteriathat is meaningful, and keep track of your impressions so that you can compareseveral activity guides.

Teachers will benefit most from a streamlined evaluation process andsimple tools. This section, along with the evaluation tools included in SectionM, will allow you to develop an evaluation strategy that meets your own andteachers' needs. Section V provides several ideas for helping teachers person-alize this evaluation strategy in a workshop setting.

For ease of understanding we have presented the evaluation process as

four steps: Step 1, Quick Review of Materials (p. 11), Step 2, Prepare YourEvaluation Tool (p. 12), Step 3, Compile Results (p.14),and Step 4,Evaluatethe Evaluation (p. 15). You will probably go back and forth between steps toassure your strategy meets your goals. Keep in mind that you don't have toreinvent the wheel. Use this section to help you modify existing tools to bestsuit your needs. (Three samples are provided in Masters 3,4, and 5.)

)i$i;:t,iA)',,l6rr1eui::re

iip;$

lid:i.lr!i€,(6t!srt::is

)l;itf,.iii6Dr;

Step 1Quick Review of Materials

This first step is a fast way to get aninitial impression of the materials youwill be evaluating. You will most likelynotice prevailing presentation styles,glaring biases or omissions, and age-appropriateness concerns. A quickreview is not necessary but is usefulin the following ways:

- To categorize materials into contentareas, age levels, and other categories.One set of criteria may not be appropri-ate for all types of materials; you maywish to refine your criteria for eachcategory.

- To reduce the number of materialsincluded in your evaluation. Once youhave categorized materials, you maywant to eliminate some categories fromyour evaluation. For example, you maydecide to evaluate only those materialsrelevant to middle school social studiesor to water quality issues. Be careful notto discard items because they are notfamiliar or do not match your particularstyle and viewpoint.

Evaluating Environmental Education Materials

- To reuiew the material with an eyetoward deciding how to apply criteria.For example, you may find that allof the materials fit basic EE goals butnot in the same way. This would leadyou to ask how well each of the goalsare met.

You can use the questions that follow tomake your initial cuts. Questions of thistype can most often be answered with asimple "yes" or "no" and still providegood direction about whether to take acloser look at the material.

- Does the activity guide focuson a desired content area?

- Are activities age-appropriate?

- Are the activities appropriate for thelearning setting (for example, outdoorcenter, classroom, factory visit)?

- Is the guide readily available to teachersin your area?

Picking StraterySometime after they discussednext workshop, Anita and Steve

developing an evaf uation session for theirgathered their teacher resource materials.

"No wonder teachers don't use all the resources we give them. No onehas enough time to go through all these materials!" Steve sighed as hepushed aside a stack of activity manuals. "l know some of these are greatresources only because I've used them, We need to have some system thatwill help teachers make choices without having to test each one."

Anita glanced at Steve's stack of resources. "OK, it's obvious we don'tneed some of these materials for a biodiversity workshop. This one doesn'taddress biodiversity and here's another that only has activities for early ele-mentary children. Let's weed out the ones that don't fit with the workshop."

Anita and Steve began sorting through the materials selecting only thosethat pertained to biodiversity. After a few moments Steve noted that he hadfound quite a few resources addressing biodiversity but one was a comicbook, two were videos, several were activity guides, and some were simplyinformational pamphlets. Using the same criteria to evaluate such variedmedia would be difficult. They decided to narrow their search to activitymanuals that dealt with biodiversity.

Step 2Prepare Your Evaluation Tool

There are tvvo ways to approach thisstep: use an existing evaluation tool ordevelop your own. While it may be moreexpedient to use an existing tool, you willwant to modify it to meet your specificneeds. Activity 3, (page 28) providesseveral sample tools. Use the ideas underapproach A as a guide to modifying thesetools for your use. Developing your ownevaluation tool may seem like a dauntingtask. Approach B breaks this task downinto substeps to make it more manageable.Remember, the shorter and simpler yourtool, the easier it will be to use.

No matter which approach you take,remember that the easier the tool is to use,the more likely your results will be easyto understand. Here are some general rulesof thumb when modifying or developingan evaluation tool:

* Format so that the page is easy to read;guidelines are easy to follow

- Make it easy to tally answers

- Have questions relate to one criterion

- Keep it short

- Do not use double negatives

- Do not ask rwo questions at once.

A. Modifying Existing ToolsMost evaluation tools were developedfor evaluation projects that meet the needsof alarge and varied audience. The criteriamay be broad and the rating scheme com-plex. Before using them, consider adaptingthem to reflect the specific needs of theteachers in your arca.If you are a bituncertain about your teachers' concerns,you could meet with a few teachers or talkto a district curriculum coordinator. In ateacher workshop you may want to brain-storm what teachers need from EE materi-als (see Activity 1). To modify existingtools, you may cut, change, or add ques-tions, and modify how questions are rated.Here are some guidelines on modifyingboth criteria and rating schemes.

Workshop Resource Manua I

Both.went back torwor:k. In a shortltime Steve and Anita reduced theirstack to a more ,mahageable size;' An$a,noted, "'We could weed out afew more if we identified resources that deal with the appropriate age level.Let's select only those thai are designed for grades 4"6 or grades 7-9."

*We could be even more specific and categorize them by learning setting,like outdoor, classroom, or nature center," Steve suggested.

This time Anita and Steve found their system was getting'too complicated.Few of the resources met all the criteria. They decided upon the followingcriteria to make their initial cuts:

- activity manual

- contain activities on biodiversity

- designed for grades 4-6 or 7.9

When they finished sorting and piling, Steve felt a sense of :accomplish"

ment. "l think we've saved ourselves a lot of time. Now that we've reducedthe numbers, we can devote more time to evaluating them," he said.

Anita pulled out a sheet of paper. "You're rightl Let's figure out what kindof things we'll look for as we evaluate."

Narrou your scope: Balancing thorough-ness and simplicity is crucial to success.Start by eliminating criteria that do notrelate to your content concerns. Thenprioritize the criteria and focus on whatwill help teachers incorporate EE intotheir lessons.

Adding Questions: You may find thatyou wish to add questions that apply tolocal education mandates or goals. \7henadding questions, frame them similarlyto existing questions so you may use thesame ranking process. To make tallyingresponses easier, keep the ideal solutionpositive. Similarly on a rating scale of oneto five, always make five the best or theworst.

Changing Citeria: Rather than eliminatea criterion, consider modifying the way itis applied to make it more relevant. Thereare several ways to ask a question aboutthe same criteria. For example, you couldchange a general question about learningstyles, "Are many types of learning situa-tions used?" to fit a specific learning sryleneed: "How many lessons incorporatestudent-directed learning?" You may wishto refine the questions to better relate tothe materials you are evaluating or teachersneeds. Use the same guidelines as aboveto write good questions.

Modifuing the rating system: Many toolsare designed to give statistically reliabledata. Therefore, the rating proceduremay be more complex than you need.You may not even need a rating systemunless you wish to compare among severalactivity guides. Instead you could makenotes about the strengths and weaknesses.

Evaluating Envtronmental Education Matetials

B. Creating Your Own ToolTo develop your own evaluation tool, youmust first create a survey instrument outof the criteria you have chosen. Then youfigure out how the answers to those ques-tions will help you compare activity guides.These two concepts go hand in hand.

Conuerting Criteia: The criteria sets inSection II will help you think about thecriteria that are important to you. To turna generic criterion into an evaluation ques-tion, think about what you really wantto know and how teachers will use theinformation. Questions can be asked togive yes/no answers, to help choose a num-ber on an answer scale. or to give a narra-tive response. For example, the criterion,"Activities encourage students to clarifytheir own values" may be converted intoseveral questions: "Do activities guide stu-dents in identifying value positions held bythemselves and others?" "To what extentdo activities allow students to form an opin-ion?" or "To what extent are studentsencouraged to defend or challenge theiropinions?" Framing the criteria will takesome practice and you may wish to do so

in conjunction with the next step.

Deueloping a Rating System: The ques-tions must help you evaluate the materials.\X/hat wili each answer mean? You can use

anything from a simple checklist to a com-plicated ranking procedure. The followingcommon methods may be appropriate foryouf use:

- Check off which criteria are met.This works best with yes/no questions.

- Each activity can be assigned a scoreand the scores averaged or addedtogether to give each guide a total score.

- If some questions are more importantthan others, think about weighting somequestions in a summary score.

- Use a narrative response style for thecriteria. This will provide more informa-tion, but it is difficult when used tocompare many activity guides. One wayto present information is to group simi-lar responses together or to develop a

short annotation for each activity guide.If more than one person is evaluating,you can indicate how often similarresponses were glven.

- The most common rype of tool providesa range of responses for each criteria.In this method, criteria measure howuell the guide meets each criteria, ratherthan a yes/no type of response. Forexample, "On a scale of 1-5, where1 = not at all and 5 - very well, howwell does the activity guide incorporatedifferent learning styles ? "

Most new evaluation tools will have a fewkinks to be worked out. We recommendthat you pilot test your tool on at least tr,vo

activify guides. See if each criterion can be

addressed the way it is asked. If not, try tophrase it differently. Also, make sure the in-formation generated by each criterion is

useful in determining whether the activityguide meets your goals and those of EE.

Step 3Gompile Results

If you are evaluating materials simply foryour own use, you may not need to com-pile the results into a publication or report.You could simply keep the rating sheetwith the activify guide or jot down noteson the strengths and weaknesses. If youdo wish to compile the results for othersthere are several approaches:

- List only the materials that have meta minimum number of criteria. Includea list of the criteria so that teacherscan determine whether the materialshave been rated on items that meettheir needs.

- Present the ratings in a matrix thatshow how each guide met the criteria.This allows teachers to select materialsthat rate high on criteria that are impor-tant for them.

- \7rite an annotation for each guide thatshares the strengths and weaknesses. Ifyou compile results for others, be sureto explain your evaluation methodsand criteria.

Workshop Resou rce Ma nu a I

I'lable 1: Sample Format for Compiling Results

Water, Precious Water Overall Score = 4This collection of lesson plans is a well-organized series of laboratoryhands-on activities.

Report Card

Teaching and Learning ................. A-

Presentation and Organization ...... A

Water Content, ,.................. B+

Grades K- 6

Disciplines:Math, Science

\ r_\l'iiii.au]. liu>.iiilg

.;.eiGl"t.idl,,

, "\i iii39,:i:la'r9lr,il.,

i riillll:,ilil).,l]:

,!ri.,,::;lYlrljlrtlirill2:.l3:::::'

" \I::: r,rtii::,rllii::,:rl

:iiiil6,,,.irrrr,,,1i,i,:,1:rrriirlr,r i:rll!'rz

iii lllll::\ir'lrll:lriiri!:urliiif ;u,ii:::t,t,:,i:":'liliarrrrZi6!.'riiirilliiurlrraiirr?all!

,,rit!1,:'ltrtltt

iUiiirriii]iii''(:irii\

it9:;$.,."r,9':rriil|ri,.''r,rri::"'rr{

.ir1111r,>,i::li.,..irS

trr,,r...,,,:r,,,:!r,r,,.irll':l:,::trl!2-t!

:;lrrr. riiililrrr".iiarl

-' ,-t '

.l t

i:1> lillf, ,,llf:l

\-r \r. :-,lr.llll,l/."llr'r!ll:lt:

..ii uir:iii5rrr'ri,t,

Hll.l:t\ l\

7rl/rrrr,;trrrr'?:rrr rz:r,l]]i:'!iarlliii:)l.i

)I l

rii!>iiiir,ll ,iii.l

i,. .,ilillllli.iirir,.-..ir?i.lr., ..iilalr.a,ii,::lrrliiii:.lll!.iiirlli.ii',iii-illll.ai:llrl. .:tl

I r.'2 .l r

..,1ir:!,..:i::ilr,,.::::

I? '-

rlrr:,,l,trr, j':,:,,ir,r,rll,,r]l:

iisr]],- l:iil

"t \ t

llultiiir.,,iiil:,l,,iiiut\tiliu,trilillrtT

:iiitrr']l'ii:ir]]'liii,r]lll,iiiliilrt,-,,,i9rr, a

iiar]i.,,r,rr,.,3,'rirr

;l:lirr;;::':A::,:.L

.,llr.u-J,,1:ialir,iiiil

".,\ I \l

illlr''iiliii'"ii!111,,.,'aiilllrrrr ,ii!r7 r!i!lu)-,,,rtirt.riiiartrrr,i...iii,l,rraii!,rr..-.r!

r lt

" I- -l!./rraiiiidilri-,illri lrllri.,lri::?:..i:::*:..:: ::a? ::...

.. \- l-\';r.rriiLill. ii:lri"iiii!

t,r2;:,,!f:li:|rr;

:'lrrrri):r-rirrrYll.:9;..!)).:;tte))e

| \,.zrrll'"'rr:lllr'r. i'riilll

Teaching and Learning: The labs emphasize the scientific processof investigation and offer many opportunities for inquiry. Suggestionsfor extended activities are included. Most teachers will want to choosefrom the wide variety.

Presentation and Organization: The directions to teachers are clearlywritten and contain well-defined objectives. The work pages are well-illustrated, uncluttered, and easily duplicated. The book's binding is alittle weak. No bibliography.

Water Content: The broad range of topics cover water awareness.the water cycle, water quality, water conservation, treatment anddistribution, and the properties of water Howeve6 there is nothingon water resource problems in California,

Teacher's Thoughts: Students and teachers both will enjoyworking through these lessons which emphasize the quantitativenature of science.

Adapted lrom Compendium for Water Resources, California Department of Educationa

Step 4Evaluate the Evaluation

Think of your evaluation technique as anevolving process. The more you reflecton the results, the more you will increaseyour skills and produce better results. Toevaluate your evaluation, make sure thatyou have met the needs of teachers who areselecting EE activity guides. Indirect evalu-ation can not, however, guarantee that thematerials will work with students.'$7hen-ever possible maintain records of teacherfeedback on their experiences with the ma-terials in the classroom. Comparing this in-formation to your evaluation results willhelp you refine your criteria and methodsthe next time you evaluate materials.

Guidelinos and Reminders:

Aim to identify strengths andweaknesses of the materials.

Esiablish a systematic procedure,follow. and document it.

lnclude a Not Applicable optionin your questions.

Pilot test your tool.

Check with teachers about theirreactions and advice.

Document your methods.

Eva lu ati n g Env, r o n me nta I Edu catto n M atet i a I s

A Look at Formal Evaluation Proiects

y F you ARE usING someone else's evaluation results, you may wish to

t critique their findings. First, examine their work closely to make sure

I the evaluation is relevant to your situation. Find out how the data were

I grth.red. If this information is not apparent or detailed enough, contact

the authors.This section gives an overview of three common evaluation methods,

described in order of increasing complexity. Each is illustrated with a real

application to help you critically review the evaluation results, understand the

strengths and weaknesses of each approach, and use the results of each method

appropriately. As you review these evaluation strategies, be sure to note the

methods used, built-in assumptions, and the format used to report results.

These and the factors listed below affecthow applicable the results are to yourspecific needs:

Evaluators: Do evaluators have relevant experience ? 'Were they familiar

with the activity guide's specific content and EE in general ? Do they have

classroom experience? Do they represent the teachers who might use

the evaluation results?

Criteria:'Were they chosen to meet specific audience needs or general

audience needs? Are criteria meaningful to you? fue criteria that are

important to you given a high priority?

Results: Do the results provide enough information to allow you tomake an informed decision? Do the recommendations take intoaccount the context in which the guides will be used?

Be wary of many "evaluations" that include no critical analysis. They

are usually reviews that list materials in categories, but without ratings. If you

do an ERIC (Educational Resources Information Center) search for evalua-

tions of EE instructional materials, you will find the majority are reviews and

formative evaluation projects. Comparative, quantitative evaluations that rate

EE activiry guides are lairly rare.

Workshop Resou rce Ma nu a I

The Expert ReviewThe Expert Review method uses one ormore select people to conduct an armchairreview of instructional materials. Theexpert is most often an impartial externalevaluator who has experience with the cri-teria. The expert must have both environ-mental education and content expertise.

This approach offers several advantages.An external expert may provide an objec-tive look at materials that are producedinternally. An expert review can provide acomparative evaluation of many resourcesin a short amount of time. This approachalso offers a relatively low cost alternativeto developing staff expertise in EE andevaluation. \7hen using the recommenda-tions from this type of review look for:

Bias - The biggest disadvantage to theexpert review model is the limited per-spective one or a few persons can bringto the task. You run the risk of a heavilybiased evaluation.

Qualifications of Experts - It may bedifficult to find one person with both con-tent and educational expertise.You maywish to research the qualifications of theexperts if they are not listed.

Support for Recommendations - Thismethod yields an educator's interpretationof what will be instructive for students.Actually using the materials may yield dif-ferent results. You may wish to find first-hand knowledge of some of the materialsto double check the recommendations ofthe expert.

The Pacific Gas and ElecticTeachefs Resource Guide,Sacramento, Caffirnias

Pacific Gas and Electric (PG6aE) recog-nized that the utility needed to evaluatethe materials they disseminate to educa-tors before ordering new materials andprinting another catalog. Since they didnot have an education specialist on staff,the company opted to conduct an evalua-tion by contracting with an EE specialistfamiliar with energy issues. The expertthey contacted recently compiled a com-puterized collection of more than 300

energy education resources and had writ-ten an energy education activity packetfor grades 4-6.6 The expert spent threedays evaluating the utility's library ofmaterials based on criteria used in otherevaluation projects. As a result of thiseffort, the expert recommended whichmaterials should be rewritten, expunged,reordered, or added to the collection.Based on these recommendations PG&Erewrote its teachers' guide.

The Grcup ReviewThe Group Review is an expansion of theexpert review. It uses a group of contentexperts, educators, and./or environmentaleducators. Their results can be tallied andstatistically analyzed to say with qualifiedcertainty that some materials are bettersuited to the chosen goals than others.Ideally, the evaluators represent a specificaudience, provide expertise across thediversity of materials, and provide a geo-graphical, cultural, and gender mix. Thisapproach is also an indirect evaluationalthough some of the evaluators may havefirst-hand classroom experience with thematerials. Things to look for when review-ing this type of evaluation project include:

Evaluators - Do they represent the spec-trum of teachers that come to your work-shops? Does the mix of evaluators givethe breadth of expertise that represents therange of materials evaluated?

Number of Evaluators - If there are notenough evaluators and responses don'tagree, it will not be possible to interpretdata with reliability. Resulting data couldbe misleading. The number of evaluatorsshould be noted along with the degree ofagreement between evaluators (often dis-played as a standard deviation).

Methods - It is important that the evalua-tion tool was tested to assure that all eval-uators were evaluating in a similar fashion,interpreting criteria the same, and that allevaluations were accurately recorded andcompiled.

Results - Are the criteria reported with theresults? How were the criteria determined?

Evaluating Environmental Education Materials

Educating Yowng People About W-ater:A Guid.e to Goals and Resources.

The United States Deparrment of Agricul-ture Extension Service gathered a reviewteam of water education specialists to eval-uate water education materials. Sixteenpeople from srare and private agenciescombined their efforts to (among othergoals) summarize information about watereducation materials. The project focusedon nonformal education materials. Theresulting guide was developed to helpdevelopment of Cooperative Extensioneducation programs and materials.

This Water Quality lnitiative Team sup-plied copies of instructional materials andrecommended other resources. They alsohelped identify water education goals andtopics. It was determined that due to thevolume of water education materials itwould be expedient to develop a short-term,initial project. The objective was to reviewand classify a selection of available materi-als as a basis for understanding what wasmissing and needed.

An initial selection for materialsto review was based on three criteria.Material was included in the review if it:

- addressed one or more of the study'sgeneral or specific goals

- presented a planned educationexperlence

- improved representation of regionalwater concerns, varied water topicsor environmental education goals.

The chosen materials were evaluatedagainst eight water topic criteria, fiveEE goals criteria, and five format criteria.The intent was to see whether the watertopics, EE goals, and preferred format werepresent. The results were compiled to showoverall strengths and weaknesses in eachinstructional material. A summary of eachmaterial provided information that willhelp educators select materials. The finalreport presents information in four ways:

- an alphabetized listing of materialsthat provides a brief description of thematerial,

- a listing of availability by stateor region,

- a listing of materials from nationalorganizations or nationally availablesources, and

- a matrix that shows review results.

The Prc-TesUPost-Test ReviewInstructional materials sometimes fallshort of expectations. Directly evaluatingthe impacts of the activities is the mostaccurate way to determine whether thematerials meet the learning objectives.The prelpost approach is the only onedescribed here that evaluates materialsthrough direct interaction with students.This model dictates that students begiven a test before they have a plannedlearning experience with the activities tobe evaluated. After the lessons, studentstake a post-test, which is similar to thefirst test. Presumably an improved scoremeans that the learning experience wasa success. A control group completes thepre-test and post-test at the same timeas the treatment group to assure that theinstructional activiry and not anotherlearning situation, was the source of theimproved scores. Any increase in the con-trol group's post-test score results froma factor other than the instruction; thisincrease should be considered when com-puting the treatment group's gain.

Teachers chosen to evaluate the activi-ties are randomly assigned to treatmentand control groups. Activities are random-Iy assigned to help increase accuracy of theevaluation results. The prelpost test meth-od is not usually used to compare amongactivity guides: it is most often used tomeasure the effectiveness of a single setof activities.

'$7hen reviewing the results of thistype of evaluation, be sure to consider thefollowing:

Test Consffuction - In using a pre-rest.you run the risk of alerting the studentsto the important concepts or simply raisingtheir curiosity about the new material.The tests should be similar but not exactor students may improve due to practice.

Biases in Teaching - Teachers who arefamiliar with the activities could do a bet-ter job at teaching than ones that are not.Activities should be randomly assignedso teachers get a mix of familiar and newactivities.

Workshap Resource Manua

Significance of Learning - Success is typi-cally defined as a better post-test score,based on the activities that are conducted.This will not guarantee that all activitiesare equally successful, or that broad goalsfor the materials are met that are notaddressed in these activities.

Proiect LearningTree,Enuironmental Edwcation Pre K-8Actiui4t GwideE

Pre/Post testing was the primary methodused to do a summative evaluation of the1.994Project Learning Tiee (PLI) ActivityGuide. PLI contracted with the NorthAmerican Association for EnvironmentalEducation (NAAEE) to evaluate the effec-tiveness of the new activities. The projectentailed several professionals and a budgetof over $50,000.

Pre-tests were given in 120 classroomsin the grade-level groups of Pre K-1, 2-3,4-6, and 7-8. (Pre K -1 students performedtasks and their teachers reported resultson computerized answer sheets.) The testswere designed to measure students' knowl-edge and attitudes about one of severalthemes, including diversity, interrelation-ships, patterns, structure and scale, andsystems. About two days after administer-ing the pre-test, teachers in 60 classroomsused between three and seven randomlyassigned activities that addressed one ofthe five strands. \ff/ithin a week or two,students in all 120 classrooms took thesame test again. All of the test data froma single classroom yielded a single scorefor that class. All the classes from the samegrade level that experienced activities ad-dressing the same PLI theme were aver-aged into one score. Teachers were alsoasked to keep a log, noting anything thatseemed significant as the evaluation pro-gressed. Logs are not always included inthis evaluation model, but can providesome significant and helpful information.This qualitative information helps the re-searchers interpret results more accuratelyand can provide new insights into the use-fulness of the materials.

.)"> * ti111S9Ui<tCD]11$lirlrAier,:](ell4

i:tiiativuiisi,

t*-\- ;

Rdfiijrrxi!:,.,a*-tlt r::::!a-,.:1!

r,l1,,E:i,,?ii:,,Xait[9,?.]i13!U}ltg9l:1,:11$?,,IiSll:::r1(B,i:li$i1.,r!:i1

illlriii6lx;iiari;riutety:itill%

,:::xr$lliiid:i*v*-

# -ii6l9iidit9:idl,ut::](*trt:iillaui:{

d;-" --lrr9,ilr@t-:irsgutr:ii€;t:G;i:tiY

iiSrri:!trllriG,llll:ilii8i:::,i@]i:ii

s,. s,.drt';\':

tJ{2,9:eilrllAGi%iAil7

'*3, F \:@rrllrlri@rlr: :t!!irllliilltllr:ii14lrr1r&igrl&rllrllli.r*lr:rr€llr..lilsrl

_: L r:

?^:l - IYt"Y ]

-". \, a -\::irlglli@!IiiaiiilU.:iii6rlr1iaiilllil,:iii@l:i6l;?Glsiuti:illtiin:::iiier:

re$r.ll19;.iiiUlllr.lll?I .... t ..':: N "l:X:l-z-*-, 1:1r1@l,rri6]]:iia

:11119lliii1s]):ili@

rX,6rgtd:Xll

* l*r-:; r.: \:))*1- ."nn

f ,l- li

iSlll:iiilYlr-rX,ia*Yl,^9ttll,r

srlltii6Pa;:tl

:-*- 4\ e:/-,t! & r\1'-{ - e

*,\1 i A

,:irlclr,tiiai4driililgut:teu:5@ti

@ut::116i:ii16rl

;itlPij,Ctl;iri* ':--tY r, I

*- 6\ i

- T"\.er- *

?lrli;lll9Lrirqrr;ir:ii16ut:1ii6ul:

!ilglld9ltaiil

:): -z:-;

/--t - E -ts'6, p

ll,:llt9l,a1@lrlr::i

k fl\

il?iaiiirr;|ril,1!]]]lli:lririiiiYrA

ii6;:16;ii:t1hl

-a a*;-,-:47,'- t6-- 5. i

id ,,.-i

$llrr::iii6iiiilrliiiliii:'

)r:iillsrrr@xl

, *.:s\:iEva I uati n g Env r ron menta I Educati on M ateti a I s

NE oFTHE BESTwAys to build teachers'skill is to have them developa survey tool and use it to evaluate some materials. If you do nothave the luxury of a few extra hours, even a short overview onevaluation can be a valuable addition to any EE workshop. Below

are three scenarios to help you plan your workshops. The activities areprovided in Section M.

What to Do in a'Workshop

Apprcach AProvide Evaluated Materialsand CriteriaProviding evaluated EE resources willreduce the effort teachers spend on select-ing instructional materials. Although thismethod often requires more of your timeand work before the workshop, it is oneof the simplest workshop strategies. Ifyou familiarize participants with the goals,criteria, and results of evaluations, theycan use this information to review otherinstructional materials.

You can either use resources that havebeen evaluated or evaluate the ones youhave. If you present materials that havebeen evaluated make sure the criteria,methods, and results are relevant to yourteachers. Be sure to provide and discussthese elements. If you evaluate the materi-als, consider using one of the evaluationtools supplied in Activity 3, page 28.(Sections II and III can help you modifythese tools.) !7hen using this method toplan a workshop, consider the followingtips:

- Before the workshop, identify some ofthe teachers' key needs for resources andthe areas of EE they want to address.Use these needs to modify the evaluationtools.

- Reserve time in the workshop for teach-ers to describe and discuss the resourcesthat have worked for them.

- Consider providing workshop partici-pants with the actual evaluation toolthat was used to evaluate the workshopmaterials. \X/ith the tool available, theycan decide which criteria are mosr im-portant to them.

Example Agenda

l. Distribute handout listing evaluatedEE instructional materials

ll. Overview of evaluation method

A. Criteria used to evaluate

B. Results of evaluation

lll. Teachers share favorite materials

Apprcach BDeveloping GriteriaThe workshop strategy described inapproach A puts you in the position ofdoing most of the work. Although thisstrategy saves time for teachers, empower-ing them with a hands-on evaluation expe-rience may increase their commitment toevaluating other EE instructional materi-als. This approach builds teachers' interestin evaluation and helps them understandhow to best choose criteria that meet theirneeds. You might consider the followingsuggestions:

- Build a rationale for evaluating EEmaterials. Using Activity 1 on page 24,help teachers think about why theywould bother evaluating activiry guides.Pair this activify with one of the twoideas below to give teachers the experi-ence of developing their own evaluationcrlterla.

- Introduce the topic with a short overview of evaluation and the goals ofEE. You can provide them with a

short version of the information onpages 4 -7. After this discussion,have teachers brainstorm and prioritizea list of criteria. You may augmentthis list with Master 2.

Workshop Resource Manual

- Alternately, get teachers thinkingabout evaluating criteria by havingthem review an activity guide and askthem for their opinions. Help them see

how their "gut reaction" can help gen-erate criteria. Have teachers work ingroups to develop criteria for EEgoals, pedagogy, and presentation. Youmay refer them to Section II. Activity 2details this strategy.

Example Agenda

l. Build a rationale (Activity 1)

ll. Overview of evaluation concepts

A. Levels of evaluation

B. lndirect vs. Direct

ll. Develop criteria (Activity 2)

V Teachers share their results

V Teachers share their favorite materials;explain why they work

Apprcach C

Do a Mini Evaluation

One of the best ways to prepare teachersto evaluate potential resources is to havethem evaluate an activity or a guide duringa workshop. While this approach is time-consuming, it provides teachers with theskills and practice necessary to becomeproficient evaluators. Given teachers'schedules, this may be the only time theyactually get to evaluate some of the mate-rials you give them.

Building on the activities suggested inthe approach above, take teachers to thenext step. Activities 3 aod 4 provide strat-egies for leading teachers through an eval-uation process. As you develop theworkshop, consider these ideas:

Give participants an opportunity todevelop their own criteria. You canstreamline these strategies by havingteachers review the EE guides whileasking: How will students react tothe activities?

'!7hat will the activitieshelp them learn? How will the activityguides affect student behaviors andparticipation in environmental issues?

Divide into groups and have eachgroup evaluate the same activity guidebut with different tools. Discuss thestrengths and weaknesses of the tools.

" One of the most prcciouggifts you can give to teachersis time to plan how to use ffienew information they acquirefrcm an inservice workshop,t'Margaret Pennack

Teacher. Mast Landing SLhool, FrceporlMaine

You can use this feedback to mod-ify the tools.

Divide teachers into groups. Eachgroup can evaluate a different guidewith the best tool. Compile the resultsfrom group reports.

Provide time for teachers to talk aboutwhich resources have worked well forthem.

ar,,:a

i:;;

3u

ler,iiiUl

)4.::a

t.,ii11

*ii.1.::':'::::

at;

l,er9l

:?),.,

L:rrl

rciii6iiii0l

ie],9,

'ali

::1,.:

riii!l7::l:l:l

i$tl

ll9rl

isuu

rrill

-,\,t 1\i\ ;:lrtaill9t:ririiar,,,.,,

laiiYrlti::rl{:,;i.:,.,tr,,,,,.:,Y,,1

:!:tl!- \"

:: I \

ia:i:rlr,tarrS>.ill:?Li;;::::t:::*:r../,iii;lrriiiii?lr,A,i:

- i - I \l

:]t:5$"-.f a;l,Ytt,{t,,

i iil+,i.iirliaril''rlriiit

1>nr{t) ii.rrrl9rrr:litlliii3i>l,,iiil9ll,:,elll{i!l:Xrll!iiilr#l':r

i!rrrr.1i6;i,iirlii

, .16

irrlg>.il€llr:iilid

riirelrrr\el,,rrliiOl,l.ii6rrl;iiiarrr':'riiiii:rt7;tlrrrr,ri6ii:!rr:ittt:

lniiiiii lllltilil1it':i(

aglllll-,glllr iiiii9l.-

.1 '-

i

,"1 - r':rrrl$"rllrailiSllrra{

" \ i-\t.-.,ri:9rra.iraiinr' :iiii

icuuu.iiiiiru:1i;i

ll6UUtiiiGUt:.'iiiB,:

;lr?l:;;il%;:lr9.f .>/--l " I

:'rliS:rr,aii9lrrr:{

?i,r|,,,,,ai:':}::rr,:ae:,:,::.!.1;:t)l:.4

:;ir/l;;$ll,:ii9t:ur..:iiii,rrtia:t,,,,.r;r;\llli:;;ii?;]]l

- rt ,* -€iiiXirrrrr:(drr:,/ t" _. I

ril':li;ir:lrriit,!:.ii11r,rl

Slul]<arrl

,l.1iii4

i ,r,tia:t,,

g.:.:. : :::::!).).).).

i lt:ii6llrr;\qut::,

idrr,:::iii rrl

!rrrrr,iigll..-.i

\ t \::il;rtrlil9i;irii

:ialrlltlia:r

,ralrrL,iriB,

/- .-

iili!r.iilii,,,

lll'lllrilllilrrrl,iiiUlllXiior:ji;;:;115]1,:;r.:::1;&,.".'.12

ir

? ,"

,:lllillrlrr:llsri

iiil+,i.iilliaril

r9llll,:{19lll

::':iiil9lllirl6ral:iiarrrl:li:il

iiit,,,%lSrrr,liOrnr

ltelltl!2i I tl

.19>:ilell,

liirYllll:\iel,,l

iiiqut;iii.rutl

riai\irrit:rl

a9:)iiij:rrl:.

llliiiil:tla;L iililou l:5;-llilirr,riiillli

:rid,$"rllrail

?:,:rrri19l:1ii€uu,.itlut,.

119lll,:l€lll'

lallll$lla*

arr.lrliiai:;rl9lftr

'rrarrrrst::rr::::1:1A::!.....lllilllr':rrlail:

,:tirl}Ulr:itl

!.,,ra,

i

lit

r:Itii

1

,,-a

iGirri

iiii:2ii:idll:iii

vl::i:.rrl

9iiiJ,lllllrri]l

,{i ril),

:i\11::,,:...::4ii:ia,]"i.:ili7llii

ii6:l::l

\,,rilal,.lai: lii:

ta:'::a:a?

lLll9l,1,rl1;ll

.:rtr::il:::liYrr,.,

R't,,,,,,,,.

t)),,...?.:i!rl,:,,io,,Y

iii9uu,iii

iaut':a:::.::l!a::,:

. lrrr,.,,

3it,:,itttliiiii9l{ltt-,iii!:lsllllr

iii@llt:iii

iiiarrlr.:iiitrr-,2rrl

r0)rrr,iiiiil9:

Aii:trtr:,

r,llOl,.L:::i1)ll.i:,,i1,,iia,t:1.li)...,:::4liii?lr,r..ri

,i::tlr..,,i

r\l.rtri

,,,,,llll:

3uUi1:

3).illr:

11,,.r-,li:

.lllli},.

:,iili:,u..,,

lXilili:rll

,rr:ili!,rt,

$!rrrrrJi15::.ri t

6tliiie.,elll{a)))..-|:19.

;l,,lti:,

l^lril,rl

li:iiidioi;r.iiii

ii:\*iii).),.::::.a)7i:!rrrairrr:

:oi.,"l'.4

riiiii9l.-::ii;rl:

rLtrrJ,rl

:,,,iut,rllll3ll::i(l ,.i:l:.v:::,,

rl::liiiit

i9uuliiil

ru..iie

\i:,iilri:,illiiill]]]:liii

)..<.rjt.:,t( \::illrtt:rt>:ttttr

L)),:.!,',!.,.::t;:..,,

,rr?trrl

.\qrrr:.,

l:adll

Eva lu ati n g Env i ro n mental Edu cati on Mater i a I s

Example Agenda

l. Build a rationale for evaluation (Activity 1)

ll. Brainstorm the goals of EE instructionalmaterials

Ill. Develop criteria (Activity 2 )

lV Evaluate an activity or activity guide(Activity 3)

A. Groups do same activity guide

B. Groups do different activity guides

V Compile results

Vl. Discuss how to adapt tools

Vll.Teachers share their favorite materials

SummaryThere is no shortage of EE materials fromwhich to choose. Only you can reallydetermine which instructional materialsbest meet your needs and the needs of theteachers you serve. Helping you makethose decisions accurately and quickly isa major goal of this unit. 'We

have lookedat some of the general principles of evalua-tion so you might create evaluation strate-gies that yield accurate and usefulinformation. In this process we have dis-cussed selecting criteria and how to bestapply them to the materials you wish toevaluate. Interpreting your results andevaluating your process are the final stepsof this procedure. We also hope you gainedsome skills to interpret the result of themore formal evaluation efforts you will en-countef.

If you find this systematic evaluationprocess valuable, you will want to shareit with the teachers with whom you work.The background information, the work-shop guidelines, and the activities providedwill help you encourage teachers to criti-cally evaluate the instructional materialsthey have and will encounter in the future.Ideally, we will all become informed con-sumers and disseminators of high-qualityEE instructional materials.

I

I

Workshop Resou rce Ma nu a l

HESE woRKSHop ACTrvrrrES AND sAMpLE To ors will help you designa short segment or develop an entire workshop that focuses on eval-uating EE materials. Using Section V as a guide, you can build these

activities into a module that fits your participants' needs.

Activities

The first two activities help develop an overview of the topic of evaluation bydefining the goals of evaluating materials and what it means to evaluate.

@ euiUing a Rationale @ Wtrat is Evaluation"l

Aaivities and Masters

This activity helps participants justifyan armchair review of EE materials.Teachers discuss benefits and goalsfor evaluating instructional materials.

Using three sample questionnaires,participants critique and modify aninstrument to help identify materialsthat meet their needs.

Immersing teachers in an evaluation project will help them gain both a betterunderstanding of the process of evaluation and confidence in their abiliry toeffectively evaluate new materials. Activities 3, 4, and 5 will also leave teachers

with a set of strategies and tools that suit their particular needs.

@ fwing Out Some Tools @Wtrat About Bias?

@ t.rrning to LookTeachers will compare activities thatcontain similar content but differentteaching methods. Participants willdiscuss aspects of pedagogy that willaffect evaluations of these types ofinstructional materials.

To introduce teachers to the conceptof evaluation, this activity asksteachers to develop a framework forevaluating EE activity guides. Theoverview of EE goals and evaluationcriteria categories will help teachersuse evaluation tools and resultsof evaluation efforts effectively. Twomasters for overheads are included.

Teachers discuss the inevitabilityof bias, then practice discoveringit in materials. Five strategies fordealing with bias are provided.

Eva lu ati ng Env i ro n mental Educ ati on Mater t a I s

Activity @ Building a Rationale

This activity helps

participants justify

an armchair review

of EE materialg

Teachers dlscuss

benefits and theirgoals for evaluating

i n str u cti on al mate r i a I s.

Obieaiues

To help reachers identify goals fortheir evaluation.

Materi.als

Markers, tape.

Time

30 minutes

ili,:-.iirL6i]]illa;:-d& i,-1 :

.i@r,.{il6liPl,*illl:ii:iiilllllrlffi iiiS

^t- .* Y.* ) i*/-itiaultl:iialliir't;iilrrYiiii*iYlllr,:lr:

t.:1@rr:iiil@rlr,

dY ril9l;lii4i,li:iiilliilil:ie!

q/ s lstltiiYilarr9,ar

ylatits?;€,6i,vas{lriillt

Outline

Set the stage by asking teachers to imaginethat they will be given a complete set ofnew resources for the school year. Althoughthey are not familiar with these EE materi-als, they are all good teachers and knowwhat works in the classroom. They will beusing their expertise to evaluate these newmaterials, and to determine how to use

them. But first, they should discuss whatthey will be looking for.

1

Divide participants into smaller groupsof three to five. Ask them ro appoint a

facilitator, recorder, and timekeeper. Giveeach group a piece of newsprint and afew markers.

2

Give the groups 15 minutes to come upwith a list of goals for EE instructionalmaterials and record them on paper. Youmight find these questions help stimulategood discussion:

- \7hat do you want teaching materialsto accomplish?

- What do you look for in a pieceof instructional material ?

- '$7hat makes activities a success

with students?

- What do you need from new materialsto help you teach well?

3

Have each group post their report on thewall. Invite all participants to walk aroundthe room viewing each others' goals.

4

Conclude by leading alarge group discus-sion on the similarities and differencesof each group's goals. Invite participantsto think about the value of taking timeto evaluate materials by asking them thefollowing questions:

- \7hat are the basic goals of evaluation?

- How can evaluation help you meet yourteaching goals?

- How can you use this exercise to helpyou think about evaluation?

War kshop Resou rc e Ma nu a I

Activity @ What is Evaluation?

To introduce teachers

to the concept of

evaluation, this

activity asks teachers

to develop a frame-

work for evaluating

EE activity guldes.

An overview of EE

goa/s and evaluation

criteria categories

will help teachers

use evaluation too/s

and the resu/ts of

evaluation efforts

more effectively.

Objectiaes

To introduce teachers to thE conceptof evaluation; to proyide a frameworkfor evaluating EE activity guides.

MatefialsA set of activity guides for each groupof three to four parficipart$, news-print, markers, and onerhead Masters1andl, if you wish.

Time

30 minutes

r6ill

Outline

Divide participants into groups of threeor four. You may group teachers randomly,or by grade level or content specialry.Explain to the teachers that they will eval-uate activiry guides.

1

Give each group copies of an activityguide. NOTE: If you wish to have teach-ers bring their favorite activity guides toevaluate, ask them to work in pairs, usingone guide between them.

2

Give the groups 10 minutes to quicklyreview the guide. Have teachers indepen-dently list their impressions of the activityguide, both good and bad. Teachers thenshare their impressions with their groupor partner. Each group should recordtheir reactions to the material.

3

Have each group report their results.Record their comments on a newsprintor overhead. Lead a discussion usingthe following questions:

- How were their results different? '!7hy?

- On what grounds do they base theirreaction to the material?

- If they had more time, what else wouldthey consider?

Using overhead Master 1 and the back-ground on pages 4 - 5, explain the levelsof evaluation. Identify which oftheir reactions relate to each level.

4

Using background from pages 7 - 9 andoverhead Master 2, provide an overviewthe three categories of criteria: EE goals,Pedagogy, and Presentation and Design.Discuss what criteria the groups used. Youcan stimulate the discussion by asking:

- \7hat questions did you ask yourself?

- On which aspects of the material didyou focus?

5

Have each group put their evaluationcriteria into the categories. Have themdiscuss any gaps they see and developadditional criteria to fill these gaps.Have them review the activity guide asecond time to rate how it meets theirnew criteria set.

6

Conclude by asking teachers to discuss:

- How is evaluation different fromjudging on opinion alone?

- How can they apply this knowledgeto looking at new EE materials in thefuture?

Evaluating Environmental Education Materials

Master I Levels of Evaluation

Reaction Evaluation

measures immediate response

to the learning experience (students)

to the materials (teachers)

Learning Evaluation

looks for changes in:

knowledge

awareness of environmental issues

attitudes

Behavior Evaluation

looks for changes in:

students' skills

student behaviors

Results Evaluation

measures impact on:

the environment

students' lives

Evaluating Environme ti: E'.tcatian Materials par^ 26 l'i/ar kshop Resource Ma nu a I

\v Master B Criteria CategOrieS

Goals of EE

. Awareness

. Knowledge

. Attitudes

. Skills

. Participation

Pedagogy. Teaching and Learning

- interdisciplinary- age appropriate-crrtical thinking- cooperative learning- assessment

. Bias

. lntegration into Curriculum

Presentation and Design. Format. Writing. Appearance. Organization

Evaluating Environmental Education Materials page 27 Workshop Resource Manual

Activity @ Trying Out Some Tools

Using sample

questionnaires,

participants critiqueitems and modify

an instrument to

help them identify

materials that meet

their needs.

Objectiuei , , l

To,help participants recognize the -

importance of how evaluation ques-tions aie worded and the length ofevaluationitogk. To give participantspractice in using several evaluationtools. To have participants rnodifyan instrument to rrrieet their evaluationneeds.

Materials

One acrivity guide for each group orcopiee of activities; eopies of each ofthe Sample evaluation too.ls providedin Masters 3, 4, and 5.

Time

One to two hours with optionalactivities

Outline

Participants will practice using differentevaluation tools to evaluate the same pieceof instructional material. This will givethem insight into how different criteria,questions, and rating strategies can affectthe results. They will use this insight toadapt the tools for their use.

1

Distribute a copy of each sample tool andeither one activify or a complete activityguide. Ask each team to evaluate theirmaterial with each of the three tools. Havethem discuss among themselves which toolthey liked best and why.

2

I7hen teams finish their reviews, lead a

discussion with the following questions:

- Did the evaluations result in equivalentinformation?

- If not, how did they vary?

- On which criteria are the answersmost varied? I07hy?

- 'Sfhat questions did they have about

using the tools?

- 'Which tool did they like best and why?

3(oPtional)

Have each group list the characteristicsof the tools they liked best. Ask them toconsider the rypes of questions, ratingstrategy, ease of use and how it related totheir teaching needs. Using these qualities,have the group modify one of the toolsor develop a new one based on their needs.

4(optional)

Have each group share their new tool,presenting the characteristics. typesof questions. and comparison srrategyto the whole group. If possible, makecopies of each new tool for all groups.

Wor kshop Resou rce Ma nu a I

Master [l The EE Gurrlculum lnventory Tool

Directions: lndicate how well the inventoried materials will helpstudents to know about or accomplish the statements for each goal level.Circle the appropriate number for each statement according to the scalebelow. Compute the Goal Level Average as directed below.

Evaluation Scale: Notatall Poorly Slightly FairlyWell VeryWell

01234

Goal Level 1: Ecological Foundations (Knowledge) Rating

1. Populations: The nature./behavior of natural populations O 1 2 3 4(birth, growth, change & its effects, death, extinction, etc.)

2. Natural Communities: Structure behavior, interaction O 1 2 3 4of individuals and populations in a particular place (forest,desert, sea, pond, etc., food chain, food web, predation, etc.)

3. Ecosystems: Structure, interaction, and effects between 0 1 2 3 4living things and the non-living environment (natural change,succession, adaptation, etc.)

4. Energy & Chemical Cycles: How energy, chemicals, and 0 1 2 3 4pollutants go through a natural system (water, oxygen,energy, carbon, etc.)

5. Balance of Nature: The tendency of natural systems to estab- 0 1 2 3 4lish balance over time when disturbed naturally or by humanpollution.

6. Humans as a Component of Natural Systems: The effect 0 1 2 3 4of human behavior on natural systems. (Development oftechnology, effects of people's attitudes, and values.)

Ecological Foundations Total: Add all circled numbers.

Ecological Foundations Average:

-Divide

by 6.

Evaluating Environmental Education Materials page 29 Workshop Resource Manual

Master $ Page 2

Goal Level 2: Issues and Values (Knowledge) Rating

7. ldentifies the difference or discrepancy between an environ- 0 1 2 3 4mental problem and issue. (Environmental problems maybe natural or human-caused while issues are the conf lictsarising from how people view the problem).

8. Communicate the way people's culture affects the environ- 0 1 2 3 4ment (politics, society, religion, etc.).

9. Communicate the way individual people affect the environ" 0 i 2 3 4ment (individual actions and behavior).

10. ldentifies (clarifies) one's own attitudes and values toward 0 1 2 3 4an issue or the environment.

11. Communicates that there may be more than one way 0 1 2 3 4to resolve an environmental issue.

12. Communicates that the difficulty in resolving environmental 0 1 2 3 4issues stems from the different attitudes and values of thepeople involved in them.

lssues and Values Total Add all circled numbers.

lssues and Values Average_Divide by 6.

Goal Level 3: lnvestigation and Evaluation (Skills, participation) Rating

13. Uses problem-solving skills to identify specific environmental 0 1 2 3 4problems and related issues.

14. ldentifies the people (actors) involved in an environmental 0 1 2 3 4issue.

15. ldentifies the beliefs and values that explain people's 0 1 2 3 4positions on an environmental issue.

16. Examines a variety of environmental issues and their 0 1 2 3 4solutions.

i7. ldentifies the ecological costs and benefits of designated 0 1 2 3 4solutions to environmental problems and issues.

18. ldentifies the human costs and benefits of designated 0 1 2 3 4solutions to environmental problems and issues.

lnvestigation and Evaluation Total: Add all circled numbers.

lnvestigation and Evaluation Average: _Divide by 6.

Evaluating Envtronmental Education Materials page 30 Wo rkshop Resau rce Ma t)u a l

Master [l Page 3

Goal Level 4: Environmental Action Rating(Knowledge, skills, participation)

19. Communicates the need for responsible citizen action 0 1 2 3 4to resolve environmental issues.

20. Communicates that there are various levels of environmental 0 1 2 3 4action (individuals, groups, organizations).

21. ldentifies examples of environmental action categories 0 1 2 3 4(persuasion, consumerism, political action, legal action,ecomanagement).

22. lxamines scenarios/case studies that would allow the 0 1 2 3 4student to apply knowledge of environmental action andin choosing responsible environmental action.

23. ldentifies the human and ecological costs and benefits 0 1 2 3 4to identified environmental actions.

24. Takes individual and/or group action on an environmental 0 1 2 3 4problem they have identified and analyzed.

Environmental Action Total: Add all numbers circled

Environmental Action Average: Divide by 6.

Goal Level 5: Teaching Methods and Practice Rating

25. Do the materials provide specific training in the use 0 1 2 3 4of problem solving skills such as observing, inferring,predicting, classifying, hypothesizing?

26. Do the materials provide opportunities to apply problem 0 1 2 3 4solving skills with simulated models, real life situations,scenarios, etc.?

27. Are the materials educationally approprrate for the intended 0 1 2 3 4student users (reading level, conceptual and skill levels, etc.)?

28. Would the materials be attractive and appealing enough 0 1 2 3 4to gain students' interest?

29. Do the materials contain goals and measurable objectives? 0 1 2 3 4

30. Do the materials provide sufficient suggestions or mecha' 0 1 2 3 4nisms for evaluation?

31. ls the information provided for students and teachers 0 i 2 3 4accurate?

32. Do the materials provide sufficient background information 0 1 2 3 4for the teacher?

33. Do the materials provide clear directions and suggestions 0 1 2 3 4for teaching?

Teaching Methods Total: Add all numbers circled

Teaching Methods Average:-Divide by 9.

Evaluating Environmental Educatian Matertals page 31 Wcrkshop Resource Manual

Mader p Page 4

Summary Data

Evaluator's Name Date

Title of Curriculum Materials Author

Publisher

Summary Graph

Directions: You may present the results of your evaluaiion on thegraph below (with the averages on pages l, 2, and 3.) Severalpeople may also choose to combine and average their scores foreach curriculum so that a group score can be graphed. Shade thebar that corresponds to each goal.

Ecological lssues

Foundations and Values

Average Average

lnvestigation Environmental Teaching

Evaluation Action Methods

Average Average Average

Copyright Date

Adapted and used with permission, J.R. Gardella, 1993. Highland Heights, KY:School of Education, Northern Kentucky University.

Evaluating Envitonmental Education Materials page 32 Workshop Resource Manual

Masbr I Development andEE Resounges Tool

Resou rce:

RateAs: ;,' ffi:;iHiltocriterion"3" No attention to criterion"NA" Not Applicable

RatingKnowledge CriteriaResources should contain essential elements of core concepts. 7 2 3 NA

Development lssues

SustainableDevelopment 1 2 3 NAPresents the need for balance between economic growth andsocial development in the interests of sustainable develop-ment and overcoming poverty and social injustice.

EnvironmentandDevelopment I 2 3 NAPresents the importance of ecological balance against theeffects of environmental destruction and the need to balanceeconomic growth and conservation through a process ofsustainable development. Presents the importance of biodi.versity and sustainabi I ity.

RightsandDevelopment 7 2 3 NAPresents the individual rights and responsibilities as citizensof local, national, and global communities. Supports the needto eliminate discrimination and other threats to human rightsfrom various sources. Presents the rights of all species asessential for biodiversity and sustainability.

GreenAid I 2 3 NAPresents aid projects that promote economic development inan environmentally responsible manner. Presents benefits toboth donors and recipients.

Global Perspective 7 2 3 NAPresents the interdependence of global economic, social and po.litical systems. A selected global perspective reflects the princi.ples of economic justice, human rights, sustainable developmentand peace.

Comments:

Evaluating Environmental Education Materials page 33 Workshop Resource Manual

Master @ Page 2

Critical Skills CriteriaResources should help participants challenge bias, support ratio-nal decision making, examine solutions and prospects for change.

Balance Issues

comprehensiveness Rating

lncludes a wide range of social-cultural viewpoints. Sensitive I 2 3 NA

to explanations of social structure. Can distinguish betweendescription and exlanation.

StereotypingPresents a range of roles and human characteristics for any 7 2 3 NA

group. ln-depth portrayals of sub"groups include accuratepresentations of the group culture, from the point of view ofthat group. Sub-groups portrayed active participants in theircommunity in a variety of decision making situations.

Misrepresentation and ValidityDoes not present unsubstantiated 'factual' claims as 'truths.' I 2 3 NA

Sources are verifiable, accurate, and up-to-date. Does notinclude misleading implications.

TechnologyDoes not imply that the world's major problems can be 7 2 3 NA

solved by further development of industry and technology.ldentifies technology as part of the technical-social complexof productive forces. Technological solutions to social issuesare seen in their political, economic, and social context.

lnterdependencePresents an issue in its connection with the wider ecological, 1 2 3 NA

economic, political, and social environment. Presents individ"uals and communities as linked.

Decision MakingPresents conditions for rational decision-making method. I 2 3 NA

Emphasizes inquiry process of hypothesis and generalization,cooperative learning, critical reflection, dissemination,andinvolvement and action.

Problem SolvingEvaluates the relevance, validity, and implications of alterna- I 2 3 NA

tive solutions. Presents the interests and values underlyrngvarious solutions. Supports a multi-disciplinary approach.

Commentsl

Evaluating Environmental Education Materials page 34 Workshop Resource Manual

Master I Page 3

Attitudes CriteriaResources should help promote understanding and solidarityamong all peoples.

EmpathySensitive to the viewpoints and feelings of other people,particularly those belonging to groups, cultures, and nationsdifferent from our own.

RespectSupports the development of self.respect and respectfor others, particularly those with national, cultural, andfamily background different from our own.

ParticipationSupports solidarity with victims of injustice in one's own andother societies. Supports participation in political decision.making at local, national, and international levels. Supportssolutions of major world issues through cooperation at levelsof individuals, organizations, and nations.

Comments:

Values CriteriaResources should help participants develop a commitment toworking with and for others in the interests of equality, equity,and sustainability.

Social JusticeExamines inequalities of wealth and power. Presents effortsto redress them through development, aid, trade, and equity.

Equality of PeoplesPresents the principles of equality and democracy whichunderlie a belief in the equality of all peoples, as the rightof all peoples to life, freedom, and self.determination.

Ecological SustainabilityPresents biodiversity as a right of all living species and as abasic principle of sustainability. Presents intra-generationaland inter-generational equity as a right of present and fuiuregenerations and as a basic principle of sustainability.Presents the satisfaction of essential human needs and aspirations as a priority and as a basic tenet of development.

Commentsl

Rating23NA

723NA

I23NA

723NA

Rating23NA

T23NA

723NA

723NA

Used with permission and adapted from "Analyzing Education Resources" by Zita Unger, 1992. tn Teaching for aSustainable World: Environmental and Develooment Education Proiect for Teacher Education. John Fien, editor.Brisbane, Australia: Griffith Universitv Press.

Evaluating Environmental Education Materials page 35

Master p California Compendium Tool

The following evaluation tool used by the teachers to rate materialswas adapted from the California Sclence Framework, Chapter 8,"l nstructional Materials Criteria."

philosophy, and suggested activities by the lessons and materials(e.g. using recycled materials and properly disposing of wastes)?

9. Are there clear linkages presented between communities of all levels("thinking globally, acting locally")?

trtrtrTtrtrtr

a...............b...............c...................d e=not applicableexcellent Poor

A. General Content

1. Are ideas expressed through unifying themes and big ideas, without facts?

2. ls content interdisciplinary?

3. Are students challenged to utilize higher'level thinking processes(i.e., inferring, relating, and applying)?

4. Are ideas presented logically and connected through the curriculum?

5. ls depth of understanding emphasized (rather than encyclopedicbreadth)?

6. Are historical, ethical, cultural, geographic, economic, andsociopol itical relationships addressed?

7. Are knowledge and learning shown to be connected to students'lives and society?

B. Presentation

tr 1. Are instructional materials clearly and engagingly writien with the main conceptswell articulated?

tr 2. Are the roles of environmental ethics, citizenship, and stewardship explored?

tr 3. ' Do lessons promote respect and caring for the environment,yet be non-dogmatic and open to inquiry and differences of opinion?

I 4. Are personal and societal values and conflicting points of view explored in context?

tr 5. Are instructional materials easy for students to use and understand?

tr 6. ls learning made accessible to all students?

tr 7. Are writing and concepts developmentally appropriate for thedesignated grade, yet sensitive to individual differences in educationalexperience and learning mode?

tr 8. ls environmental responsibility modeled in design, underlying

tr 10. Are vocabulary words defined in context and not obscuring learning goals?

tr 11. ls the layout of instructional materials interesting and appealing?

Evaluating Environmental Education Materials page 36 Workshop Resource Manual

Master p Page 2

C. Pedagogy

tr 1. Does almost half of the curriculum have students engage in active learning?

tr 2. Do students construct knowledge through research, discussron,and application to gain conceptual understanding?

tr 3. Are evaluation devices included and appropriate? (Highest pointsfor authentic, performance-based assessment devices.)

tr 4. Are instructional materials sensitive to social, economic, and cultural diversity?

tr 5. Do lessons encourage students to develop awareness, knowledge,and strategies for responsible action?

tr 6. Are group./cooperative learning strategies used?

tr 7. Is intergenerational responsibility, linking today's actionswith future consequences, implicit in instruction?

D. Presentation

tr 1. Are instructions for the teacher clear and concise?

tr 2. Are lesson objectives and outcomes clear and appropriate?

tr 3. Are materials easily integrated into an established curriculum?

L--l 4. ls background information for the teacher adequate and accurate?

I 5. Can the materials be adapted to varied learning environments(large or small classes, of mixed levels, from rural or urban settings)?

I 6. Are consumable instructional materials of good quality, easily duplicatedfor student use, and in sufficient quantity to support the objectives?

tr 7. Are equipment and materials listed and reasonably accessible?

I B. Are a variety of instructional strategies, expanded learningenvironments, and resources suggested in the curriculum's design?

9. ls the time required to complete each lesson indicated?

10. Do the materials clearly list the subject discipline(s) integratedinto each lesson?

Used and adapted with permission by the California Depattment of Education from theEnvironmental Education Comoendium for Water Resources. The California Departmentof Education. The California Department of Water Resources: Sonorna State University.March 1 992.

trtr

Evaluating Environmental Educatian Matertals page 37 Wor kshop Resou rce Ma nua I

Activity @ Learning to Look

Teachers will com-

pare activities with

similar content

objectives but differ

ent teaching meth-

ods. Participants

uvll/ discuss aspects

of pedagogythat

will affect evalua-

tions of these types

of instructionalmaterial.

Objectiues

To introduce teachers to rhe power ofpedagogy and how it will effect theirevaluation. ,' '

MsturiAk'Activitiis f,rom Master 6 and7 ortrvo activities'that cover the same

topic Optional: Copies of Master3,4, and 5. '

Time

40 minutes

Preparation

This activity suggests teachers comparetwo EE activities for students. One pairis provided, though you may substituteanother from your own materials. If so,look for activities that are designed forthe same age level, cover the same content,and have similar learning objectives, butuse different teaching methods to reachthe goals. For example, the activities onMasters 6 and 7 look at the value of farm-lands but use slightly different approaches."Down on the Farm" uses simulation anddiscussion. "Vanishing Farms" uses simu-lation and role play.

Other potential sets of activities are:

"'W'ater, Water Everl.wher e" fr om EarthMatters, "How Wet is Our Planet" fromAquatic'WILD,and "Eighty-six Gallonsper Day" fuom Liuing Lightly on tbe Planet;

"Bye Bye Birdie" from Earth Matters byZPG and "Here Today, Gone Tomorrow"from Proiect'WILD;

"'STatch on'Wetlands" from Project Learn-ingTree and "The Great Swamp Debate"in NatureSco pe's'Wading into Wetlands.

Outline

Start a discussion among teachers abourpedagogy and the design of instrucronalmaterials. Ask them to think about hou-these concepts affect the wav tve evaluatean activity or activity guide. Erplain tlaara single activity rarely incorporates all rhe

goals in the instructional guide. Duplir:atethe activities so each participant receives

a pair.

IDivide rhe participants into teams of rourto five. Hand out copies of Masters 5 and7 to everyone. These activities har-e similarcontent but different obiectives. Challengethe teams to critique both activities. norinqthe teaching methods, student outcomes.and ultimate objectives. You mav u'ish tohave the teams use one or more of the elal-uation tools from Masters 3, 4, or 5.

2

Lead a discussion about the differencein the two materials using the follou'ingquestions:

- 'Would each activity meet your

teaching objectives?'!7hy or whv not?

- \X/hat makes a good learningexperience?

- 'What criteria would you use toevaluate each activity?

- '!7hat

criteria ought you use for thepedagogical aspect of the materials?

Workshop Resou rce Ma nu a I

Master p Down On The Farm

Objectives:

Students will:

. Measure and mark off an area ofto feed them for one year.

land which represents the amount of land required

. lnvestigate the conflict over private vs. public control of farmland resulting fromagricultural zoning.

Materials:

Student information sheet provided; a longtape measure(at least 71 yards long)or, lor a handy measuring device,use a yardstick, four small nails, 7l yards of kitestring, and a hammer, You will also need four stakes marked with brightly coloredpaint or material.

Advance Preparation:To make the measuring device, pound two small nails into each end of a yardstick.Tie one end of the string to the nails at one end of the stick.

Overview:It takes approximately 1.2 acres of land to feed one person each year, This includesgrazing land for dairy and beef cattle as well as cropland to grow grains, fruits, and vege'tables. Between 1967 and 1973 the amount of cropland and potential cropland convert-ed to urban uses was 875,000 acres peryear. This rate of cropland conversion is causingsome concern in that we may be presenting future generations with a shortage of pro-ductive land. In an effort to stem the tide of cropland conversion, many communities aredeveloping land use plans which include zoning laws to preserve farmland. The conf lictthat these laws create will be debated byyour students as they investigate this issue.

1, Ask students how much land they think would be required to feed each of them for aone-year period. Round the L2 acre figure off to one acre and take your students outto measure an acre of land on your school grounds. An acre is 4,840 square yards(4,047 square meters). A square 69 yards, 20.5 inches on each side is an acre. (lf theland you have to work with is not large enough, measure a one-half acre area. A halfacre square is 49 yards, 7 inches on each side.)

lf you use the measuring device suggested above, have students wrap the stringaround the yardstick, Each loop around the length of the yardstick equals two yardsof string. Have them wind the string until they have 69 yards, 20.5 inches and cut thestring. Then use the string like a tape measure to mark off the four corners of theacre. Have some students hammer a stake into the ground at each corner.

2. Discuss the figures below that indicate what is happening to our individual shareof cropland on a global basis.

Year1650

t9752t35

Gropland per person

4 acres

1 acres

l/4 acres (projected)

The decline of arable land per person is projected to decrease as the world populationcontinues to expand. Most of the land suitable for farming is now in use. Only 11 per-cent of the world land is arable. The rest is too steep or cold or too dry to supportagricultUre.

Evaluating Envtronmental Education Materials page 39 Workshop Resource Manual

Master @

Repr i nted w ith perm issi o nfrom Living Lightlv on thePl a net. throu s h a rra nse-ment with Schlitz Audu-bon Center of the NationalAudubon Society, 1111 E.

Brown Deer Road, Milwau.kee. Wl 53217.

Page 2

3. Either read the information on the student sheet or distribute copies for stuce-:sto read on their own. Discuss the issue of agricultural zoning and its infringe-r,e-:on individual property owner's right to sell their land as they choose. Revie,,,,ti-:discussion questions on the student's sheet and have students express thetr 0,,,r-points of view on the issue.

Extensions:Write the following statement on the board: 'Asphalt is the land's last crop." O.: s:a:=ment was made by M. Rupert Cutler, former United States Assistant Secretary c' :g. .

culture, in reference to the problem of vanishing farmland.) Have students use:- s asan introductory sentence in an essay expressing their views on the subiect of fa.,r a-:conversion.

To help your students understand the special nature of topsoil, have them plar: c:.-seeds in a container of topsoil and in a container of subsoil. Subsoil is the layer cf r-:-ent-poor soil to be found under humus.rich topsoil. lt provides a poor growing n el --and will help students to realize that not all land is suitable for growing food.

Student lnformationHow would you react to this situation if it were taking place in your community?

Voices are rising along with tempers in the crowded room in City Hall. There is a :rt :hearing being held to give citizens a chance to express their views about the propcsecnew zoning laws. Zoning laws set aside areas of land for specific uses. When a car* --ty has zoning laws, some areas are zoned for homes, others for industry, and othe. a-.eas are zoned for open space. ln this community new zoning laws will restrict the ;sesof farmland. There is a hot debate taking place between people who are divided in t.e "opin ions.

The proposed zoning would require that farmland be sold in tracts no smaller than rriacres and that the lands remain as open space or farmland.

The community is near a growing city. The trend has been for farmland to be soid a.;divided into subdivisions and shopping malls. ln many cases, it was more profitab e':.farmers to sell their land to developers than to continue to farm. Many farmers are a--gered over the proposed denial of iheir right to sell their own land for uses other il.anfarming. They believe that people should have the right to do what they want with lle .own land.

Other people in the room are concerned about the community's loss of farmland. The1,,view the land as a valuable, irreplaceable resource. They argue that farmland should n:tbe viewed as something to be bought or sold, but as a resource to be preserved for fr,-ture generations They believe that preservation of farmland is a social obligation. Thisleads them to believe that the public, in this instance, has the right to exercise controlover these valuable Iands. The debate rages on.... What do you think?

Discussion Questions. How would you react to the proposed zoning restrictions if you were afarmer?

A housing developer?

. Do you believe that the public, in the form of government, has the right to imposerestrictions on individual landowners? State reasons for you beliefs.

' Should land use decisions we make today consider the needs of generations whichwill follow us? Why or why not?

v

\/

Evaluating Envtronmental Education Matertals page 40 Workshop Resou rce Ma nua I

Master f,l Vanishing Farms

Objective:Students will gain an understanding of the pressures to develop open space and farms.

Summary:Students will role-play'farmer, crops, farm animals and the different types of intereststhat also desire land.

Time:45 minutes

Student Grouping:Whole class

Materials;10-30 foot piece of string with ends tied together to make a circle. (Longer string forlarger groups)

Background lnformation:Every year three million acres of farmland are lost to some sort of development. lf thiscontinues, eventually there will not be enough land to grow food. Localities have begun tomake ordinances and laws that facilitate maintaining agricultural land. The issue of pri-vate rights versus the common good is brought into play in these situations.

Marin County, California, Agriculture Facts:

The debate over land use is very real to Marin residents. For years there have been effortsto preserve county agricultural lands. Many people have been involved, particularly sincethe 1970's, in making it feasible for farmers to keep farming in spite of the skyrocketingprices of land in the area. The Williamson Act, passed in 1965 and amended in 1970,allows for reduced property taxes for agricultural lands within certain areas. MALI theMarin Agricultural Land Trust, purchases agricultural easements, restricting future usesof property to agriculture and recreation. Without these types of provisions, much moredevelopment would be inevitable.

Preparation:1. This is a very simple activity. You just need to be familiar with the exercise so it will

flow smoothly in class.

2. Acquire your string and decide where you will lay it out.

Procedurer1. Place the string on the ground in a circle.

2. Assign one student the role of "Farmer."

3, The farmer lists the crops and livestock he would need in order to feed and clothethe local people. As the farmer names the things on his list (like corn, wheat, cows,sheep, fruit) a student is selected to step into the circle and represent the commoditynamed.

4. One by one the rest of the students become people who also want to use the farm-land: for example, a road builder, someone developing a shopping center, someonedigging for water, a school builder, a hotel owner. These people move to the edgeof the farmland, pick up the string and move it inward, making the circle smaller.

Evaluating Environmental Education Matetials page 41 Workshop Resource Manua I

Master I Page 2

5. Developers continue until the farm circle is too small to hold all the farmers livestod<and crops. There has been too much development in the area and thus the furm hasvanished.

6. Discuss the activity with the students.

Questions for Discussion:. How does development pressure affect food production?

. Would this affect prices?

. What can all those developers do?

. What can farmers do?

. How does development and loss of farmland affect the families in the community?

Extensions:. Have a farmer and developer come to the class and talk about their ideal plan

for land use in the area.

. Find some old pictures of the school area (check with a local historical societyor local paper) and compare the open space then and now.

How does Marin Agricultural Land Trust work?Marin Agricultural Land Trust (MALT) seeks to permanently protect Marin's agriculturallands by purchasing agricultural easements from ranchers.

An easement is a legal property righi and can address many of the ways in which landcan be used. An agricultural easement is one which allow the land to be used for agricul-tural purposes. MALT buys agricultural easements and then permanently assures thatthey are not sold off to allow subdivision of the land for development.

Agricultural easements are always sold willingly by ranchers to allow them to get cashfor farming operations, to make improvements, or meet other needs. The farmer sellingthe agricultural easement still owns the property, and the land is guaranteed to stay inagricu ltural production.

MALT was formed by a coalition of ranchers and environmentalists in 1980 andis considered a national model for agricultural land preservation.

Used with permission andada pted f rom " Vani sh ingFarms" by LeeannTourtillott and Ellen Rilla,1992. ln Food forThought: AgriculturalClassroom Activ ities forGrow i ng Mi nds. Un iversityof Ca I iforn i a CooperativeExtension, Marin Agricul-tural Literacy Project.

Evaluating Environmental Education Materials page 42 Workshop Resource Manual

Activity @ What About Bias?

Teachers discuss the

inevitability of bias

and practice discov-

ering it in materials.

Objectiues

To enable teachers to detect biasin teaching materials and help themto develop strategies that overcomebiases in EE instructional materials.

MateiakHandouts made from Masters 6and /.

Time

30 minutes . ,i ' i

Outline

Begin by explaining that all materials havesome bias toward the author's, producer's,or funder's viewpoints. '$Thether from in-dustry. textbook companies. government.or environmental organizations. most ma-terial includes a particular bias.

1

Have teachers brainstorm the types of biasthat they have detected in instructionalmaterials. Some possible answers are: gen-deq age, culture, resource use vs, preserva-tion.

2

Divide the participants into small groupsand distribute Masters 6 and 7 to each.Ask the groups to list examples of wherethe author's bias comes through.

3Have groups share their insights. Notewhether the groups came up with the samebiases and examples. If there are differenc-es, ask the group to consider how theirpersonal biases affected their perception ofthe materials.

4

Ask how teachers might approach materi-als with bias. Add to their list any of thefollowing:

- Not use the material

- Accept bias as is

- Balance this with other materialsof an opposite bias

- Have students discover the biasas a learning experience

- Modify materials to reduce bias

Have each group discuss what they mightdo with these activities given the bias theyfound. Record all srrategies on an over-head or newsprint.

5(Optional)

Ask participants to generate questions foran evaluation tool that would address theirconcerns about bias.

Evaluating Environmental Education Materials

t& \ ;_'*

:ilSlllrr:ii€.$llrli:,?ffi l]]t-'.t*- *.- l

iillt:$,!!ttlptdlgdi116ti:iiGulv;i#

lSgl|:;llllSlUrt:iiiilat:* s.-.* le.i -€u'x --SX,r@i:llblXli

$xili$iiu9;iiii] j:,;i!, e 1*-"

;T,16i;lY,;{lql:Xl9u;i1?utisux

{Xli&lltr6C;l- :1/ :€"' di99arsrrr:ii'a){ii--r e\- )" : d

;tYlai$'rii4:,it:taY7t18+r.i118::* ar: € -'

Xr@jrll%lSX9;lgYli&rrrt

rrl!@,rr.lliGillil:l];i

".: a,"* Y -,"*'J*,

]ry.,:rill$lliY::;iry

.g: * tr

'ra6lllY,6tryl:;li*iu:'€iillH:llrililBl.l* lr di9rr*16SYll#ii9*

*9,*$$trffi \-:&- tr

iSiiallilgllrl.iili@rr!

6lll:lidietXiq6ilPXrlSll:&trir:rriii$u:!llalEtkll

iu*t:litlle,x@i:l6lliZ::illllllllalig

[iffi---"t{- } aliWll,:]:\leil:::lriili.- q

-..* /"-i; Jsli6r*lliAil@lll:11[9]l

;1c';:iiGiillPtili;l;l!t:1111$+,ti:iiiidrti,li:6

", e :}* *__

ll:ii1@lll::iialiYi;)

.-B,re -ts i

1@rr,l16|gl#:

:lreill:r:i6riilH]l:*i:

il.ailtl6le,*$?

l@lrr:iiiidili9ai*

6 {-x'9irrriiiffiir:]:iilsrdilH- kr,:".ilillYlilaiSr]i@]]:

. Hl er" ". '

4Blll:r,ale}]lgE;*rAS9llllig,i',9,s \ \

i* t*u *.q &f *'*l,rtliurriil6liii:liil':qi e.-A d-d * *?*.Uirli8]1lliiili6iillSi6rlBllt:ri{i4!lllr;i\6lriliir_,"w g"

lgulalyit rrtlwtrliri

t-'--{(- 6- {lW];i$rrr.:181r;*r€#iqotri$lX(e*iti;illr9lliaiaryl

iisisilll9rXlrei;r!.

slli6llgbiiqA;;li91l;*ex,|6Xilffiill'drllgrlire

tuli9!:errri6ll)iYi6lidryiitqo:G

illt

$ti$ll

!Slrl,i11S

t9!111t

:i&

larsrl:t!v:.)):

y^,lll9il,i!glui:

Textbook definitions

may be helpful inproviding additional

examples for these

terms. These defini-

tions refer to the

techniques and

terms described in

this unit.

Resources

Glossary

Evaluation Tool,/Instrument - The test,questionnaire, or checklist used to measurethe performance of the instructional mate-rials to be evaluated.

Formative Evaluation - Evaluation doneduring development of instructional mate-rials for the purpose of rewriting andimproving the quality of the final product.

Measurement - Provides quantitative de-scriptions of the characteristics of avariable.

Qualitative Data - Non-numerical infor-mation often obtained through open-end-ed questions, observations, or interviews.

Quantitative Data - Numerical informa-tion from an evaluation. This informationcan be manipulated and may be reportedas an average, rank, median, mean, orstandard deviation.

Reliability - For our uses this refers to theconsistent nature of the evaluation tools'performance. Does the tool yield consis-tent results under similar circumstances?The degree of consistency determines thereliability of the tool.

Significance - The odds that observed dif-ferences or similarities are due to the"real" influence of variables and notchance.

Standard Deviation - A statistical funcdonthat serves as measurement of the degreeof agreement between responses for a spe-

cific criterion on a given piece of material.

Summative Evaluation - An evaluationdone to examine the end result of a projector activity. Materials developers mav con-duct a summative evaluation to measurewhether their product does what it rvas

intended.

Validity - An estimate of the degree towhich an evaluation tool measures u-hatit is designed to measure.

Variables - All the factors that could atttoresults of an evaluation project. For exam-ple, the evaluator may tire with a lengthltool and therefore rush through the evalu-ation of the last few pieces of instrucrionalmateriai.

Workshop Resource Manua I

Evaluated EE Activity Guides

Tbe Cwrricwla and Compendia Proiect,Six Volumes: Energy Resources,'WaterResowrces, Inte grate d Waste Management,Air Quality, Human Communities, andNatural Communities. California Depart-ment of Education, 1992-95. Sacramento,CA. These compendia contain the resultsof evaluations of K-12 environmentaleducation curricula based on presentation,pedagogy, teacher usability, general con-tent, and subject-specific content. Eachcompendium includes a "report card,"evaluator cornments, a brief description,and sample pages. Available from:

Bureau of Publications, Sales UnitCalifornia Department of EducationP.O.Box271Sacramento, CA 9 5812-0271

@ (916) 44s-1260Fax: 9L6 323-0823

"Curriculum Marerials for EE."D.L.Haury.J.E. Heimlich, and M. Norton, edited bvR.J. Wilke. In Enuironmental EducationTeacher Resowrce Book, A Practical Guidefor Enuironmental Education, L9 9 3.Milwood, NY Kraus International Publi-cations. A selected, annotated listing ofreadily available materials for use in envi-ronmental education. Included are curricu-lum guides, supplementary materials,activities, nonprint media, modules, etc.),program frameworks, and resource materi-als. Available from:

Kraus International Publications358 Saw Mill River Rd.Milkwood, NY 10546

@ (914) 762-2200

Educatin g Youn g Peop le Ab owt WaterA Gwide to Goak and Resources withan Ernphasis onNonformal and SchoolEnichment Settings. Elaine Andrews,1.992. Madison, 'WI: University of Wiscon-sin. This guide was developed by the WaterCurriculum Needs Assessment Projectto help curriculum coordinators select and

develop water quality training programsand curricula. Includes summaries of 63reviewed curricula. Available from:

Elaine Andrews

Universiry of WisconsinEnvironmental Resources Center216 Agricdture Hall1450 Linden DriveMadison, W'I53706

@ (608)2G2-6e68

Energy and the Enuironment: ATeacheisResource Guide. Pacific Gas and ElectricCompany, 1991. San Francisco, CA.A catalog ofenergy education resourcesfor elementarg secondary and highereducation. Available from:

Pacific Gas and ElectricEnergy and Environment Department77 Beale,28th FloorSan Francisco, CA 94106

@ $1s) 973-4361

"Evaluation of Selected RecyclingCurricula: Educating the Green Citizen,"S. Boerschig and R. DeYoung. InJowrnalof Enuironmental Edwcation, 24 :17 -22.1993.Evaltation of 14 solid wasre curric-ula. Provides a ratings marrix of evaluatedmaterials and recommendations for devel-opment of new materials.

Evaluation Tools"Analyzing Educational Resources," ZitaUnger, edited by John Fien. InTeacbing fora Sustainable'Woild: Enuironmental andDeuelopment Education Project for Teach-er Edwcation. 1,992. Brisbane, Australia:Griffith University Press. This workshopoutline includes guidelines for group dii-cussions, overhead masters) readings, andan evaluation instrument to help partici-pants become engaged in evaluation issues,particularly bringing to light rhe range ofideologies and viewpoints rhat are embed-ded in resources. Available from:

Dr. John FienFaculry of Environmental Sciences

Griffith UniversityNathan, Brisbane

Australia 4111fax 61-7-875-7459

,:,,,1a!ii*,,eii*i

riiiiajrtrrriGllr:,;j]

iriir!ltrtt,:at;G,:^:

,ll.,{,r.J:lt,reili;rl:l,$lllrriL:,rl%a'::;.:::t:F. * - '

l--" -:;l;t,,l?;,lr',,@,,,,$?rrlalll,,:il-,;lr-lr

* i,ler.rer,:#lro::4

- l:1 -ir:\

":

aiiU<i!:,r,,}:"'grl

:.lll::ri19i;l!'t,19li::ti_tilCt::6tt:t,?i''iii1plu,t:l:i..,,Crtr.i:l

I .... x /i

n'."ri, .*,ir ,-.1\

:9rrdl19llell!'iiit2l,.. .lillllr,llir. :

i,) ii'lis.iiiiiqad\iisiriia::l:13'llr1e];*lrir@:::i7l:lllelrl;Itiirri.l

l,,Xr,r*t;,l,:.rqi.*/r:_i

' ri .\* -li:l -, " \.:"- $*l-"",:::"\ rs\

;lli.:iitL!iriu:ilCrrr,r,iirrriru,,,.t:ra/|:-r1.r1

1-- * ,l,?il!ill',: i.9rr,"lrl?il"r t'-"-'.-, ;''- !

: ' .{."r:,1"? :, --r rri:lllllrn{:i,,,tr,, i:lYrrf

'*.'i"r':.t t .. - lgrrrr:llllr'?81!llllr'r',i.{

::rll*::,:a:.::4i,:a:aa:.:tlilaalS::arlr:''1i

ll!lrirrl|':iiia:1,,:117r.'

r''-t * -':irrallXl,lrei*9ierrrill;?tlait S

l::ti'riii0!;llll:'3

rc;l:r/6iIai..?,.,

!\ " r \I /. . -,

;llt:ri$ril:'.r,,c,,rrr

i.*,,0*i.ii.ll;

Eva I u ati n g Env t ron menta I Edu cati o n Materi a I s

"A Checklist for Reviewing EnvironmentalEducation Programs," Fred C. Niedermey-er.In Journal of Enuironmental Edwca-tion, 23 (2):46- 50. 1.992. Enumerates, in achecklist form, the characteristics of well-developed, effective instructional programsto provide criteria for their review

Enuironmental Education CwniculwmInuentory. J.R. Gardella, 1993. HighlandHeights, KY School of Education, North-ern Kentucky University. Includes twoforms that use accepted goals of environ-mental education in a checklist formatwith a data summary device to identify theemphasis of the evaluated curricular mate-rials. Available from:

Norrh l(entucky UniversityEducation DepartmentHighland Heights, KY 41099

@ \606) s72-s624

"Evaluating Sponsored Materials," MaryE. Haas. In How To Do It in the SocialStudies Classroom Seies.19 8 5 .

\Tashington, D.C.: National Council forthe Social Studies. Two evaluation formsare provided to carefully evaluate spon-sored materials and guest speakers. Avail-able from:

The National Councilfor the Social Studies

3501 Newark St., N$7'Washington, DC20016

@ (202) 966-7840

Additional Readings

lf you are interested in more informationon evaluation or want to do a formalevaluation project we recommend theseresources.

"The CIPP Model for Program Evalua-tion," D.L. Stufflebeam, edited by G.F.Madaus, M. Scriven, & D.L. Stufflebeam.In Eualuation Models. 1993. Boston: Klu-wer-Nijhoff. The CIPP approach is basedon the view thar rhe most important pur-pose of evaluation is not to prove but toimprove. Evaluation is seen as a tool bywhich to help make programs work bemerfor the people they are intended to serve.Available from:

Kluwer Nij hoff Publications101 Phillip DriveNorwell, MA 02061

@ (617) 871-6600

Citeria for Eualwating InstructionalProdwcts. James L. Moseley, 1987. ERICDocument Number: 281,-870. This tive-page checklist presents a rating scale ap-proach to evaluation instructionalproducts and educational media. Fifteencategories of evaluation criteria are includ-ed. Available from:

ERIC555 NewJersey Ave., NW\Tashington, DC 20208- 57 20(202) 21,9-2088

@ (8oo) s3B-3742

"Evaluating Environmental Education inthe Context of a Junior High School StateStudies Program," Dean Bennett. InJ ournal of Enuironmental Education,13(4): 13-18 . 1982. Describes the MaineStudies Curriculum, a state-validated dem-onstration project taught as part of theregular school curriculum. Also describesthe curriculum evaluation procedure andresults.

Worhshop Resource Manual

"Evaluation of Environmental EducationPrograms at the Elementary and SecondarySchool Levels," Bernard J. Lucko, John EDisenger, and Robert E. Roth. InJoumalof Enuironmental F ducation,l3 (4) : 7 -12.1982. Provides an overview of recent andcurrent activity in environmental educa-tion, particularly as it relates to programevaluation in elementary and secondaryschools. AIso includes malor steps in con-ducting an evaluation based on the goalsof environmental education.

"Evaluating Instructional Materials," TimS7entling and'$7i11iam E. Piland. In LocalLeader Guide IX: Locally Direaed Eualu-ation Handboo&. Second Edition.l 982.Springfield, IL: Illinois State Board of Edu-cation. One of 12 guides that have beendeveloped to facilitate evaluation by andfor local education agency personnel in Illi-nois. Covers the evaluation of both printand audiovisual instructional materials.Available from:

Curriculum Publ ications

Clearing House'Western lllinois Universiry

Horrabin Hall #46

Macomb, IL 61455

@ (8oo) 322-390s

"Evaluation of Natural Resource Educa-tion Materials: Implications for the Re-source Manager," Gerri Pomerantz. Int ournal of Enuironmental Education, 22(2):16-23.1.990. An anaiysis of elementa-ry school and natural resource lessons thatfocus on ecological principles, resourcemanagement issues, and analytical skill de-velopment. The conclusion is that veryfew of the lesson materials help to developcritical thinking skills and behaviors neces-sary for environmental action.

Exp eimental and Qwasi-Exp eimentalDesigns for Research. Donald T. Campbelland Julian C. Stanley, 1963. Chicago, IL:Rand McNally College Publishing Co.Examines the validity of 16 experimentaldesigns against 12 common threats to val-id inference. Available from:

Rand McNally College Publishing Co.444 N. Michigan Ave.

Chicago, lL 60611

@ (312) 321-17sl

Procedures for Deueloping an Enuiron-mental Education Curricwlum. HaroldHungerford and Robert Ben Peyton,1986.EE Series 22. Paris, France: UnitedNations Educational,Scientific and Cultural Organization. ERICdocument: ED 277 569. This publicationis an update of a 1980 document andrelates trends in environmental educationto curriculum development. Availablefrom:

Unesco

Environmental Education Unit7 place de Fontenoy

75352 Paris 07 SP

FRANCE

A Scope and Sequence for EnergyEducation Grades K-rz. Sonoma StateUniversiry Environmental Studies andPlanning Department, 1989. Aconceptualmatrix for evaluating criteria related toenergy education. Available from:

Sonoma State UniversityEnvironmental Studies and PlanningDepartment1801 East Copati Ave.

Rohnert Park, CA 94928@ (707) 664-2306

Standards for Eualuation of InstrwctionalMaterials with Respect to Social Content.California State Department of Education,1986. Overview of the California "Educa-tion Code" standards, which are intendedto help dispel negative stereotypes by em-phasizing people in diverse, positive, andcontributing roles. Available from:

Caiifornia State Departmentof Education

P.O.Box271Sacramento, CA 9 5 802-027 1

@ (e16) 6s3-7026

:@t:l(9>ue%q:SYhSll:6;:la;1:,!ili;3|:,,:1!r!!llAli-ilailGll;

t""t::t"":tt:"t*i I

i h "1; :'^lirs4igliaiiYi6,,

" :H:V \"i

:l9lliiiilvla$i:llrl;ir!it iiillwll.ilgll:*',ri:ii*llr.iiil6iiiiElllli

-is:,r, - !lltalicrralirlri:lr.: :

iilalilllriiilidltii6;:iliiit]tiau-::t]]ai:i:r:i:tiii6li:i;6r::*il{

l[ii;i&tt:ira$lu#i):- z- #-l* ;

.i$iieli%;!?:a

-b .- :') *:qliiidllln(alXliii!lrr:diu ii:111@ttt:i1@ut a:9- y,e,J4l

';,'a\a rl

ii:!rui6ri:tiaiii:, rra;il)*'*-" a*,-,Y,1: \

ialli:iii6l:ilGlillrl:iiiSltlaii:,AVlarrl6rXiaWle;:rrqi:it'Gtn6tii%rr;llalrWl;iXlrlr;:?3Yiiatilridiu::riau

rcff:ii:i6:ii;ui:,:ii16rr:

'"-.\ _.r : .

e>l13r:iielllr,:r,ii{elliiil9'hil9r;

1;a::]ir,:il.f, :,.t: L ;l

it$ryi$Lt${;l:::k::* \fl

,rlillilSl16ll9l,:iilliilsuhrleuiigtal1i):lgliiehrtl!:1

-::Lr* I

:rrr%:grarll9r;

"il) ," ""*.:

lii;iiiutli$1i6;iil

:gu;itgtaiiisSal:;: L T * i

-*i*'*"q1l

6t:*6t:tat]3iec,s,&,\- e

eli;ll:llr:iS9lli6:i€xtei*i@t*rl9

;iiiillr:iiilililllliiiler;

al- lt-,Y ^,

;r:ri@1,:iiilSlr:lCli;

I , .- i@uuiettal9uiili!xllellillt9r;i(1iiiiliBlrXlB!1elli;!

illiiiiSl::lgll:;:i?li

n-- F-..q --le'-v--s,r,r!rr$91!rle!i.elli

li:lii&l*iia\:iiili$rrllal:YASr;3ii:a!xy,&{,6,r

Ev a I u ati n g Env t ro n mental Edu cati o n Mater i a I s

AQwalitatiue Stwdy of How OneTeach er S elects Inshuctional Materials.John Elliot, 1986. ERIC document: ED269-381,. This case study explores waysin which a primary grade teacher selects in-structional materials. Areas coveredunder the study include: the text as a setcurriculum, reflections on the teacher'schoices of instructional materials, thesource of the teacher's instructional ideas,the teacher as "scrounger," instructionalmaterials and parent involvement, and in-structional materials as outside experience.Available from:

ERIC555 New Jersey Ave., N-W\Tashington, DC 20208- 57 20(202) 219-2088

@ (Boo) s3B-3742

"\7hat is the Purpose of Evaluation?"George T. O'Hearn. In Journalof Enuironmental Education, 1.3 (4): 1-3.1982. Provides a broad, working definitionof environmental education, suggestingthat such broad definitions necessitate theidentification of a few measurable objec-tives as "indicators" of an environmentaleducation program's impact. Discusses thepurpose of and methods for evaluation.

ENoNotEs

1. Kirkpatrick, Donald. "Evaluation," Trainingand Deuelopment Handbooh edited byRobert L. Craig, third edition. New York:McGraw-Hill, 1987.

2. United Nations Educational, Scientific, andCultural Organization. 1,97 8. Fir,al Report,Intergouernmental Conference on Enuiron-mental Education, Organized by Unesco inCooperation with UNEP, Tbilisi, Georgia,USSR, 14-26 October 1.977,Pans: UnescoEDI}'IDI49.

3. rbid

4. California Department of Education, Califor-nia Department of \7ater Resources andSonoma State University. EnuironmentalEducation: Comp endium for'Water Resources.Sacramento, CA, 1.992.

5. Pacific Gas and Electric Company. Energl'and the Enuironment: A Teacbers ResourceGuide. San Francisco, CA, L99L.

6. Tourtillott, Leeann. Conserue and Renetu,

Energy Education Actiuities for Grades 4 - 5.

Rohnert Park, CA: Sonoma State Universin',1990.

7. Andrews, Elaine. Educating Young People

about'V/ater: A Guide to Goals andResources with an Emphasis on Nonformaland School Enrichment Settings. Madison,'WI: University of 'Wisconsin, 1992.

8. As of this printing Project Learning Tree has

not yet published the results of their evalua-tion. For more information on this evalua-tion project contact Kathy McGlauflin, Direc-tor, Project Learning Tree, 1111 19th Street

N!7, Suite 800,'Washington, DC 20035.(202) 463-2468.

Wor kshop Resou rce Ma nu a I

r,i..|: I

,r .' ' l',:rf .r. r:, il|irJ;iilL]:il]],:l,'iit:r:_rtii.:i:t1,lit..lt',j

,

Ii'l,,t'.::t::.;, :t,.rt:1.-.t:ijr..-!,r!:+:!lrt::.i1iti,..-.-..,"