assessment - university college dublin | ucd is ireland’s

23
UCD TEACHING AND LEARNING/ RESOURCES www.ucd.ie/teaching Learning Journals and Logs ASSESSMENT Author: Jennifer Moon Email: [email protected] Date: 20 th January 2010

Upload: others

Post on 12-Sep-2021

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: ASSESSMENT - University College Dublin | UCD is Ireland’s

UCD TEACHING AND LEARNING/ RESOURCES

www.ucd.ie/teaching

Learning Journals and Logs

ASSESSMENT

Author: Jennifer Moon

Email: [email protected]

Date: 20th January 2010

Page 2: ASSESSMENT - University College Dublin | UCD is Ireland’s

Learning journals, logs and reflective diaries are terms often used interchangeably.However,thepurposesofthemmaydifferslightly.Whenkeepingalearningjournal,theemphasisisonmakingexplicitandrecordingthelearningthatoccurs.Reflectivediaries,as the name suggests, are more concerned with demonstrating reflection on anexperience,whilelogsarearecordofeventsthathavehappened.Theyusually,however,allhaveanaspectofreflectioninthem.For the purposes of this document they will be dealt with together, under the title oflearning journals, butwhile readingplease bear inmind that the terms canbeused todistinguishbetweensimilarbutdifferentdocuments.ThesolecontributortothisdocumentisJenniferMoon([email protected])BScMPhilMEdMSc PhD,whoworks in staff and learning development in the Staff DevelopmentUnit,UniversityofExeter.Shehasconsiderableandrecentexperienceaslearner,andasateacherboth inhighereducationandcontinuingprofessionaleducation,hasrunawiderange of courses includingmany in personal development and journalwriting. Shehaspublished books on reflection in learning and professional development, on learningjournalsandonrunningshortcoursesandworkshops(allKoganPage).WhatisaLearningJournal/ReflectiveDiaryJournals,logs,diaries,portfoliosarecontainersforwritingthatisrecordedoveraperiodoftime.Thewritingmayaccompanyaprogrammeoflearning,work,fieldworkorplacementexperienceoraresearchproject.Thejournals/logsanddiariescancomeinmanydifferentguisesandbeusedtofulfildifferentpurposes.Workonprofilingandrecordingachievementcanincorporatejournaltechniquesandthiswebpagewillcovermanyideasthatarecommonalsotoworkondevelopingandmanagingportfolios.Learningjournals/diariesandportfoliosareincreasinglyusedinhighereducationasmeansoffacilitatingorofassessinglearning.Theyhavemanydifferentpurposesandthestructurethatisintroducedneedsbothtorelatetotheirpurposeandtothestyleofthelearner.Generallyspeaking,theyseemtobehelpfulinpersonalisinganddeepeningthequalityoflearningandinhelpinglearnerstointegratethematerialoflearning‐suchasthatfromdifferentmodulesortheoreticalandpracticallearning.Theymaybehighlystructuredor'free'andtheyhavebeenusedtoimprovelearninginvirtuallyeverysubjectareaincludingmathematicsandthesciences,andparticularlyinprofessionaldevelopment.

Page 3: ASSESSMENT - University College Dublin | UCD is Ireland’s

LearningjournalsLearningjournalsarenotonething.Theycomeinallshapesandsizes,includingtapesandvideosandinelectronicforms.Whilethefocusofthisarticlewillbeonpenonpaperjournals,itisinterestingtospeculateonhowtheuseofdifferentmediamightaffecttheprocessofwritingandthelearningthatresults.Generallyspeakingwhatdistinguishesalearningjournalfromotherwritingisthatitfocusesonongoingissuesovertimeandtherewillbesomeintentiontolearnfromeithertheprocessofdoingitorfromtheresultsofit.Thissuggeststhatitisnot,simply,aneventsdiaryorarecordorlog.Alearningjournalisusuallyavehicleforreflection.Itseemsreasonabletoassumethatalladultsandolderchildrenreflect,butsomemorethanothers.Forsome,reflectionrepresentsanorientationtothewaytheylivetheirlives.Othersreflectwhenthereisanincentivetoreflect,orwhenguidanceorconditionsintheirenvironmentareconducivetoreflecting.Alearningjournalmightbeseenasaparticularaccentuationoftherightconditionsforreflectionandforlearningfromtheprocess.Therearemanydifferentwordsthatmaymeanthesameasalearningjournalsuchaslog,diary,common‐placebook,think‐placeandmanyportfoliosincludereflectivewritingthatissimilartothedescriptionabove.Asjournalwritingbecomesmorecommoninhighereducation,theremightwellbeassumptionsthattheprocessshouldonlybeofinteresttothoseworkingindisciplinesthatare,insomeway,literary.Thisisnotthecase.Journalwritingcanbeofuseatmoststagesofeducation(fromfiveorsixup),acrossanydisciplineandcanbenefitanysituationinwhichsomeoneistryingtolearnsomething.Inhighereducation,mostworkonjournalsseemstohavebeendonebyteachereducators,butthevalueofwritingajournalcanbeasgreattophysicsormathsstudents.ThisisparticularlythecasesincetheremaybeadditionalbenefitstogainfromthepracticeofwritingitselfaswellasotherlearningALearningjournalis:• diverse;• notnecessarilywritten‐butmostofthetimeassumewrittenformat;• generallyreflectiveandaccumulatedoveraperiodoftimewiththeintentiontolearn,i.e.notpurelydescriptive;• flexible(itcanbestructuredorunstructured)• ausefulback‐uptolearning;• somethingthataccentuatesfavorableconditionsforlearning‐e.g.space,time,reflection.• applicabletoalldisciplines(notjustliterary).

Page 4: ASSESSMENT - University College Dublin | UCD is Ireland’s

Whywritelearningjournals?'Whywritealearningjournal?'Thequestionmeansboth'Whatwillyougetoutofit?'and'Forwhatpurposeisitset?'.Weconsiderthefirstquestionbyprovidingsomethingoftheessenceofjournalwriting,throughthecommentsofwriters.Inwritingajournal'wetakesomethingfrominsideourselvesandwesetitout:itisameansofdiscoveringwhoweare,thatweexist,thatwechangeandgrow.Thepersonaljournalhasbeenusedforhundredsofyearstoarticulatethehumandramaoflivingandtoexplorenewknowledge.'(Wolf,1989)'Journalsallowareticentstudenttoestablishanopinionaboutatopicbeforebeingaskedtospeakaboutitpublicly….Journalwritingturnsstudentsintoactivelearners­it'sdifficulttofallasleepwhilewriting!Journalwritingalsohelpsstudentstorelaxwhentheywriteandhelpsthemtofindtheirownvoiceandrhythm'(Carlsmith,www)'…thatiswhatmyjournalsareabouttothisday.MomentsofbeingintheworldthatIwanttosave.PicturesoftheworldthatIhavewitnessed…..Torereadthejournalistoseeoneselfseeing.'(Grumet,1990)'Keepingajournalisahumblingprocess.Yourelyonyoursenses,yourimpressionsandyoupurposelyrecordyourexperiencesasvividly,asplayfully,andascreativelyasyoucan.Itisalearningprocessinwhichyouarethelearnerandtheonewhoteaches.'(Holly,1991:4)'Oneofthemostengagingusesofpersonalstudentjournalsisasamirrorofthemind.Inthismode,journalsinvitelearnerstofindlanguagedeepwithinselftoarrayone'shopes,dreams,disappointments,concernsandresolves…..Theresultisthatstudentsoftenexpressastonishmentanddelightatthekaleidoscopicself­portraitswhichemergefromthepagesoftheirnotebooksastheyjourneythroughacourse.'(Bowman,1983)'Journalwritingholdsbeforethewriter'seyeoneimageafteranotherforcloserinspection:isthisoneworthmorewords,moredevelopment?….Intheacademicworld,whereweteachstudentstogainmostoftheirinformationfromreadingandlistening,wespendtoomuchtimetellingourstudentshowtoseeordoingitforthem.That'snothowIwouldencouragecritical,creative,orindependentthinking.Ourstudentshavegoodeyes;letsgivethemnewtoolsforseeingbetter:journal

Page 5: ASSESSMENT - University College Dublin | UCD is Ireland’s

writingis,ofcourse,oneofthosetools.'(Fulwiler,1986)'Finally….Iwouldliketorecommendsomespecificactionsthatlearnersmighttaketoimprovetheirlearningandstudying….tokeepareflectivediary,makinganentryatleastonceaweek.'(Hartley,1998­intheconcludingremarkstohisbookonlearningandstudying).Wehavemadenoattemptheretoseparatethequotationsintothosethatarisefromjournalworkinformaleducational,professionaldevelopmentsettingsorthosethatarepersonal.Thereisoftennotacleardividinglinebetweenthem,evenbetweenthosethatare'set'andthosethatarestartedfrompersonalvolition..Intermsofpurposesforjournals,aperusaloftheliteraturemightsuggestthateverytimealearnerchoosestowriteorisaskedtowritealearningjournal,adifferentpurposefortheprocessisgiven.InareviewofoverahundredpapersonjournalwritingIfoundaroundeighteenpurposesforjournalwriting.(Moon,1999).Itisimportanttonotethatmostjournalswillfulfillmorethanonepurpose,andthatthepurposessetbyatutorarenotnecessarilythesameasthosethatwillbefulfilledorperceivedbyastudent.AnexampleofthiscomesfromSalisbury(1994).ThestudentsstudiedbySalisburyperceived‐rightlyorwrongly‐that'selfflagellation'wasvaluedbytutorswhosettheirjournalwork.Theypurposefullysetouttoprovideplentyofconfessionalandflagellatorymaterial.Theeighteenpurposesare:• Torecordexperience• Tofacilitatelearningfromexperience• Tosupportunderstandingandtherepresentationofthatunderstanding• Todevelopcriticalthinkingorthedevelopmentofaquestioningattitude• Toencouragemetacognition• Toincreaseactiveinvolvementin,andownershipof,learning• Toincreaseabilityinreflectionandthinking• Toenhanceproblemsolvingskills• Asameansofassessmentinformaleducation• Toenhancereflectivepractice• Forreasonsofpersonaldevelopmentandselfempowerment• Fortherapeuticpurposesorasmeansofsupportingbehaviourchange• Toenhancecreativity• Toimprovewriting• Toimproveorgive'voice';asameansofselfexpression• Tofostercommunication;inparticularreflectiveandcreativeinteractionwithinagroup• Tosupportplanningandprogressinresearchoraproject• Asameansofcommunicationbetweenonelearnerandanother

Page 6: ASSESSMENT - University College Dublin | UCD is Ireland’s

LearningfromLearningJournalsWefocusonfourmeansoflearningfromjournals.1. Firstly,we learn because journalwriting is a process that accentuates favourable

conditionsforlearning.

‐Itproducesintellectualspaceinwhichwecanthink.‐ Italsoencourages independent learning ‐youhave towriteyourown journalandbecause you 'own' the learning, it is likely that it will be more meaningful to you(Rogers,1969).‐Writingajournalalsoprovidesafocusingpoint,anopportunitytoorderthoughtsandtomakesenseofasituationorofinformation.‐Learningfromajournalenhanceslearningskillsbecauseitforcesthelearnertocopewith'messyinformation':ideasthatarenotstraightforward.‐Itcounteractswhatmightbeseenasspoon‐feedingwithhandoutsandlecturenotesonthewww.

2. Secondlyjournalwritingencouragesreflectionandreflectionisassociatedwith

deepapproachestolearning,orwithdeeplearning.Indeeplearning,theintentionofthelearneristodevelopapersonalunderstandingofthematerialandtorelateittowhatisalreadyknown.Thefreedomofjournalwritingcansupportthelearner'sattempttounderstand.

3. Thirdly,writinginajournalencouragesmetacognition,andthelearningofthose

withametacognitiveviewoftheirfunctioningisgenerallybetter(Flavell,1979).Itislikelythatmuchfreewritinginjournalswillcontainsomemetacognitionandifjournalsarestructured,thenmetacognitioncanbebuiltin.

4. Lastly,theactofwritingisassociatedwithlearningortheenhancementoflearning.Thereisaconsiderableliteratureontherelationshipofwritingtolearning,howitforcesalearnertoclarifyherthoughts,howitisapowerfulformoffeedbacktothelearner,howitfocusesattentionandtellsthelearnerifs/hedoesordoesnotunderstand.Aparticularlyinterestingconsiderationrelatesmorespecificallytothelanguagethattendstobeusedinwritingjournals.Peopleuseexpressivelanguageinjournals,alanguagemorelikeconversationthanmostotherformsofwritingand,possibly,morelikethelanguageofthought.Wemaybeabletolearnbetterforthisformoflanguagethantheformalacademiclanguageintowhichweareinductedinmoreadvancedformsofeducation.Elbow(1981)triestogetatthemeaningofthis.Whenhisstudentsareworkingonjournals,hedescribesthemasworkingwithpowerorvoice,'Iliketocallthispower"juice".Themetaphorcomestomeagainandagain.IsupposebecauseIamtryingtogetatsomethingmysteriousandhardtodefine."Juice"combinesthequalitiesofmagicpotion,mother'smilkandelectricity.SometimesIfearIwillneverbeclearaboutwhatImeanbyvoice.Voice,inwriting,implieswordsthatcapturethesoundofanindividualonthepage…Writingwithno

Page 7: ASSESSMENT - University College Dublin | UCD is Ireland’s

voiceisdead,mechanical,faceless.Itlacksanysound'.(Elbow,1981:286­7)

TheProcessofwritingreflectively:PresentationandDeepeningofReflectionIf we set reflective writing as a task, we should know what we mean by it. This isparticularlyimportantfortworeasons.Therearemanyreportsthatsomestudentsdonotfinditeasytowritereflectively‐perhapseitherbecausetheyhavethenotionofacademicwriting so ingrained as a habit, or because they are simply not reflective. To teachstudents to write reflectively, one needs to know what one means by it. The secondreasonrelatestoassessment.Cansomethingbeassessedifthereisnotgoodknowledgeofwhatitis,ofitsprocess?Experienceofintroducingreflectiveactivitiesinaworkexperiencemodulehassuggestedthatatwo‐stageguidanceprocesstoreflectionmaybehelpfultostudents.The'presentingreflection'stageutilisesapproachesthatintroducetheideaofreflection.Asecondstageofguidancefocusesondeepeningreflection.Atbothstages,multipleapproachesprovidingdifferentideasandactivitiesaroundreflectionseemtobemoresuccessfulthanattemptsatverbalinstruction.Suggestedactivities/approachesarelistedbelowwithreferencestosomeresourcesintheAppendices.Startingtheprocess:ThePresentationofReflectiontoStudentsThefirststage­presentingreflection:Whatisreflection?Studentsneedtoknowthattheyallcanreflect,butthatitmaynotbeahabitthatsomeuse. It can be helpful to give them a simple definition such as that used earlier in thispaper.Ifstudentsfeelthattheydonotknowwhat'beingreflective'is,itcanbeusefultoalmost trick themintobeingreflective foramomentaskingthem, forexample, to thinkaboutwhattheyhavelearntfromexperiencesofpaidworketc.Itmaythenbehelpfultousethemapofreflectivewriting(Appendix3)asanindicationofthekindofeventsthatmightbeinvolvedintheprocessofreflection.Considerwhyreflectionisbeingusedtofacilitatethisareaoflearning? Theresponsewilldependonthepurposefortheworkinwhichreflectionisinvolved.Theanswermightincludethefollowingtheideathatweusereflectioninordertolearnfromsituations inwhich there isnocurriculumbutwherewehave tomakesenseofdiverseobservations, ideas and data as well as personal research (e.g. by asking questions).Reflection is used to make sense of unstructured situations in order to generate new

Page 8: ASSESSMENT - University College Dublin | UCD is Ireland’s

knowledge. It is important tobeclear that theactivitymightbe introducing theskillofreflective learning or generating knowledge by using reflection to make sense ofsomething.Considerhowreflectiondiffersfrommorefamiliarformsoflearning?We tend to use reflection when we are trying to make sense of how diverse ideas fittogether,whenwearetryingtorelatenewideastowhatwealreadyknoworwhennewideaschallengewhatwealreadyknow(i.e.takingadeepapproachtolearning).Reflectionis theprocessweusewhenworkingwithmaterial that ispresented inanunstructuredmanner‐notorganisedandpurifiedasinatraditionalcurriculum.Theissuesaroundtheuseofthefirstperson­'I'.Most studentswill have learnt that they should not use the first person singular in anacademicenvironment.They canbe confused if theyare suddenlybeingencouraged touse'I'.Itmaybehelpfulheretotalkaboutthemannerinwhichknowledgeisconstructedwiththeinvolvementoftheindividualknower.Theuseofthefirstpersonacknowledgesthisprocess.Giveexamplesofreflectivewriting­goodandpoor.Studentsfindrealexamplesofreflectivewriting,learningjournals,evenpublishedwork(fictionorbiography)helpful.ExamplessuchasTheParkandThePresentationprovidesanexampleofreflectivewritingtowhichwewillreferseveraltimesinthissection.Itconsistsofthreeaccountsofthesameevent,writtenatthreedifferentlevelsofreflectivity.Italsoprovidessomecriteriathatattempttodistinguishbetweenthelevelsofreflection.Atthisstageofpresentingreflection,itwillbesufficientsimplytopresenttheaccountswithoutthecriteria(possiblyjustthefirsttwo)andusethemasabasisfordiscussion.Studentscanbeaskedwhichisthemostreflectiveandwhy.Generatediscussionsofstudents'conceptionsofreflectionItisusefulatsomestage(perhapsasaspin‐offfromanotheractivity)toencouragestudentstotalkaboutwhattheythinkreflectionis.ThiswillprovideanopportunityformisconceptionstocometolightForexample,somestudentswillconsiderthatyouonlyusereflectionwhensomethinghasgonewrong‐decidingwhatcouldbedonebetternexttime.

Page 9: ASSESSMENT - University College Dublin | UCD is Ireland’s

EnablepracticeandprovideopportunitiesforfeedbackStudentscanbeaskedtoreflectontheirownperformanceinsomething‐forexample,theirperformanceingivinga5minutetalk.Theytalkandthenwriteareflectiveaccountofhowtheirperformancewent,weaknessesandstrengths,assessmentagainsttheirexpectations,relationshipstopresentationsgivenbeforeetc.TheimpactoftheactivitycanbeincreasediftheyareaskedtowriteadescriptiveaccountoftheirperformancebeforetheywritereflectivelyGiveastartingexercisethateliminatestheblankpage.Blankpagesarethreateningtomany(butexcitingtosome).Itisagoodideatogetstudentsstartedontheirreflectiveworkbygettingthemtodosomereflectivewritingbeforetheyknowtheyhavereallystarted.Thiswillmeanthedevelopmentofsomestructuresuchasquestionsthatwillstimulatereflectivewriting.Haveothertoolsavailabletohelpstudentstogetstarted.Thereareplentyofexercisestoencouragereflectivewriting.Theuseoftheseexercisesinoccasionalclasssituationscanhelpstudentstoexpandtheareasinwhichtheyarethinkingandtobegintodeepentheirreflection.Expecttohavetosupportsomestudentsmorethanothers.Itmaybepossibletodevelopasystemofpeersupport.Beopenaboutyourneedtolearnaboutthisformoflearningandhowtomanageit.Demonstratingthatitisnotonlystudentswhoneedtolearntoreflectcanbeveryhelpfulforstaffandstudents.Staffmightwritealearningjournalabouttheprocessofhelpingstudentstolearnreflectively‐andshareelementsofitwiththestudents.Whataretheproblemswithstartingtowriteajournal?Thingstothinkabout:• planning• purpose• needtobeseentofitwithincoursedesign• getthetitleright• isitvoluntaryorcompulsory?• willitbefreewritingorstructured?• howyouwillmanagethedemandsoftheclass• how,andif,itwillbeassessed• issuesoftrustinassessment

Page 10: ASSESSMENT - University College Dublin | UCD is Ireland’s

• ifthereistobeanysharedreading,thinkaboutprivacyandconfidentiality• whenisthejournaltobewritten?• evaluation• medium• format• howwillthestudentslearntowritereflectively?• willtherebelimitsonlength?• howoftenwillstudentswriteintheirjournals?Thesecondstage,deepeningreflectiveworkThedeepeningofreflectiveactivitydependspartlyondevelopingawarenessoftheconstructednatureofknowledgeandunderstanding,forexample:

‐ thateventscanbeconceivedofdifferentlyaccordingtotheframeofreference;‐theroleofemotionsinguidingourconceptionsofeventsorpeople;

‐ thatdifferentdisciplinesrelyondifferentstructuresofknowledgeandhavedifferentwaysofworkingwithknowledge.

UseexamplestodemonstratedeeperreflectiveactivityWesuggestedtheuseofmaterialsuchas'ThePresentation'.Thefocusnowwouldbeonthethirdaccountandtheuseofthecriteriathatdistinguishthedeeperaccounttothemoredescriptiveaccounts.Introduceaframeworkthatdescribeslevelsofreflection:AnexampleisHattonandSmith(1995).Theframeworkbelowresultedfromworkwithstudents' reflective writing work and below it is in simplified form. It influenced thecriteriausedin'ThePresentation'Descriptive writing: This is a description of events or literature reports. There is nodiscussion beyond description. This writing is considered not to show evidence ofreflection.Itisimportanttoacknowledgethatsomepartsofareflectiveaccountwillneedtodescribethecontext‐butinthiscase,writingdoesnotgobeyonddescription.Descriptivereflection:Thereisbasicallyadescriptionofevents,buttheaccountshowssomeevidenceofdeeperconsiderationinrelativelydescriptivelanguage.Thereisnorealevidenceofthenotionofalternativeviewpointsinuse.Dialogicreflection:Thiswritingsuggeststhatthereisa'steppingback'fromtheeventsand actions which leads to a different level of discourse. There is a sense of 'mulling

Page 11: ASSESSMENT - University College Dublin | UCD is Ireland’s

about', discoursewith self and an exploration of the role of self in events and actions.There is consideration of the qualities of judgements and of possible alternatives forexplaining and hypothesising. The reflection is analytical or integrative, linking factorsandperspectives.Criticalreflection:Thisformofreflection,inadditiontodialogicreflection,showsevidencethatthelearnerisawarethatthesameactionsandeventsmaybeseenindifferentcontextswithdifferentexplanationsassociatedwiththecontexts.Theyareinfluencedby'multiplehistoricalandsocio‐politicalcontexts',forexample.(HattonandSmith,1995)Introduceexercisesthatinvolve'standingbackfromoneself'.E.g. students write about their own processes of learning using a semi‐objective andcriticalstance. Introduce exercises that involve reflection on the same subject from differentviewpointsofpeople/socialinstitutionsetc.E.g.studentscouldbeaskedtoreflect(ortalk/present)onaneventinashopfromthepointofviewofthesupervisor,customer,counterassistant,onlookerandsoon. IntroduceanexerciseinreflectiononthesamesubjectfromviewpointsofdifferentdisciplinesIn terms of different disciplinary standpoints, students might be asked to describe achild'spetdog fromthepointofviewofpractitioners insociology,psychology,medicalsciences,English,artandsoon.Introduce an exercise that involves reflection that is influenced by emotionalreactionstoeventsStudents can be asked to describe a real or imaginary event and to write fictitiousreflective accounts at periods after the event, each account illustrating a change ofemotional orientation to the event. The important point here is that emotional stateinfluences themanner inwhich a subject is viewed. If the state changes, the viewmaychange.

Page 12: ASSESSMENT - University College Dublin | UCD is Ireland’s

Collaborative methods of deepening reflection ­ e.g. critical friends and group,activitiesetc.Somemethodsinvolvesmallgrouporpairwork.Thegroupswillneedtohavecommonideas about methods by which to deepen reflection and to see themselves as peerfacilitators.Thegroupsorpairsmaywork togetheroveraperiod, learninghowbest tohelpeachotherbypromptingandaskingquestions,queryingframesofreferenceandsoon.Second­orderreflectionSecondorder reflection is represented in any technique that requires a student to lookthrough previous reflective work and write a reflective overview. One of the mostconvenientwaystodothisisthedoubleentryjournal.Studentswriteonlyononepageofadoublespreadorononehalfofaverticallydividedpage.Theyleavespaceblankuntilatanother time, they go through the initialmaterialwriting generating further commentsthatemergefromtheirmorecoherentoverviewoftheinitialwork.Journalsindisciplinarycontexts­someexamplesoftheirusesTheusesof journalsaredescribedbelowunderdisciplineheadings,butmanycouldbeappliedinmuchbroaderfields.Thesciences,engineeringandmathematicsSelfe,PetersenandNahrgang(1986)studiedthemannerinwhichjournalwritingcouldhelpmathematicsstudents.Initiallytheintentionwastocomparethetestgradesofagroupwhowereaskedtowritejournalswithanothergroupusingtraditionalmethods,however,theyfoundthattheinfluenceofjournalwritingwasmoresubtle.Thegeneralfindingintheinitialinvestigationwasthatjournalwritingwasnobetternorworsethanactivitiesinvolvingtestingorquizzesatpromotinglearning.Onasubsequentmoresubtleinvestigation,journalsappearedtofacilitatelearninginanumberofways.Byencouragementtothinkinamannerthatwastheirownandtousetheirownlanguage,thestudentswereabletodeveloppersonalconceptualdefinitionswhichweremuchmoreunderstandablethantechnicaldefinitions.Theconcretenatureofthisthinkingfacilitatedcomprehensionandapplicationofabstractconceptsandtheybegantoevaluateorappreciatetheusefulnessoftheconcepts.Theothertwoeffectsrelatetotheabilitytosolveproblems.Therewasevidenceinthewritingthatstudentswererecordingstrategiesthattheyfoundhelpfulinproblemsolving.Furthermore,inwritingaboutproblemsinsteadofjustworkingoncalculation,theywerecomingtosolutionsthroughthewriting.Anexcerptfromthewritingofonestudentillustrateswellthelastpoint:

Page 13: ASSESSMENT - University College Dublin | UCD is Ireland’s

'IseenothingincommonwiththethreefunctionsexceptthatthederivativehasapowerofN‐1justlikealltheotherderivativeshave.Oh‐waitasec,nowIseehowyoudidit.Youtookthederivativeofthefirsttermand….'(Selfe,PetersenandNahrgang,1987:200)Selfe worked also with engineering students (Selfe and Arbabi, 1986), with a primaryintention of introducing more writing into the course. The students, in a structuralanalysisanddesignclass,wereaskedtowriteatleastapageaweekontheirexperiencesof the course.While their initial reactionwas negative, and for a few (around 10%) itremainednegative,mostfoundthat'ithelped..(themto)...clarifytheirthoughts,workoutstrategies for solving engineering problems, understand the important aspects of thestructurescourseandidentifyareasinwhichtheyneededmorehelp'(p185).Incontrastto a control group, those who had written in journals wrote final reports that 'weregenerally more coherent, organized and complete and in their description of methodsused to solve engineering problems were more complete. Instructors felt much moreinformedabouttheirstudents'processesoflearning.Grumbacher (1987) focused on the ability of physics students to solve problems. Sheobserved thewriting processes of students whom she considered to be good problemsolvers. They articulate the problems clearly, use visualization and verbalization in thesolving and they are aware of the relative appropriateness of their responses. Moresignificantly 'they use their learning logs to synthesize their new knowledge aboutphysicswiththeirpriorknowledgeandexperiences'(p325).EnglishandalliedsubjectsLindberg(1987)appliestheuseofjournalstohelplearnerstogainadeeperunderstandingoftextsinEnglish.Heusesadoubleentryjournal.Thisisaformofjournalwhereonesideofthepage‐oronecolumn‐isforthedescriptivewritingandtheothercolumnisforreflectiveobservationorfurtherprocessingofthistype.InLindberg'sjournaldesignstudentswritetheirobservationsofandreactionstothetextononesideofthepage.Thesemayinclude'timeswhenyourreadingchangesyouaresurprisedorpuzzled…somethingjustdoesnotfit…yourfirstimpressionoftheending'.Whenthestoryisread,theyareaskedtogobackandmakesenseoftheobservationsintheothercolumn.Thesejournalsarediscussedinaplannedseriesofonetoone'conferences'withtutors.Assessmentisofapaperthatstudentswriteonthebasisoftheirjournalentries

Page 14: ASSESSMENT - University College Dublin | UCD is Ireland’s

ThehumanitiesandsocialsciencesWagenaarintroducedajournalinsociologyinordertoencouragestudentstorelatethetheorytaughtonthecoursetotheirownobservationsandexperiences(Wagenaar,1984).Insodoinghesuggeststhat 'thejournalassignmentcanassisttheinstructorinmeetingthe higher level cognitive objectives in her or his course'. He describes it as 'Anintellectualexerciseinreflexivity'whichexploitsthehigherorderfunctionsofevaluation,application(Bloom1956).Studentswerenotaskedtowriteabouttheirfeelingsbutfromthe examples given, feelings were present and acceptable in their expressions. Theprocessof journalwritingwasrelativelysimple.Twoelementsweretobepresent‐ theobservationofbehaviourandthediscussionintheoreticalterms.Baltensperger (1987),workingwith geography students describes a similar element inhisworkwith classroom journals. He posed a question to students and asked them towritetheirresponsesfirst.Hewouldthenaskthequestionagain,requestingoralreportsbased on the written ideas, and then would open the discussion for more generalcomment.Hefoundthisavaluablemeansofcombatingproblemsofpoorresponsetooralquestionsposedinclass.Intwopapers,separatedby14years,Hettichadvocatedjournalsinpsychology(Hettich,1976,1990). Journalshelpstudentstoconnectcourselearningtotheirrealexperiencesandobservations.Theemphasisisonthecoursematerialandinstructionsindicatedthatentriescouldcontain'examplesthatshowcomprehensionoftheconcept;applicationorexperimentation with principles; and analysis, evaluation and synthesis of courseconcepts'(Hettich,1990). Inaprojecttoinvestigatetheimpactofdifferentformsofwritinginanthropology,Creme(1999)describesthreedifferentusesofjournalsinasocialanthropologydepartment.Projectlogsaccompaniedafirstyearcoursewhichintroducedthetheoreticalandconceptualbasistoanthropologyresearch.Logsaccompanyaresearchprojectandcontainrecordingsandreflectionontherecords.Thesecondformoflogiscalledarecordofstudy.Studentsdevelopanaccountoftheirlearningfromdifferentsourcesthroughoutthecoursewithafocusonthedevelopmentof'understandingofcentralcourseconcepts'.Thethirdtypeofjournalwasusedinafirstyearmultidisciplinarycourseondeath.Intheirjournals,studentsexploredtheirreactionstovariousaccountsofdeathandtotheirpersonalexperiences.LanguagesMulhaus and Loschmann (1997) discuss issues relating to foreign language learning inmodularized higher education. The change from integrated programmes to theaccumulation ofmodulesmeans that students enrolling on amodule are likely to havediversebackgroundsinrelationtothesubjectmatter.Adistancelearningapproachwas

Page 15: ASSESSMENT - University College Dublin | UCD is Ireland’s

adoptedwith students usingworkshop time for advice and progress checks.With thisapproach,thewritersconsideredthatstudentsneededtogiveattentiontotheirlearningstrategiesandtheyusedjournalstoaddressthatneed.Journalsincludedthewrittenworkof students ‐ video summaries, vocabulary lists, worksheet tasks and self‐reflectivecomments.Amarkingschemeassessed'theinteractionwiththematerialandthedepthofthe learning process as evident from the students' vocabulary lists, translations andcomments'(p25).ArtssubjectsOneformofjournalthatisanaturalaccompanimenttothecreationofmusicanddramaistheproject typeof journalwhere thedevelopmentof ideas is recordedand consideredreflectively.Theaimistoenhancethethoughtprocessesthatcontributetotheproject.Withartanddesignstudents,Davies(1998)setsasimilarjournalthataccompaniesprojectwork,butwherethemainaimisoneofassessment.Daviessuggeststhattherecanbetoomuchfocusontheoutcomeofartstudentworkandnotontheall‐importantprocessofit.Thejournalisalocationfortherecordingofthinkingprocesses.BusinesssubjectsandlawNovember (1993) commentsofhis subject ‐ commerce ‐ that the studyenvironment isnot conducive todeep approaches to learning.Heuses journals in order todeepen thequality of learning in a final year course. In his experience, the best results occurwiththerehasbeenconsiderableguidanceonthejournalwritinggiven.Onemethodistoaskstudents towritea listofproblemsor issuesofconcern,andthentoexamineeach inasystematicmanner.Every‐so‐oftenheasksstudentstoreflectonquestionsthatareraisedintheirwriting.TheAssessmentofReflectiveWritingThere aremanywhoargue that journals and reflectivewriting shouldnotbe assessed.Thiscommentfromconvincedproponentofjournalssumsupsomeoftheissues.'How can youmark an individual's own personal development? I think it's a right andproper part of education for us to encourage students to express their feelings so thatthey know it's alright to have those feelings. However, for me to mark those feelingsseems inconsistent and incongruent.Marks can also create a barrier or obstacle to thepersonfindinghisorherownvoice…'(SisterCraigcitedinDillon,1983)

Page 16: ASSESSMENT - University College Dublin | UCD is Ireland’s

Intermsofresearch,SumsionandFleet(1996)questiontheassessmentofreflectioninmoregeneralterms.• Theynotethatthereisnoevidencethatareflectiveprofessionalismoreeffectivethananon‐reflectiveprofessionalorthatprogrammesthatpromotereflectionleadtobetteroutcomes.• Inaseriesofexperimentstheysoughttoexploretheissueintheirworkwithfinalyearearlychildhoodeducationstudents.• Theyfacedthedifficultyoffindingasuitablesetofcriteriabywhichtocodethe'reflectiveness'ofthestudents'work.• Theyreviewedandrejectedanumberofinstrumentsbecausetheyweretoocomplexortheywerecomposedoftoomanycategoriestoobtainsufficientinter‐coderreliability.• Theyconcludedthatthereisnogoodmeansofassessingreflectivewritingatpresent.Itisveryeasytogoalongwithargumentsthatassessmentofreflectivewritingorjournalsisintrusiveonpersonaldevelopmentoristoodifficult.However:• wedoneedtodevelopmeansofassessmentwhile,atthesametime,takingabroadviewoftheprocess.Studentsarebeingassessedonjournalsandreflectivewriting.Evenwherejournalwritingisaassessedonthebasisof'competent'or'notyetcompetent',orpassorfail/notyetpass,thecriteriabeingusedareoften'gutreactions'orpersonalinterpretationsandthisqualityofmarkingisnotfairtostudents.Whenrefectionandreflectivepracticearesohighlyesteemedinsomeareasofeducationandprofessionaldevelopment,weshouldbeabletodobetterthanthis.• Thesecondjustificationisbasedontheobservationthat,evenablelearnersmaynotfindreflectivewritinginajournaleasy(WildmanandNiles,1987).Unlessteachershaveanunderstandingofthetaskthattheyaresettingandthequalitiesinitthatconstituteagoodperformance,theywillnotbeabletohelptheselearners.• Thirdly,reflectivewritingisrootedinthenatureofthehighereducationsystemasitseemstoexist.Theriseofthe'strategic'studentiswelldocumented(egKneale,1997).The'strategicstudent'isboundonsuccessinherstudiesfortheminimumoutputandwillthereforenotputeffortintotasksthatarenotassessed.Ifwebelievethatjournalscontributetolearning,thentheassessmentofjournalsmaybenecessaryinordertoensuretherequisitestudenteffort.IssuesinassessingjournalsThe firstquestiontobeasked inassessing journals iswhetherweare lookingtoassessthe process of reflection or the product of learning. For example, Selfe, Petersen andNahrgang(1986)describetheuseofjournalswithmathematicsstudents.Theaimofthework isnot todevelop reflective skills in these students,but to improve their learning.

Page 17: ASSESSMENT - University College Dublin | UCD is Ireland’s

The quality of their reflection is incidental. An essay or viva will test the learning.Alternativelypartsofthe journalcouldbere‐writteninsummaryatassessmenttimetofocus on the learning. In contrast, journals used in teacher education are often toencourage the process of reflection and the learning that results is relatively lessimportant.

It isveryimportanttorememberthatassessingjournalshasaneffectonthemannerinwhichthejournaliswrittenbecausestudentswilltrytowritewhattheythinkisrequired.Theymaybewaryofrevealingpersonaldifficultiesorconcerns.AssessingthereflectiveprocessofajournalAssumingthatitisthereflectionthatisimportant,howdoweapproachassessment?Somegeneralcriteriacanhelpfullyindicateadequacy.Alistmightinclude:Length,Presentationandlegibility,Numberofentriesorregularityofentries;Clarityandgoodobservationinpresentationofeventsorissues;Evidenceofspeculation;Evidenceofawillingnesstoreviseideas;Honestyandself‐assessment;Thoroughnessofreflectionandself‐awareness;Depthanddetailofreflectiveaccounts;Evidenceofcreativethinking;Evidenceofcriticalthinking;EvidenceofadeepapproachtothesubjectmatterofthejournalRepresentationofdifferentcognitiveskills(synthesis,analysis,evaluationetc);Relationshipoftheentriesinthejournaltoanyrelevantcoursework,theories,etc.Matchofthecontentandoutcomesofthejournalworktocourseobjectives,learningoutcomesforthejournalorpurposesthatthejournalisintendedtofulfill.Questionsthatarisefromthereflectiveprocessesandonwhichtoreflectfurther.Thereareanumberofschemesofassessmentcriteriathataredevelopedfromtheworkofothers.TheexampleofHettich(1976and1990)andhisuseoftheBloom'sTaxonomywasgivenearlier.ThesixcategoriesinBloom'scognitivedomainare(inorderofincreasingsophistication),knowledge,comprehension,application,analysis,synthesisandevaluation.WedmanandMartin(1986)usedVanManen'slevelsofreflectivity(VanManen,1977):Level1‐technicalrationality:effectiveapplicationoftechnicalknowledgeinordertoreachknownoutcomes.Level2‐practicalrationality:teachers'abilitytodealwithpracticalactionswheretherearemultiplefactorsinoperation.Thereisanabilitytocopewiththeconfusionandmakeanassessmentofthelikelyeducationalconsequences.Level3‐InWedmanandMartin'sterms,thislevelofcriticalrationality'focusedon

Page 18: ASSESSMENT - University College Dublin | UCD is Ireland’s

incorporatingconsiderationofmoralandethicalcriteriaintodiscourseaboutpracticalaction.Thecentralquestionatthislevelwaswhicheducationalgoals,experiencesandactivitiesledtowardformsoflifethatwerejustandequitable.' TheworkofHattonandSmith(1995)arosefromsomewhatmorethoroughresearchthanmanyofthestudiesdescribedearlier.Whiletheworkderivesfromthecontextofteachereducation,thedescriptionsofreflectivenessinwritingaregenerallyapplicable.Asummaryoftheframeworkwithquotationsfromitisprovidedbelow.Descriptive writing (which is considered not to show evidence of reflection), is adescriptionofeventsorliteraturereports.Thereisnodiscussionbeyonddescription.Descriptivereflection:Thereisdescriptionofeventsbutsomejustificationinrelativelydescriptivelanguage.Thepossibilityofalternativeviewpointsindiscussionisaccepted.Reflectionmaybe'basedgenerallyononeperspectivefactorasrationale'or,presumablyinamoresophisticatedform,isbased'ontherecognitionofmultiplefactorsandperspectives'.Dialogicreflection:'Demonstratesa"steppingback'fromtheeventsandactionsleadingtoadifferentlevelofmullingaboutdiscoursewithselfandexploringthediscourseofeventsandactions'.Usesthe'qualitiesofjudgmentsandpossiblealternativesforexplainingandhypothesising'.Thereflectionisanalyticalorintegrative,linkingfactorsandperspectives.Itmayrevealinconsistency'inattemptingtoproviderationalesandcritique'.Criticalreflection:'Demonstratesanawarenessthatactionsandeventsarenotonlylocatedwithinandexplicablebymultipleperspectives,butarelocatedinandinfluencedbymultiplehistoricalandsocio‐politicalcontexts'(HattonandSmith,1995)Aninterestingquestionthatemergesconcernsthedegreetowhichlearnerscanbecoachedtowriteatthesedifferentlevels.Forexample,Dart,Boulton‐Lewis,BrownleeandMcCrindle(1998)usedtheHattonandSmithdescriptors(above)asanassessmenttoolintheirresearchontheincreaseofknowledgefromtheuseoflearningjournals.Mostoftheirsubjectsproducedwritinggaugedtobe'descriptivereflection'.Ifthesubjectshadbeencoachedinthedifferentdegreesofreflectivenessintheirwriting,wouldtheyhaveorcouldtheyhavefunctionedatamoresophisticatedlevel,then,perhapschangingtheresultoftheexperiment?Theformsofassessmentofreflectivewritingabovemostlyservetheirpurposewithinthecontextoftheiruse.Theyareprobablyasreliableasanyotherformofassessmentinhighereducationbuttherearesomeremainingquestionsoftheirvalidityifweareunsurewhatreallyhappenswhenstudentswritereflectively.Withaframeworkfortheprocessofreflectivewritingandtakingtheliteratureofreflectionintoaccount,itispossibletoconsiderdevelopmentofcriteriaforassessmentpurposes.Italsobecomespossibletoshowstudentswhyjustdescriptiveworkisnot'reflective'inthatitonlycoversasmall

Page 19: ASSESSMENT - University College Dublin | UCD is Ireland’s

partoftheoverallprocess.Exampleofassessmentindicatorsforreflectivewriting

Purpose.Thelearnerdemonstrates:‐awarenessandunderstandingofthepurposeofthejournal,usingthepurposetoguideselectionanddescriptionofevent/issueonwhichtoreflect.Thelearneridentifies:‐herorhisownpurposeforthejournalorjournalentry.Thedescriptionofaneventorissue:‐ispresent.Thedescription:‐providesanadequatefocusforfurtherreflection;Itincludes:‐astatementofobservations‐commentonpersonalbehaviour;‐commentonreaction/feelings;‐commentoncontext.Additionalideas:‐arepresent.Thelearnerdemonstrates:‐theintroductionof(any)additionalideastothedescription;theadditionof:‐furtherobservations;‐relevantotherknowledge,experience,feelings,intuitions‐suggestionsfromothers;‐newinformation;‐formaltheory;‐otherfactorssuchasethical,moral,socio‐politicalcontext.Reflectivethinking:‐ispresent.Thelearnerdemonstrates:‐theabilitytoworkwithunstructuredmaterial‐thelinkingoftheoryandpractice;‐theviewingofanissue/eventfromdifferentpointsofview;‐theabilityto'stepback'fromasituation;‐metagcognitiveprocesses;‐'cognitivehousekeeping';‐applicationoftheoreticalideas;‐considerationsofalternativeinterpretations;etc.OtherprocessingThereisevidenceofotherprocessing‐e.g.‐newideasaretestedinpractice;‐newideasarerepresented,forexample,inafirstdraftorgraphicformetcandthereisevidenceofreviewandrevisioninalatercopy.

Page 20: ASSESSMENT - University College Dublin | UCD is Ireland’s

AproductresultsTheisastatementof:‐eithersomethingthathasbeenlearnedorsolvedthatrelatestothepurposeortheproblematicnatureofthedescriptionorthereisasenseofmovingon.Forexample,thereIsidentificationofanewareaforfurtherreflectionoranewquestionIsframed.

Page 21: ASSESSMENT - University College Dublin | UCD is Ireland’s

ReferencesinreflectivewritingBaltensperger, B (1987) 'Journals in economic geography', in T Fulwiler, The JournalBook,Heinemann,Portsmouth,NewJerseyBloom,B(1956)TaxonomyofEducationalObjectives,Longmans‐Green,NewYorkBurnard,P(1988)'Thejournalasanassessmentandevaluationtoolinnurseeducation',NurseEducationToday,8,pp105‐107Carlsmith(www)'Academicalnotebook'Crème,P(1999)'Newformsofstudentandassessmentinsocialanthropology:researchthroughpracticeattheUniversityofSussex',NationalNetworkinTeachingandLearningAnthropologyFDTLProject1997–1998Dart,B,Boulton‐Lewis,G,Brownlee, JandMcCrindle,A(1998) 'Changeinknowledgeoflearning and teaching through journal writing', Research Papers in Education 13, (3),pp291–318Davies,J(1998)PapergivenatImprovingStudentLearningConference,Brighton(1998).Dillon,D(1983)'Self‐discoverythroughwritingpersonaljournals',LanguageArts,60,(3)pp373–379Elbow,P(1981)WritingwithPowerTechniquesforMasteringtheWritingProcess,OxfordUniversityPress,NewYorkFlavell,J1979)'Metacognitiveaspectsofproblem‐solvingbehaviour',inLResnick,(ed),Thenatureofintelligence,LawrenceErlbaum,HillsdaleFulwiler,T(1986)'Seeingwithjournals',TheEnglishRecord,32,(3),pp6‐9Fulwiler,T(1987)TheJournalBook,Heineman,Portsmouth,NewHampshireGrumbacher,J(1987)'Howwritinghelpsphysicsstudentsbecomebetterproblemsolvers',inTFulwiler,TheJournalBook,Heinemann,Portsmouth,NewHampshireGrumet,M(1990)'Retrospective‐autobiographyandtheanalysisofeducationalexperience',CambridgeJEd.20,(3),pp321‐325Hartley,J(1998)LearningandStudyingRoutledge,LondonHatton,NandSmith,D(1995)'Reflectioninteachereducation‐towardsdefinitionandimplementation',TeachingandTeacherEducation,11,(1),pp33‐49Hettich,P(1976)'Thejournal,anautobiographicalapproachtolearning',Teachingof

Page 22: ASSESSMENT - University College Dublin | UCD is Ireland’s

Psychology,3,(2),pp60‐61Hettich,P(1990)'Journalwriting:oldfareornouvellecuisine?',TeachingofPsychology,17,(1),pp36‐39HollyM(1991)KeepingaPersonal‐ProfessionalJournalDeakinUniversityPress,VictoriaKneale,P(1997)'Theriseofthe"strategicstudent":howcanwecope?',inMArmstrong,GThompsonandSBrown,(eds)FacinguptoRadicalChangesinUniversitiesandCollegesSEDA/KoganPage,LondonKing,PandKitchener,K(1994)DevelopingReflectiveJudgement,Jossey‐Bass,SanFranciscoLindberg,G(1987)'Thejournalconference:fromdialectictodialogue'inTFulwiler,TheJournalBook,Heinemann,Portsmouth,NewHampshireMoon,J(1999)ReflectioninLearningandProfessionalDevelopment,KoganPage,LondonMoon,J(1999a)LearningJournals;ahandbookforacademics,studentsandprofessionaldevelopment,KoganPage,LondonMcCrindleAandChristensenC(1995)'Theimpactoflearningjournalsonmetacognitiveprocessesandlearningperformance',LearningandInstruction5,(3),167‐85Moon,J(2001)ShortCoursesandWorkshops:improvingtheimpactoflearning,trainingandprofessionaldevelopment,KoganPage,LondonMulhaus,SandLoschmann,M(1997)ImprovingindependentlearningwithauralGermanprogrammes',inR.Hudson,SMaslin‐ProtheroandLynOates,FlexibleLearninginAction,SEDA/KoganPage,LondonNovember,P(1993)'Journalsforthejourneyintodeeplearning',ResearchandDevelopmentinHE,16,pp299‐303Progoff,I(1975)AtaJournalWorkshop,DialogueHouseLibrary,NewYorkRogers,C(1969)FreedomtoLearn,CharlesE.Merrill,ColumbusOhioSalisbury,J(1994)BecomingQualified‐anethnographyofapost‐experienceteachetrainingcoursePhDthesis,UniversityCollegeofWales,CardiffSelfeC,Petersen,BandNahrgang,C(1986)'Journalwritinginmathematics'inAYoungandTFulwiler(eds)WritingAcrosstheDisciplines,Boynton/Cook,UpperMontclair,NewJerseySelfeCandArabi,F(1986)'WritingtolearnEngineeringstudentsjournals'InAYoungandTFulwiler,WritingAcrosstheDisciplines,Boynton/Cook,UpperMontclair,NewJerseySumsion, J and Fleet, A (1996) 'Reflection: can we assess it? Should we assess it?',AssessmentandEvaluationinHE21,(2),pp121–130

Page 23: ASSESSMENT - University College Dublin | UCD is Ireland’s

VanManen,M(1977)'Linkingwaysofknowingandwaysofbeing',CurriculumInquiry,6,pp205–208Wagenaar,T(1984)'Usingstudentjournalsinsociologycourses',TeachingSociology,11,pp419‐437Wedman,JandMartin,M(1986)'Exploringthedevelopmentofreflectivethinkingthroughjournalwriting',ReadingImprovement,23,(1),pp68‐71Wildman, T andNiles, J (1987) 'Reflective teachers, tensions between abstractions andrealities',JournalofTeacherEducation,3,pp25‐31Wolf,M (1989) 'Journalwriting:ameans toanend ineducatingstudents toworkwitholderadults',GerontologyandGeriatricsEducation,10,pp53‐62Young,AandFulwiler,T(1986)WritingacrosstheDisciplines,Boynton/Cook,UpperMontclair,NewJersey