the missing link? using reflection advance qualitative ...d-scholarship.pitt.edu/32957/1/corrall...

The Missing Link? Using Reflection to Advance Qualitative Evaluation and Assessment

Sheila Corrall [email protected]

School of Computing and Information

Information Culture & Data Stewardship

Outline•  Defini,onsofkeytermsandconcepts(seealsoCorrall,2017)•  Background

–  Reflec,veprac,ceinlibraries–  Reflec,onandreflexivityin(qualita,ve)research

•  Tools,modelsandframeworksfacilita,ngreflec,vity–  Reflec,veprocessfordevelopmentalevalua,on[handout]–  Triangulatedreflexiveinquiryques,ons[handout]–  Readingdatathreeways–  Analy,cmemosreflec,ontopics

•  ReferencesAcknowledgement•  [email protected]


WorkingDefini,onsofKeyTermsandConcepts•  Developmentalevalua-on–anevalua,onwherethefocusison

changeorinnova,onincomplexdynamicenvironments–  alsoknownasreal-,meevalua,on,emergentevalua,on,ac,on

evalua,on,andadap,veevalua,on(PaVon,2016)•  Reflec-on–theabilitytothinkaboutthepast,inthepresentfor

thefuture(Carroll,2009)–  aformofmentalprocessingthatweusetofulfilapurposeorto

achievesomean,cipatedoutcome(Moon,2006)•  Reflexivity–theabilitytolookbothinwardsandoutwardsto

recognizeconnec,onswithsocialandculturalunderstandings(Fook&Gardner,2007)–  thinkingcri,callyaboutwhatyouaredoingandwhy,

confron,ngando^enchallengingyourownassump,ons,andrecognizingtheextentthatyourthoughts,ac,onsanddecisionsshapehowyouresearchandwhatyousee(Mason,2002)


Reflec,vePrac,ceinLibraries•  Widelyrecognizedasessen,alforprofessionaldevelopment•  Par,cularlyassociatedwithimprovingthecompetenceof

librariansasinforma,onliteracyinstructors/teachers+suppor,ngstudentsasreflec,velearners(Corrall,2017)

•  Alsodescribedasacrucialcomponentofwhatittakestobeanevidence-basedprac,,oner-researcher(Wilson,2016)–  andafocusareaforassessmentlibrarians(Oakleaf,2013)

•  Butsignificantdeficienciesinthetreatmentofreflec,oninlibraryliteratureonresearch,evalua,onandassessment–  coverageispoorwhencomparedwithcognatedisciplinesØ  offersliVleornoprac,calguidanceonusingreflec,onor

onbecomingareflec,veandreflexiveresearcher


Reflec,onin(QUAL)ResearchandEvalua,on

•  Explicitintegralpartofclassicfour-stepac,onresearchcycle–  plan,act,observe,reflect

•  Assumeddimensionofconceptual/designphaseofresearch–  reflec,veinquiry(Hernon,Dugan&Nitecki,2011)

•  Alsoimportantinformula,ngques,ons,reviewingliterature,interpre,ngresults,andvalida,ngfindings

•  Corecharacteris,cofqualita,veresearch,par,cularly(concurrent)datacollec,on/analysisandreport-wri,ng–  especiallyethnography,phenomenologyandcasestudies


ReflexivityinQualita,veResearchandEvalua,on•  Thequalita,veresearcheristheinstrumentofinquiry

–  yourprofessionaland/orpersonalbackground,interests,andmo,va,onsmayhaveimplica,onsforyourstudy

Areyouanoutsideorinsider?Areyouanexpertorlearner?Areyoudoingyourresearchon,for,orwithyourpar,cipants?

Ø  Thereportofaqualita,vestudyshouldexplaintheposi,onandperspec,veoftheresearchertostrengthenitsvalidityHowareyourexperiences,thoughts,andfeelingsaffec,ng

howyouconductyourstudy,whatyouseeorhearinthefield,andhowyouanalyze,represent,andinterpretyourdata?

•  Qualita,veresearchersshouldreflectonsuchques,ons,andrecordtheirresponsesinfieldnotesand/oranaly,cmemos–  memoingfacilitatesthetransi,onfromcodingtorepor5ng


Using Reflective Practice to Investigate an Issue or Evaluate an Initiative

AData-Based,Story-Based,Engagement-BasedInterac,veApproachØ  Reflec,vePrac,ceProcessfor

DevelopmentalEvalua,on(PaVon,2011)


2.Sharestories

3.AnalyzepaVernsand

themes

4.Iden,fyimplica,ons:

ac,ons,lessons

1.Ques,onfor

focusedreflec,on

Reflec,vePrac,ceCycleforOngoingDevelopmentalEvalua,on

REFLECTIVEPRACTICE

(PaVon,2011,p.269)


Reflec,vePrac,ceforDevelopmentalEvalua,on1.  Iden,fyafocusforinquiryandlearningthatprovides

direc,onandvisiontothedesiredchange.2.  Turntheconcept,idea,value,orvisionintoanexperien,al

inquiryques,on.3.  Sharepersonalexperiencesthatrespondtotheques,on.4.  Groupmemberscanaskshortclarifyingques,ons.5.  Par,cipantsiden,fypaVernsandthemesinthestories.6.  Ifthereismorethanonegroup,eachgroupreportstheir

themesandpaVernstothefullgroup.7.  Thegrouppicksoneortwothemesfordiscussionthathave

importantimplica,ons(andlessons)fortheworkathand.8.  Generateac,onagreementsandnextstepsforreflec,on.cf.What?SoWhat?NowWhat?(Driscoll/RolfeinCorrall,2017)


Reflec,vePrac,ceProcess:ExamplesandGuidelines1.  Iden,fyafocus:Transformingacademicliaisonfroma

service-as-supporttoanengagementorpartnershipmodel2.  Formulateaques,on:Whatexperienceshaveyouhadof

successfullyengagingorpartneringwithstudentsorfaculty?3.  Sharepersonalexperiences:Mustusefirsthandexperiences

andtellthestoryin3-5minutes(don’texplainoranalyze)4.  Askclarifyingques,ons:Agreeconfiden,ality,beprepared

foremo,ons/strongfeelings,andenforce,mesensi,vely.5.  Iden,fypaVernsandthemes:Commonspecificobserva,ons

andmorecategoricalgeneraltopicalcross-cupngmeanings.6.  ReportpaVernsandthemes:Iden,fyperson(s)toreportout.7.  Discussandanalyzeimplica,ons:Lookforlessonslearned.8.  Agreeac,onsandnextsteps:Formulateanotherques,on…


Becoming a Reflective and Reflexive Researcher Frameworks/Models/ToolsYouCanUseØ  TriangulatedInquiry:ReflexiveQues,ons

(PaVon,2015)Ø  ReadingData:ThreeLevels(Mason,2002)- Literally,Interpre,vely,Reflexively

Ø  Analy,cMemos:Reflec,onTopics(Saldaña,2015)


ReflexiveScreens:

culture,age,gender,class,socialstatus,educa,on,

family,poli,calpraxis,language,values

Myself,asinquirer:HowdoIknowwhatIknow?

Whatshapesandhasshapedmyperspec,ve?

Peopleinthese5ngbeingstudied:

Howdotheyknowwhattheyknow?Whatshapesandhasshapedtheirworldview?Howdotheyperceiveme?Why?HowdoIknowthat?HowdoIperceivethem?

Audiencesforthestudy:

HowdotheymakesenseofwhatIgivethem?Whatperspec,vesdotheybringtothefindingsIoffer?Howdotheyperceiveme?HowdoIperceivethem?

(PaVon,2015,pp.72,604-605)

ReflexiveQues,ons:TriangulatedInquiry


HowDoYouIntendto“Read”YourData?(Mason,2002)•  Aliteralreadingwillfocusonelementssuchasform,content,

structure,sequence,style,andlayout–  formandstructureofdialogue,wordsandlanguageused

•  Aninterpre-vereadingwillgothroughorbeyondthedatatoconstruct/documentwhatyouthinktheymeanorrepresent–  notonlywhatiscontainedinadocumentorimage,but

whatorwhoisnotthere;alsothecontextofproduc,on,consump,on,interpreta,onanduse

•  Areflexivereadingwilllocateyouaspartofthedataandexploreyourrole,perspec,ve,andinfluenceintheirgenera,onandinterpreta,on(e.g.,viatriangulatedinquiry)

Yourinten,onshaveimplica,onsforhowyoucollect/generateyourdataandfieldnotes,andrepresentthemastextorgraphics


Reflec,onTopics:Reac,veorProac,veMemo-Wri,ng•  Howyoupersonallyrelatetothepar,cipantsand/orthephenomenon•  Yourstudy’sresearchques,ons•  Yourcodechoicesandtheiropera,onaldefini,ons•  Thepar,cipants’rou,nes,rituals,rules,roles,andrela,onships•  EmergentpaVerns,categories,themes,concepts,andasser,ons•  Thepossiblenetworksandprocesses(links,connec,ons,overlaps,flows)

amongthecodes,paVerns,categories,themes,concepts,andasser,ons•  Anemergentorrelatedexistenttheory•  Anyproblemswiththestudy•  Anypersonalorethicaldilemmaswiththestudy•  Futuredirec,onsforthestudy•  Theanaly,cmemosgeneratedthusfar•  Thefinalreportforthestudy

CodingDataßàUnderstandingPhenomena:ReciprocalRela,onship

MemosasConversa5onswithYourselforLe<erstoaFriend(Saldaña,2015)


ReferencesCarroll,M.(2009).Frommindlesstomindfulprac,ce:Onlearning

reflec,oninsupervision.PsychotherapyinAustralia,15(4),40-51.Corrall,S.(2017).Crossingthethreshold:Reflec,veprac,cein

informa,onliteracydevelopment.JournalofInforma5onLiteracy,11(1),23-53.

Fook,J.,&Gardner,F.(2007).Prac5singcri5calreflec5on:Aresourcehandbook.Maidenhead,UK:OpenUniversityPress.

Hernon,P.,Dugan,R.E.,&Nitecki,D.A.(2011).Engaginginevalua5onandassessmentresearch.SantaBarbara,CA:LibrariesUnlimited.

Mason,J.(2002).Qualita5veresearching(2nded.).London:SAGE.AvailableathVp://www.sxf.uevora.pt/wp-content/uploads/2013/03/Mason_2002.pdf.

Moon,J.A.(2006).Learningjournals:Ahandbookforreflec5veprac5ceandprofessionaldevelopment(2nded.).Abingdon,UK;NewYork:Routledge.


ReferencesOakleaf,M.(2013).Buildingtheassessmentlibrarianguildhall:Criteria

andskillsforqualityassessment.JournalofAcademicLibrarianship,39(2),126-128.

PaVon,M.Q.(2011).Developmentalevalua5on:Applyingcomplexityconceptstoenhanceinnova5onanduse.NewYork:GuilfordPress.

PaVon,M.Q.(2015).Qualita5veresearch&evalua5onmethods(4thed.).ThousandOaks,CA:SAGE.

PaVon,M.Q.(2016).Stateoftheartandprac,ceofdevelopmentalevalua,on.InM.Q.PaVon,K.McKegg&N.Wehipeihana(Eds.),Developmentalevalua5onexemplars:Principlesinprac5ce(pp.1-24).NewYork:GuilfordPress.

Saldaña,J.(2015).Thecodingmanualforqualita5veresearchers(3rded.)(pp.43-65).London;ThousandOaks,CA:SAGE.

Wilson,V.(2016).Prac,,oner-researchersandEBLIP.InD.Koufogiannakis&A.BreVle(Eds.),Beingevidencebasedinlibraryandinforma5onprac5ce(pp.81-91).London:Facet.

the missing link? using reflection advance qualitative ...d-scholarship.pitt.edu/32957/1/corrall...

Documents