testing a test: evaluating our assessment tools
DESCRIPTION
This slideshow was used for teacher training workshops I conducted in the fall of 2011 at the Center for English as a Second Language, University of Arizona (Tucson, USA).TRANSCRIPT
![Page 1: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/1.jpg)
‘Tes%ngatest’–Evalua%ngourAssessmentTools
EddyWhite,Ph.D.
AssessmentCoordinator
CenterforEnglishasaSecondLanguage
UniversityofArizona
![Page 2: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/2.jpg)
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
2
![Page 3: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/3.jpg)
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
3
![Page 4: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/4.jpg)
(1994‐2009)
![Page 5: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/5.jpg)
![Page 6: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/6.jpg)
Classroom-based Assessment
• AssessmentofLearning
• AssessmentforLearning
![Page 7: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/7.jpg)
![Page 8: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/8.jpg)
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
8
![Page 9: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/9.jpg)
Thegoalofassessmentisto...
9
![Page 10: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/10.jpg)
The goal of assessment has to be, above all, to support the improvement of
learning and teaching.
(Fredrickson&Collins,1989)
10
![Page 11: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/11.jpg)
definiGon:ClassroomAssessment
Assessment
Planning
CollecGng
Analyzing
ReporGng
![Page 12: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/12.jpg)
ESLAssessment‐Purposes• idenGfystrengthsandweaknessesofindividualstudents,
• adjustinstrucGontobuildonstudents’strengthsandalleviateweaknesses,
• monitortheeffecGvenessofinstrucGon,• providefeedbacktostudents(sponsors,parents,etc.),and
• makedecisionsabouttheadvancementofstudentstothenextleveloftheprogram.
12(Source:ESLSeniorHighGuidetoImplementaGon,2002)
![Page 13: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/13.jpg)
Consider • Researchsuggeststhatteachersspendfromone‐quartertoone‐thirdoftheirprofessionalGmeonassessment‐relatedacGviGes.
• Almostalldosowithoutthebenefitofhavinglearnedtheprinciplesofsoundassessment.
(S%ggins,2007)
![Page 14: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/14.jpg)
Teacherslearnhowtoteachwithoutlearningmuchabouthowtoassess.(Heritage,2007)
14
![Page 15: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/15.jpg)
Assessmentliteracy
• thekindsofassessmentknow‐howandunderstandingthatteachersneedtoassesstheirstudentseffecGvely
• AssessmentliterateeducatorsshouldhaveknowledgeandskillsrelatedtothebasicprinciplesofqualityassessmentpracGces
(SERVECenter,UniversityofNorthCarolina,2004)
![Page 16: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/16.jpg)
AssessmentLiteracy
Know‐howandunderstandingteachersneedtoassessstudentseffec%velyand
maximizelearning
![Page 17: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/17.jpg)
• Wemaynotlikeit,butstudentscananddoignoreourteaching;
• howeveriftheywanttogetaqualificaGon,theyhavetoparGcipateintheassessmentprocesseswedesignandimplement.
(Brown,S.2004.Assessmentforlearning.LearningandTeachinginHigherEduca0on,1,81‐89)
Importanceofclassroomassessment
![Page 18: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/18.jpg)
43/W
![Page 19: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/19.jpg)
Whoaretheassessment‘deciders’atyour
insGtuGon?
![Page 20: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/20.jpg)
Classroom-Based Assessment: Challenges, Choices, and Consequences
![Page 21: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/21.jpg)
AssessmentFrameworks
![Page 22: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/22.jpg)
Assessmentframework
• ‐theseriesofassessmenttools(exams,tasks,projects,etc.)thatarescoredandusedtoarriveatasumma%vegradeforacourse
• ‐itshouldbeskills‐basedandknowledge‐based(i.e.SsdemonstratewhattheyknowaboutandcandowithEnglish)
• basedonlearningoutcomes
![Page 23: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/23.jpg)
(Rowntree,1987)
• Thespiritandstyleofstudentassessmentdefinesthedefactocurriculum.
defacto=exisGnginfact,actual,whetherintendedornot
![Page 24: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/24.jpg)
![Page 25: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/25.jpg)
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
25
![Page 26: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/26.jpg)
QuizGme!
26
![Page 27: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/27.jpg)
AssessinganEnglisharGclesquiz
Context
• ConversaGonclass(listening&speaking)
• high‐beginnerlevel27
![Page 28: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/28.jpg)
Whatisafundamentalproblemwiththisquiz?
28
![Page 29: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/29.jpg)
Answer
29
![Page 30: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/30.jpg)
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
30
![Page 31: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/31.jpg)
Whatisatest?
31
![Page 32: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/32.jpg)
Atest...• isamethodofmeasuringaperson’sability,knowledge,orperformanceinagivendomain.
• isaninstrument–asetoftechniques,procedures,oritems–thatrequiresperformanceonthepartofthetest‐taker.
32
![Page 33: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/33.jpg)
Tests–measuringfunc%on
33
![Page 34: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/34.jpg)
Atestmustmeasure• Sometestsmeasuregeneralability,whileothersfocusonveryspecificcompetenciesorobjecGves.
• Examples
• AmulG‐skillproficiencytestmeasuresgeneralability;
• aquizonrecognizingcorrectuseofdefinitearGclesmeasuresveryspecificknowledge.
34
![Page 35: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/35.jpg)
• Atestmeasuresperformance,...
• but,theresultsimplythetest‐takersability,orcompetence.
35
![Page 36: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/36.jpg)
• Performance‐basedtestssamplethetest‐takersactualuseoflanguage,
• butfromthosesamplesthetestadministratorinfersgeneralcompetence.
36
![Page 37: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/37.jpg)
• Awell‐constructedtestisaninstrumentthatprovidesanaccuratemeasureofatest‐taker’sabilitywithinaparGculardomain.
• Construc%ngagoodtestisacomplextask.
37
![Page 38: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/38.jpg)
Yourassessmentprac%ces?
38
![Page 39: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/39.jpg)
Thinkaboutwhatis
happeninginyourcontextandyour
assessmentpracGces
![Page 40: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/40.jpg)
• Inventories• Checklists• PeerRaGng• SelfRaGng• Journals• Porkolios• ObservaGons• Discussions• Interviews
• True–FalseItem• MulGpleChoice• CompleGon• ShortAnswer• Essay• PracGcalExam• Papers/Reports• Projects• QuesGonnaires• PresentaGons
YourassessmentpracGces?
Howdoyouassessyourstudents?
![Page 41: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/41.jpg)
Foryou,whichofthefourskillsaremore/lesschallengingtotest?
41
![Page 42: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/42.jpg)
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
42
![Page 43: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/43.jpg)
QuizGme!
43
![Page 44: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/44.jpg)
2010
![Page 45: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/45.jpg)
• Exploringhowprinciplesoflanguageassessmentcanandshouldbeappliedtoformaltests.
• Theseprinciplesapplytoassessmentofallkinds.
• Howtousetheseprinciplestodesignagoodtest.
45
![Page 46: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/46.jpg)
• Whatarethe‘fivecardinalcriteria’thatcanbeusedtodesignandevaluatealltypesofassessment?
46
![Page 47: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/47.jpg)
Q.HowdoyouknowifatestiseffecGve,appropriate,useful,or,indown‐to‐earth
terms,a“good”test?
47
![Page 48: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/48.jpg)
Fivekeyassessmentprinciples?
• Discuss• 3minutes
• Hint(fivenouns)48
![Page 49: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/49.jpg)
Fivekeyassessmentprinciples
• PracGcality• Reliability• Validity• AuthenGcity• Washback
49
![Page 50: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/50.jpg)
50
![Page 51: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/51.jpg)
KeyAssessmentPrinciples
![Page 52: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/52.jpg)
• ThesequesGonsprovideanexcellentcriteriontoevaluatethetestswedesignanduse.
52
![Page 53: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/53.jpg)
53
![Page 54: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/54.jpg)
1.PracGcality
54
• IstheprocedurerelaGvelyeasytoadminister?
![Page 55: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/55.jpg)
Prac%calityconsidera%ons
• thelogisGcalandadministraGveissuesinvolvedinmaking,givingandscoringanassessmentinstrument
• theamountofGmeittakestoconstructandadminister
• theeaseofscoring• easeofinterpreGng/reporGngtheresults
55
![Page 56: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/56.jpg)
AneffecGvetestisprac%cal.
Thismeansthatit:• isnotexcessivelyexpensive• stayswithinappropriateGmeconstraints
• isrelaGvelyeasytoadminister,and
• hasascoring/evaluaGonprocedurethatisspecificandGmeefficient
56
![Page 57: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/57.jpg)
ThevalueandqualityofatestsomeGmeshingeonsuchni`y‐gri`yprac%cal
considera%ons.
57
![Page 58: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/58.jpg)
• InclassroombasedtesGng,_________isalmostalwaysacrucialpracGcalfactorforbusyteachers.
58
![Page 59: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/59.jpg)
59
![Page 60: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/60.jpg)
2.Reliability
60
• Is all work being consistently marked to the
same standard?
![Page 61: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/61.jpg)
• Areliabletestisconsistentanddependable.
• Ifyougivethesametesttothesamestudentormatchedstudentsontwodifferentoccasions,thetestshouldyieldsimilarresults.
61
![Page 62: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/62.jpg)
Whatfactors
contributetothe
unreliabilityofatest?
62
![Page 63: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/63.jpg)
TestUnreliability‐contribuGngfactors
• Studentrelatedreliability• Raterreliability(inter,intra)• Testadministra%onreliability
• Testreliability63
![Page 64: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/64.jpg)
Q.Whatisonekeywaytoincreasereliability?
A.Userubrics
64
![Page 65: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/65.jpg)
• Rubricsarescoringguidelines.• Theyprovideawaytomakejudgmentsfairandsoundwhenassessingperformance.
• Auniformsetofpreciselydefinedcriteriaorguidelinesaresetforthtojudgestudentwork.
65
![Page 66: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/66.jpg)
66
![Page 67: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/67.jpg)
3.Validity
‐ mostcomplexcriteria
‐ mostimportantprinciple
• Does the assessment
measure what we
really want to
measure?
![Page 68: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/68.jpg)
Validity‐definiGon
• ‘Theextendtowhichinferencesmadefromassessmentresults
areappropriate,meaningful,andusefulintermsofthepurposeof
theassessment.’
(Gronlund,1998,p.226)68
![Page 69: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/69.jpg)
• Avalidtestofreadingability...
• actuallymeasuresreadingability–
• notmathskills
• orpreviousknowledgeinasubject
• norwriGngskills• norsomeothervariableofquesGonablerelevance
69
![Page 70: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/70.jpg)
Howisthevalidityofatestestablished?
1. Contentvalidity
2. Facevalidity
70
![Page 71: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/71.jpg)
Contentvalidity• Ifatestrequiresthetest‐takertoperformthebehaviorthatisbeingmeasured...
• itcanclaimcontent‐relatedevidenceofvalidity(contentvalidity)
• e.g.Atestofaperson’sabilitytospeakanL2requiresthestudenttoactuallyspeakwithinsomesortofauthenGccontext.
• AtestwithpaperandpencilmulGplechoicequesGonsrequiringgrammaGcaljudgmentsdoesnotachievecontentvalidity.
71
![Page 72: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/72.jpg)
AnotherwayofunderstandingcontentvalidityistoconsiderthedifferencebetweendirectandindirecttesGng.
• directtes%ng–involvesthetest‐takerinactuallyperformingthetargettask
• indirecttes%ng‐studentsnotperformingthetaskitself,butarelatedtask.
• e.g.tes%ngoralproduc%onofsyllablestress
72
![Page 73: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/73.jpg)
Toachievecontentvalidityin
classroomassessment,trytotestperformance
directly.
73
![Page 74: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/74.jpg)
74
![Page 75: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/75.jpg)
Howisthevalidityofatestestablished?
1. Contentvalidity
2. Facevalidity
75
![Page 76: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/76.jpg)
Facevalidity
• Theextenttowhichstudentsviewtheassessmentas:
1. fair2. relevant3. usefulforimprovinglearning
• Facevalidityreferstothedegreetowhichatestlooksright,andappearstomeasuretheknowledgeorabiliGesitclaimstomeasure.
76
![Page 77: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/77.jpg)
Highfacevalidity:thetest...• iswell‐constructed,expectedformatwithfamiliartasks
• isclearlydoablewithinallouedGme• hasitemsthatareclearanduncomplicated• direcGonsthatarecrystalclear• hastasksrelatedtocoursework(contentvalidity)
• hasadifficultylevelthatpresentsareasonablechallenge
77
![Page 78: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/78.jpg)
• Mostsignificantcardinalprincipleofassessmentevalua%on.
• Ifvalidityisnotestablished,allotherconsideraGonsmayberendereduseless.
78
![Page 79: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/79.jpg)
79
![Page 80: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/80.jpg)
4.AuthenGcity
80
• Arestudentsaskedtoperformreal‐worldtasks?
![Page 81: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/81.jpg)
Testtaskauthen%city
• tasksrepresent,orcloselyapproximate,real‐worldtasks
• thetaskislikelytobeenactedinthe“realworld”
• notcontrivedorarGficial81
![Page 82: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/82.jpg)
AuthenGcitychecklist• Isthelanguageinthetestasnaturalaspossible?
• Aretopicsascontextualizedaspossibleratherthanisolated?
• AretopicsandsituaGonsinteresGngenjoyable,and/orhumorous?
• IssomethemaGcorganizaGonprovided,suchasthroughastorylineorepisode?
• Dotasksrepresent,orcloselyapproximate,real‐worldtasks?
82
![Page 83: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/83.jpg)
83
![Page 84: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/84.jpg)
5.Washback
84
• Does the assessment have positive effects on learning
and teaching?
![Page 85: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/85.jpg)
Washback=theeffectoftesGngonteaching
andlearning
85
‐ posi%vewashback‐ nega%vewashback
![Page 86: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/86.jpg)
Washback• Classroomassessment:theaffectsofanassessmentonteachingandlearningpriortotheassessmentitself(preparaGon)
• Anotherformofwashback=theinformaGonthat‘washesback’tostudentsintheformofusefuldiagnosesofstrengthsandweaknesses.
• Formaltestsprovidenowashbackifstudentsreceiveasimpleleuergradeorsingleoverallnumericalscore.
86
![Page 87: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/87.jpg)
Atestthatprovidesbeneficialwashback...
• posiGvelyinfluenceswhatandhowteachersteach
• posiGvelyinfluenceswhatandhowstudentslearn
• offerslearnersachancetoadequatelyprepare• giveslearnersfeedbackthatenhancestheirlanguagedevelopment
• providescondiGonsforpeakperformancebythelearner
87
![Page 88: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/88.jpg)
Teachers’challenge
• tocreateclassroomteststhatserveaslearningtoolsthroughwhichwashbackis
achieved
88
![Page 89: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/89.jpg)
89
![Page 90: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/90.jpg)
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
![Page 91: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/91.jpg)
Q.HowdoyouknowifatestiseffecGve,appropriate,useful,or,indown‐to‐earthterms,a“good”
test?
91
![Page 92: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/92.jpg)
Answer.A‘good’test:• canbegivenwithinappropriateadministraGveconstraints,
• isdependable,• accuratelymeasureswhatyouwantittomeasure,
• thelanguageinthetestisrepresentaGveofreal‐worldlanguageuse,and
• thetestprovidesinformaGonthatisusefulforthelearner.
92
![Page 93: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/93.jpg)
• TheseprincipleswillhelpyoumakeaccuratejudgmentsabouttheEnglishcompetenceofyourstudents.
• TheyprovideusefulguidelinesforevaluaGngexisGngtests,anddesigningourown.
93
![Page 94: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/94.jpg)
AssessmentLiteracy
Know‐howandunderstandingteachersneedtoassessstudentseffec%velyand
maximizelearning
![Page 95: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/95.jpg)
• Thereisnogewngawayfromthefactthatmostofthethingsthatgowrongwithassessmentareourfault,
• theresultofpoorassessmentdesign‐andnotthefaultofourstudents.
(Raceetal.,2005)
![Page 96: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/96.jpg)
• Improvingstudentlearningimpliesimprovingtheassessmentsystem.
• Teachersoxenassumethatitistheirteachingthatdirectsstudentlearning.
• InpracGce,assessmentdirectsstudentlearning,becauseitistheassessmentsystemthatdefineswhatisworthlearning.
(Havnes,2004,p.1)
![Page 97: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/97.jpg)
• ThereissubstanGalevidencethatassessment,ratherthanteaching,hasthemajorinfluenceonstudents’learning.
• ItdirectsauenGontowhatisimportant,actsasanincenGveforstudy,andhasapowerfuleffectonstudent’sapproachestotheirwork.
(Boud&Falchikov,2007)
RethinkingAssessmentinHigherEduca0on
![Page 98: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/98.jpg)
“We owe it to ourselves and our students to devote at least as much energy to ensuring that our assessment practices are worthwhile as we do to ensuring that we teach well”.
Dr.DavidBoud,UniversityofTechnology,Sydney,Australia
98
![Page 99: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/99.jpg)
![Page 100: Testing a Test: Evaluating Our Assessment Tools](https://reader038.vdocuments.site/reader038/viewer/2022110114/547c1ad1b47959bb508b45c2/html5/thumbnails/100.jpg)
ThankyouforyourGmeandparGcipaGon