english language proficiency indicator - isbe.net · progress in achieving english language...
TRANSCRIPT
EnglishLanguageProficiencyIndicator
ILAccountabilityTAC,April2,2018
ESSARequirements
Annuallymeasure….forallpublicschoolsintheState,progressinachievingEnglishlanguageproficiency,asdefinedbytheStateandmeasuredbythe[state’sELPassessment]withinaState-determinedtimelineforallEnglishlearners—• ineachofthegrades3through8;and• inthegradeforwhichsuchEnglishlearnersareotherwiseassessed…duringthegrade9throughgrade12period
ILTACApril2018ELP 2
PrimaryConsiderations
• UnderESSA,theELPindicatormaytakeindividualstudentcharacteristicsintoaccount.
• Twoimportantcharacteristicsthatinfluencewhetherastudentisreclassified:– Student’sinitiallevelofEnglishproficiency,– TimeinthesystemasanEL
• Growthconsiderations– Growthtrajectoriesarerarelylinear
• Fastergrowthearly;slowsdownovertime• Growthisoftenconsistentacrossgradesamonginitiallevels• StudentsenteringinthelatergradestendtoscoresimilarlyanddisplaysimilargrowthtrajectoriestostudentswiththesameELDlevelwhohavebeenintheschoolsystemlonger
ILTACApril2018ELP
Goldschmidt,P.&Hakuta,K.(2017).IncorporatingEnglishLearnerProgressintoStateAccountabilitySystems.WashingtonDC:CouncilofChiefStateSchoolOfficers
3
ILStatePlan
• IllinoisproposesatargetedmaximumtimelineoffiveyearsforEnglishLearnerstoachieveELP
• Proficiencyhasbeenestablishedasacompositeproficiencylevelscoreof4.8orabove
• ELsmustmakeannualprogresstowardsthecompositescoreof4.8oraboveonACCESS2.0withinfiveyears.Studentsmeasuretowardproficiencyisindividuallybasedonentrylevelperformance.Astudentismakingprogressprovidedthattheyscoreatorabovetheircalculatedinterimtarget
ILTACApril2018ELP 4
ILStatePlan
ILTACApril2018ELP
• Theinterimtargetiscalculatedbyinterpolatingbetweenthestudent’sentrylevelACCESS2.0scoreandtheminimumexitscoreof4.8.
• Allstudentsareprovided5yearstoexitregardlessoftheircompositeproficiencylevelinyear1.
5
ThoughtsfromtheDecembermeeting
• Needadditionalinformationtohelpevaluatethereasonablenessofthismodelandproposeadditionaloptionsforconsideration:– TechnicalcharacteristicsoftheACCESS2.0– SummaryofwhatotherWIDAstatesaredoing/planning– CharacteristicsofthepopulationofELLtesttakersinIL– PotentialimpactofproposedELPindicatorfordifferenttypesofschoolsinIL
ILTACApril2018ELP 6
WIDAStates
ILTACApril2018ELP 7
Graphicfromwww.wida.us
ACCESSforELLs2.0
• Purpose:meetstateandfederalrequirementsfortheannualassessmentofEnglishlearnersto…– Identifytheappropriatelanguageinstructioneducationalprogram(LIEP)placement
– DetermineifstudentsaremakingacceptableprogressinEnglishlanguageproficiency
– DetermineifstudentshaveattainedsufficientEnglishlanguageproficiencytobereclassified
– SupporttheevaluationofLIEPprograms
8ILTACApril2018ELP
TakenfromWIDAInterpretiveGuide
Listening
Speaking
Reading
Writing
Process, understand, interpret and evaluate spoken language in a variety of situations
Process, understand, interpret and evaluate written language, symbols and text with understanding and fluency
Engage in written communication in a variety of situations for a variety of purposes and audiences
Engage in oral communication in a variety of situations for a variety of purposes and audiences
Language Domains
SlidedevelopedbyWIDAConsortium
AssessedGrades
ILTACApril2018ELP
Online-StagedAdaptiveDesign
10
SlidedevelopedbyWIDAConsortium
OnlinevsPaper
ILTACApril2018ELP
FromWIDAInterpretiveGuide
11
DomainScores
• ScaleScores:Withineachdomainscoresarereportedonaverticalscalethatrangesfrom100-600.– appropriateformakingcomparisonsacrossgradeswithinadomain– confidenceintervalsbasedonCSEMs.
• Proficiencylevelscores–gradespecificinterpretationsofstudentperformanceinacontentdomainbaseduponthePLsintheELDStandards.– Proficiencylevelscoresarepresentedaswholenumbersfollowedbya
decimal.Thewholenumberindicatesthestudent’slanguageproficiencylevel(from1-6)basedontheWIDAELDStandards.Thedecimalindicatestheproportionwithintheproficiencylevelrangethatthestudent’sscalescorerepresents,roundedtothenearesttenth.(SeeAppendixDintheInterpretiveGuide)
– Thesamescaledscoreisassociatedwithadifferentproficiencylevelacrossgrades.“AReadingscalescoreof355forafifthgradestudentisinterpretedasLevel4.0.ThesamescalescoreforafourthgraderresultsinLevel4.6,andforathirdgradestudentthatscalescoreresultsinLevel5.2.”
– Appropriateforcomparisonsacrossdomainswithinagrade.ILTACApril2018ELP 12
ProficiencyLevels
• Level1:Entering• Level2:Beginning• Level3:Developing• Level4:Expanding• Level5:Bridging• Level6:ReachingLevelDefinitionsestablishedinconsiderationof:
– LinguisticComplexity:Extentoffunctionallanguage(textordiscourse)– VocabularyUsage:Comprehensionanduseofthetechnicalvocabularyofthecontentareas
– LanguageControl:Comprehensionanduseofphonological,syntactic,andsemanticstructure&rules
ILTACApril2018ELP 13
FromWIDAInterpretiveGuide
CompositeScoreCategories
ILTACApril2018ELP14
SlidedevelopedbyWIDAConsortium
CompositeScores&ProficiencyLevels
• CompositeScaleScores:Weightedsumacrossdomainscaledscores(compensatory)– ConfidenceintervalsbasedonclassicaltesttheorySEMs
• CompositeProficiencyLevelScores:derivedfromthedomainscaledscores,notthedomainproficiencylevels(i.e.,baseduponthecompositescaledscore).
• CautionsfromWIDA:– TheOverallScoreishelpfulasasummaryofotherscoresandbecause
sometimesyoumayneedasinglenumberforreference.However,it’simportanttoalwaysrememberthatitiscompensatory;aparticularlyhighscoreinonedomainmayeffectivelyraisealowscoreinanother.Similaroverallscorescanmaskverydifferentperformancesonthetest.
– Nosinglescoreorlanguageproficiencylevel,includingtheOverallScore(Composite),shouldbeusedasthesoledeterminerformakingdecisionsregardingastudent’sEnglishlanguageproficiency.
ILTACApril2018ELP 15
FactorstoConsiderinDefiningELPIndicator(Reviewed12approvedplansforWIDAstates)
• HowisEnglishLanguageProficiencyDefinedontheACCESS2.0?– 5.0PL-9states;– 4.6-1state;4.5–1state;4.8–IL
• Whatisthemaximumnumberofyearsastudenthastoachieveproficiency?
– 5year:4states– 6years:8states
• Whatfactorsinfluencethenumberofyearsastudenthastoreachproficiency?
– Baselineproficiencylevel(11states)– Baselinegradelevel(3)
• Howisannualprogressevaluated?– ComparegaininPLscoretothatdeemednecessarytoachieveproficiencyina“fixed”
numberofyears:8states– EvaluatewhetherobservedSGPisontrackwiththatneededtobeproficientinadefined
numberofyears(adequategrowthpercentile):3states– ComparegaininPLscoreagainstacommonfixedcriterionforallstudents-1state(HI)
requires1.0PLgainperyear
ILTACApril2018ELP 16
FactorstoConsiderinDefiningELIndicator
• Aretargetsforperformancereseteachyear?– Yes:redistributethegapnecessarytoachieveproficiencyinthedefinedamountof
timeeachyear(3states)– Yes:throughannualrecalculationofAGPs(3states)– No:mustachievetargetsestablishedinbaselineyear1tomakeadequateprogress
(6states)• HowisschoolperformancequantifiedwithintheELPindicator?
– Percentageofstudentsachieving/exceedingtheannualtargetstowardproficiencyintheschool(i.e.,scoreorAGP):8states
• MAalsoconsidersthepercentageofstudentsthatachievedproficiencyontheACCESS2.0
– Assignpointstoeachstudentbasedondegreetowhichprogresstargetismetusingavaluetableapproach:4states
• Establishaweightedrateoftargetattainment:2states(DE,PA)• Createanindexscorefortheschoolbasedonsumofpointsearnedbyeachstudent:1state(SD)
– ConsideraggregateresidualdifferencebetweenobservedPLscoreandtargetPLscoreoverstudentsandassignagrade(A-F)tothisresidualforreporting:(NM)
ILTACApril2018ELP 17
WYELPProgress
ILTACApril2018ELP 18
Delaware/PennsylvaniaApproach• GrowthisbasedontheverticalscaleofACCESS2.0• Studentshavebetween3-6yearstoexit(composite5.0or
higher)dependingoninitialaccesslevel• Thestateproducesanindexthatmeasures“percentof
annualtargetachieved”andallowsabonusofupto10%ifthetargetisexceeded– E.g.ifthetargetgrowthis20pointsandthestudentearns15pointsthe
studentreceives.75
• Studentswhodonotexitinthedesignatedattainmentyearareeligibleforpartialpointswhentheyexit– .75,.5,or.25foreachof1,2,or3+yearsof“lateexit”
• Totalschoolindexscoreistheaverageofeachstudent’sscore(essentiallyaweightedattainmentindex)
ILTACApril2018ELP 19
Delaware
ILTACApril2018ELP 20
DEGrowthIndex
ILTACApril2018ELP 21
SDGrowthIndex
ILTACApril2018ELP
Astudentisexpectedtomake20%progresstowardproficiencyeachyear.
22
Michigan–AdequateGrowthPercentiles
• Eachstudent’sattainmentgrowthtargetisthescalescore(SS)ataPL4.5atthegradelevelfortheyearthattheyareexpectedtoreachattainment
• Eachstudent’sinterimgrowthtargetsarecalculatedannuallyusingtheadequategrowthpercentile(AGP)AnAGPisaquantitativedescriptionofthegrowthnecessarytobeconsistentlyachievedtoreachproficiencyinasetnumberofyears
• Theannualresetallowsforavariablegrowthtrajectorydependingoneachstudent’sprogressovertimewhilestillrequiringthattheATbereachedwithintherequirednumberofyears.Thisyearlyresetrecognizesthenonlineargrowththatstudentsatvaryingproficiencylevelsmakewithinayear’stime.
ILTACApril2018ELP 23
NewMexicoELPIndicator• AllELstudent’sELPscoresare
comparedtotheirpersonalizedannualtargets.TheresidualsareaccumulatedforallELPstudentsintheschool.Apositivevalueindicatesthatstudentsare,onaverage,exceedingtargetsandanegativevalueindicatesthattheyarenotmeetingtargets.
• Schoolsearnagradeontheindicatorbasedontheresidualscoreobtained.(Cut-scoreshavenotyetbeendetermined).
ILTACApril2018ELP 24
DataAnalyses
ILTACApril2018ELP 25
DescriptiveSummary
• Approximately188KELstudentsinIL• Overall,about80%ofexamineesarelevel3andbelow• About22KELinHSgrades
– 11%Level1– 23%Level2– 47%Level3– 17%Level4– About2%Level5+
• About17KStudentsserved3+yearsinHS– 7%Level1– 23%Level2– 51%Level3– 17%Level4– About1%Level5+
ILTACApril2018ELP 26
GrowthSummary
• Growthishighestforlowerperformingstudentsinearliergrades
• Studentsingrades9-12demonstratethelowestgrowth(oftennegative)regardlessofstartinglevelandregardlessofwhethertheyhavebeenreceivingservicesforlessthan3yearor3+years
• Forstudentsinthelowergradesbelowlevel5averagegrowthisoftensufficienttoreachlevel5in3orfeweryears;thisisnotthecaseforstudentsinmiddleandespeciallyhighschoolgrades
ILTACApril2018ELP 27
Discussion
• WhatguidancedoestheTAChavetoimplementafairandtechnicallydefensibleapproachformeasuringEnglishlanguageprogress?
• Howcanweensurethemethodbalancestheimportanceofprogresstoexitinareasonabletimeframeandsettingexpectationsthatareobtainable?
• Whatadditionalinformation/analysesarehighestprioritytoinformthesedecisions?
ILTACApril2018ELP 28