So Much Data – So Much Data – Where Do I Start?Where Do I Start?

Assessment & Accountability Assessment & Accountability Conference 2008Conference 2008

Session #18Session #18

04/20/23

Purpose of the PresentationPurpose of the Presentation

Review what MI-Access reports are Review what MI-Access reports are providedprovided

Review what information can be found Review what information can be found on the MI-Access reportson the MI-Access reports

Talk about how the results might be Talk about how the results might be usedused

04/20/23

Three MI-Access AssessmentsThree MI-Access Assessments

Participation: For students who have, or function Participation: For students who have, or function as if they have, as if they have, severesevere cognitive impairment cognitive impairment

Supported Independence: For students who Supported Independence: For students who have, or function as if they have, have, or function as if they have, moderatemoderate cognitive impairmentcognitive impairment

Functional Independence: For students who Functional Independence: For students who have, or function as if they have, have, or function as if they have, mildmild cognitive impairmentcognitive impairment

04/20/23

Selecting the Right Selecting the Right AssessmentAssessment

The IEP Team determines which assessment is The IEP Team determines which assessment is appropriate appropriate

MI-Access Web page: www.mi.gov/mi-access. MI-Access Web page: www.mi.gov/mi-access. Look under “IEP Team Information” section. Look under “IEP Team Information” section.

The data will only be helpful IF the appropriate The data will only be helpful IF the appropriate assessment is selectedassessment is selected

04/20/23

Content Areas AssessedContent Areas Assessed

All three populations are assessed in three All three populations are assessed in three content areas in the fallcontent areas in the fall

1)1) English language artsEnglish language arts2)2) MathematicsMathematics3)3) ScienceScience

ELA and mathematics are assessed in grades 3-8ELA and mathematics are assessed in grades 3-8

Science is assessed in grades 5 and 8Science is assessed in grades 5 and 8

Grade 11 MI-Access students are assessed in the Grade 11 MI-Access students are assessed in the spring (ELA, mathematics, and science)spring (ELA, mathematics, and science)

04/20/23

04/20/23

MI-Access ReportsMI-Access Reports

Report TitlesReport Titles

Online Online State State

ResultsResults

District District Results Results FolderFolder

School School Results Results FolderFolder

Classroom Classroom Results Results FolderFolder

Online Online OnlyOnly

Individual Student Individual Student ReportsReports

XX

RostersRosters XX XX XX

Summary ReportsSummary Reports XX XX XX

Demographic ReportsDemographic Reports XX XX XX

Item Analysis ReportsItem Analysis Reports XX XX XX

Parent ReportsParent Reports XX

Student LabelsStudent Labels XX

District ComprehensiveDistrict Comprehensive XX

ISD ComprehensiveISD Comprehensive XX

04/20/23

Individual Student Reports: DesignIndividual Student Reports: Design

Student demographic information Student performance

summary

Earned/points possible by

component or strand

Individual item analysis for

released items

Individual Student Report:Individual Student Report:Participation ELAParticipation ELA

04/20/23

How Participation Scores How Participation Scores are Derivedare Derived

• Two types of itemsTwo types of items• activity-based activity-based

observationobservation• selected-responseselected-response

• PAA and SAA observe PAA and SAA observe and score studentand score student

• PAA and SAA scores PAA and SAA scores are added together to are added together to determine total item determine total item pointspoints

MI-Access Participation MI-Access Participation Scoring RubricScoring Rubric

Score Point/Score Point/Condition CodeCondition Code DescriptionDescription

33Responds correctly with no assessment Responds correctly with no assessment administrator assistanceadministrator assistance

22Responds correctly after assessment Responds correctly after assessment administrator provides verbal/physical cuesadministrator provides verbal/physical cues

11Responds correctly after assessment Responds correctly after assessment administrator provides modeling, short of hand-administrator provides modeling, short of hand-over-hand assistanceover-hand assistance

AA Incorrect responseIncorrect response

BB Resists/RefusesResists/Refuses

CCAssessment administrator provides hand-over-Assessment administrator provides hand-over-hand assistance and/or step-by-step directionshand assistance and/or step-by-step directions

04/20/23

Individual Student Report:Individual Student Report:

Supported Independence MathematicsSupported Independence Mathematics

How SI Scores are DerivedHow SI Scores are Derived

• Two types of itemsTwo types of items• activity-based activity-based

observation (except observation (except science)science)

• selected-responseselected-response

• PAA and SAA observe PAA and SAA observe and score studentand score student

• PAA and SAA scores are PAA and SAA scores are added together to added together to determine total item determine total item pointspoints

MI-Access Supported Independence MI-Access Supported Independence Scoring RubricScoring Rubric

Score Point/Score Point/Condition Condition

CodeCodeDescriptionDescription

22Responds correctly with no assessment Responds correctly with no assessment administrator assistanceadministrator assistance

11Responds correctly after assessment Responds correctly after assessment administrator provides verbal/physical cuesadministrator provides verbal/physical cues

AA Incorrect responseIncorrect response

BB Resists/RefusesResists/Refuses

CCAssessment administrator provides hand-over-Assessment administrator provides hand-over-hand assistance and/or step-by-step directionshand assistance and/or step-by-step directions

04/20/23

Individual Student Report:Individual Student Report:Participation ScienceParticipation Science

04/20/23

Individual Student Report:Individual Student Report:Functional Independence ScienceFunctional Independence Science

How FI Scores are DerivedHow FI Scores are Derived

No scoring rubric for Functional IndependenceNo scoring rubric for Functional Independence

Only selected-response itemsOnly selected-response items

Student receives 1 point for each correct response Student receives 1 point for each correct response

The ONLY exception is ELA Expressing IdeasThe ONLY exception is ELA Expressing Ideas Open-ended response to a promptOpen-ended response to a prompt Scored using a 4-point rubricScored using a 4-point rubric Student receives may receive up to 4 points per promptStudent receives may receive up to 4 points per prompt

04/20/23

Other Differences:Other Differences:Condition CodesCondition Codes

P/SI Condition Codes: All Content AreasP/SI Condition Codes: All Content Areas− A = responds incorrectlyA = responds incorrectly− B = resists/refuses B = resists/refuses − C = responds only after the assessment administrator provides hand-C = responds only after the assessment administrator provides hand-

over-hand assistance or step-by-step directionsover-hand assistance or step-by-step directions

FI Condition Codes: ELA Expressing Ideas OnlyFI Condition Codes: ELA Expressing Ideas Only− A = off topic A = off topic − B = illegible B = illegible − C = written in a language other than EnglishC = written in a language other than English− D = blank/refused to respond D = blank/refused to respond

All condition codes = zero pointsAll condition codes = zero points

Other Differences:Other Differences:Performance Level ChangePerformance Level Change

STUDENT PERFORMANCE SUMMARYSTUDENT PERFORMANCE SUMMARYEarned Points Possible:Earned Points Possible: 0/450/45

Scale Score:Scale Score: 23912391

Performance Level 2007:Performance Level 2007: Emerging Toward the Performance Standard Emerging Toward the Performance Standard (High)(High)

Performance Level 2006:Performance Level 2006: Emerging Toward the Performance Standard Emerging Toward the Performance Standard (Low)(Low)

Performance Level Change:Performance Level Change: ImprovementImprovement

ONLY for Functional Independence ELA and Mathematics

Performance Level ChangePerformance Level Change

Fall 2006 Fall 2006 AchievementAchievement

Fall 2007 AchievementFall 2007 Achievement

EmergingEmerging AttainedAttained SurpassedSurpassed

LowLow MidMid HighHigh LowLow HighHigh Low Low MidMid HighHigh

EmergingEmerging

LowLow NN II II SISI SISI SISI SISI SISI

MidMid DD NN II II SISI SISI SISI SISI

HighHigh DD DD NN II II SISI SISI SISI

AttainedAttainedLowLow SDSD DD DD NN II II SISI SISI

HighHigh SDSD SDSD DD DD NN II II SISI

SurpassedSurpassed

LowLow SDSD SDSD SDSD DD DD NN II II

MidMid SDSD SDSD SDSD SDSD DD DD NN II

HighHigh SDSD SDSD SDSD SDSD SDSD DD DD NN

SI = Significant Improvement; I = Improvement; N = No Change; D = Decline; SD = Significant Decline

Using ISRs for Using ISRs for Instruction and CurriculumInstruction and Curriculum

Compare ISR to other data you have for the student. Is this what you would Compare ISR to other data you have for the student. Is this what you would expect? expect?

If prior year and performance level change data are provided, is the If prior year and performance level change data are provided, is the student making progress over time?student making progress over time?

Using released item booklets (www.mi.gov/mi-access), match to curriculum Using released item booklets (www.mi.gov/mi-access), match to curriculum and instruction. Has the student been taught this? If yes, are teaching and instruction. Has the student been taught this? If yes, are teaching methods effective for this student?methods effective for this student?

Identify strengths to reinforceIdentify strengths to reinforce

Identify areas where additional instruction is neededIdentify areas where additional instruction is needed

Confirm this is the right assessment for the student to takeConfirm this is the right assessment for the student to take

04/20/23

04/20/23

RostersRosters

List scores by individual studentList scores by individual student

One report for each grade and content areaOne report for each grade and content area

Provided at class, school, and district levels Provided at class, school, and district levels (not state)(not state)

04/20/23

RostersRostersShow number assessed and mean scale score or mean earned points

at top left

List results by student

04/20/23

RostersRosters

For FI ELA and mathematics, shows current year scale

score, two years of performance level (high, mid,

and low), and performance level change

04/20/23

RostersRosters

Shows earned points by component or strand, EGLCE or EB, and overall

Using Rosters for Using Rosters for Instruction and CurriculumInstruction and Curriculum Best place to gather information about assessment items, because Best place to gather information about assessment items, because

results are provided by EGLCE or EBresults are provided by EGLCE or EB

www.mi.gov/mi-accesswww.mi.gov/mi-access

Supported Independence Grade 5Supported Independence Grade 5ELA: ComprehensionELA: Comprehension Math: GeometryMath: Geometry Science: ReflectingScience: Reflecting

R.NT.e4.SI.EGO4aR.NT.e4.SI.EGO4a

Identify what makes Identify what makes stories fiction vs. fact and stories fiction vs. fact and why an author makes that why an author makes that

choicechoice

G.LO.e1.SI.EGO2AAG.LO.e1.SI.EGO2AA

Identify positions of objects in Identify positions of objects in space using terms describing space using terms describing relative position (first, last, relative position (first, last, above, below, next to, etc.)above, below, next to, etc.)

R.RO.SI.EB.II.1.e.4aR.RO.SI.EB.II.1.e.4a

Develop an awareness of natural world. Develop an awareness of natural world.

Key concepts: Nature, observation, Key concepts: Nature, observation, personal safety.personal safety.

Real-world contexts: Caring for the Real-world contexts: Caring for the environment; pollution; recycling; environment; pollution; recycling;

water safetywater safety

04/20/23

Summary ReportsSummary Reports

Executive summaries of student scores Executive summaries of student scores

Provided at school, district, and state levelsProvided at school, district, and state levels

One report for each grade and each content area One report for each grade and each content area

assessed (ELA, mathematics, and/or science)assessed (ELA, mathematics, and/or science)

Report generated ONLY when there are 10 or Report generated ONLY when there are 10 or

more students at the same grade level taking more students at the same grade level taking

the same assessmentthe same assessment

04/20/23

Supported Independence District Supported Independence District Summary Report: ELASummary Report: ELA

Summary results shown for current year at top right

Supported Independence District Supported Independence District Summary Report: ELASummary Report: ELA

04/20/23

Shows # and % of students that earned

each earned points total

Functional Independence Functional Independence Summary Reports: ELA and Mathematics Summary Reports: ELA and Mathematics Grades 4, 5, 6, 7, and 8 Grades 4, 5, 6, 7, and 8 ONLYONLY

Number and Percent of Students by Performance Level Change Number and Percent of Students by Performance Level Change (from 2006 to 2007)(from 2006 to 2007)

Fall 2006Fall 2006Fall 2007Fall 2007

EmergingEmerging AttainedAttained SurpassedSurpassed

EmergingEmerging 104 (30.2%)104 (30.2%)

not gainingnot gaining

8 (2.3%)8 (2.3%)

gaininggaining

8 (2.3%)8 (2.3%)

gaininggaining

AttainedAttained 32 (9.3%)32 (9.3%)

decliningdeclining

24 (7.0%)24 (7.0%)

maintainingmaintaining

8 (2.3%)8 (2.3%)

gaininggaining

SurpassedSurpassed 48 (14.0%)48 (14.0%)

decliningdeclining

8 (2.3%)8 (2.3%)

decliningdeclining

104 (30.2%)104 (30.2%)

maintainingmaintaining

04/20/23

PERFORMACE LEVEL CHANGE — YEAR-TO-YEAR TRANSITIONS

04/20/23

Functional Independence Summary Functional Independence Summary Reports: Performance Level ChangeReports: Performance Level Change

““Gaining” means there was an improvement from 2006 to Gaining” means there was an improvement from 2006 to 20072007

““Maintaining” means scores that were “proficient” stayed the Maintaining” means scores that were “proficient” stayed the samesame

““Not gaining” means scores that were “not proficient” stayed Not gaining” means scores that were “not proficient” stayed the samethe same

““Declining” means there was a decline from 2006 to 2007Declining” means there was a decline from 2006 to 2007

Functional Independence Functional Independence Summary Reports: ELA and Mathematics Summary Reports: ELA and Mathematics Grades 4, 5, 6, 7, and 8 Grades 4, 5, 6, 7, and 8 ONLYONLY

Years Years SummarizedSummarized

Students Students MatchedMatched# %# %

Performance Level ChangePerformance Level Change

Significant Significant DeclineDecline

# %# %

DeclineDecline

# %# %

No ChangeNo Change

# %# %

ImprovementImprovement

# %# %

Significant Significant ImprovementImprovement

# %# %

2006 & 20072006 & 2007 344 93.5344 93.5 88 25.688 25.6 0 0.00 0.0 232 67.4232 67.4 0 0.00 0.0 24 7.024 7.0

04/20/23

PERFORMANCE LEVEL CHANGE — SUMMARY

Using Summary Reports for Using Summary Reports for Instruction and CurriculumInstruction and Curriculum

Look at progress collectively. How did our fifth graders do this Look at progress collectively. How did our fifth graders do this year compared to last year? year compared to last year?

Look at progress over time. Are there trends to be aware of?Look at progress over time. Are there trends to be aware of?

Compare state assessment data to other data. Is this what we Compare state assessment data to other data. Is this what we would expect?would expect?

Within the performance levels, where are clusters of students?Within the performance levels, where are clusters of students?

04/20/23

Functional Independence Functional Independence Summary Report: ScienceSummary Report: Science

04/20/23

P/SI Parent ReportsP/SI Parent Reports

Page 1 shows student’s earned

points and performance level for current year.

Page 2 shows appropriate scoring rubric, and ELA and mathematics scores

by component or strand

P/SI Parent ReportsP/SI Parent Reports

ELA

Mathematics

Page 3 shows science scores by strand and ELA

individual student item analysis

P/SI Parent ReportsP/SI Parent Reports

Science

ELA

Page 4 shows mathematics and science individual

student item analysis

P/SI Parent ReportsP/SI Parent Reports

Mathematics

Science

04/20/23

Functional IndependenceFunctional Independence Parent Reports (Grades 4, 5, 6, 7, & 8) Parent Reports (Grades 4, 5, 6, 7, & 8)

Page 1 shows the student’s scale

score and performance level for 2007 and 2006.

Also shows “Performance

Level Change” for current and past

year.

04/20/23

Page 2 shows student’s earned points for ELA and mathematics, and his/her

scale scores in a range

ELA

Mathematics

Functional IndependenceFunctional Independence Parent Reports Parent Reports

04/20/23

Page 3 shows student’s earned points and scale score range for science. Also

shows the student’s item analysis for

ELA

Science

ELA

Functional IndependenceFunctional Independence Parent Reports Parent Reports

04/20/23

Page 4 shows student’s

individual item analysis

Science

Mathematics

Functional IndependenceFunctional Independence Parent Reports Parent Reports

04/20/23

Item AnalysesItem Analyses

Provide detailed, aggregated information on Provide detailed, aggregated information on released itemsreleased items

Can be used to identify areas of collective strengths Can be used to identify areas of collective strengths and areas that need improvementand areas that need improvement

Provided at school, district, and state levelProvided at school, district, and state level

Provided only when 10 or more students in the same Provided only when 10 or more students in the same grade take the same assessmentgrade take the same assessment

04/20/23

District Item Analysis: ELADistrict Item Analysis: ELA

Shows # and % of students who selected each answer choice for each released Word Recognition and Text Comprehension item

04/20/23

District Item Analysis: ELADistrict Item Analysis: ELA

Shows the # and % of students at (1) each score based on a 4-point rubric, and (2) each condition code

04/20/23

District Item Analysis: ELADistrict Item Analysis: ELA

Also shows the # and % of students that received specific comment codes

04/20/23

Using Item AnalysesUsing Item Analyses

Use the Item Analysis Report Use the Item Analysis Report along with the Released Item along with the Released Item Booklet to identify collective Booklet to identify collective strengths and weaknesses strengths and weaknesses (available at www.mi.gov/mi-(available at www.mi.gov/mi-access) access)

04/20/23

Using Item Analyses: Using Item Analyses: ELA State Fall 2006ELA State Fall 2006

Functional Independence Grade Functional Independence Grade 33

Informational passage about a Informational passage about a chameleonchameleon

EGLCE being measured: Make EGLCE being measured: Make inferences, predictions, and inferences, predictions, and conclusionsconclusions

Only 62.1% answered the item Only 62.1% answered the item correctly (A)correctly (A)

C was the incorrect answer C was the incorrect answer chosen most often (23%)chosen most often (23%)

04/20/23

Using Item Analyses: Using Item Analyses: ELA State Fall 2006ELA State Fall 2006

Same narrative passageSame narrative passage

EGLCE being measured: EGLCE being measured: Identify main ideas and Identify main ideas and details details

74.7% of students 74.7% of students answered the item correctly answered the item correctly (B)(B)

A was the incorrect answer A was the incorrect answer chosen most often (13%)chosen most often (13%)

04/20/23

Using Item Analyses: Using Item Analyses: ELA State Fall 2006 Expressing IdeasELA State Fall 2006 Expressing Ideas

EGLCE being measured: EGLCE being measured: Write/draw personal narrativeWrite/draw personal narrative

Only 7% of students received Only 7% of students received a “4,” and 23% received a “3”a “4,” and 23% received a “3”

Comment code given most Comment code given most often = “Showed limited often = “Showed limited development with insufficient development with insufficient details and/or examples”details and/or examples”

04/20/23

Using Item Analyses: Using Item Analyses: Math State Fall 2006Math State Fall 2006

Functional Independence Functional Independence Grade 7 Grade 7

EGLCE being measured: EGLCE being measured: Solve problems using dataSolve problems using data

Only 33.6% answered Only 33.6% answered correctly (A)correctly (A)

C was the incorrect answer C was the incorrect answer chosen most often (54.6%)chosen most often (54.6%)

04/20/23

Using Item Analyses: Using Item Analyses: Mathematics State 2006Mathematics State 2006

EGLCE being measured: EGLCE being measured: Recognize representations Recognize representations for whole numbers to for whole numbers to 10,00010,000

96.5% answered correctly 96.5% answered correctly (C)(C)

B was the incorrect answer B was the incorrect answer chosen most often (2.2%)chosen most often (2.2%)

04/20/23

Using Assessment ResultsUsing Assessment Results

““Results have the greatest impact when people are Results have the greatest impact when people are given opportunities to discuss them—teachers with given opportunities to discuss them—teachers with teachers, coordinators with teachers, coordinators teachers, coordinators with teachers, coordinators with coordinators, and teachers with parents. You with coordinators, and teachers with parents. You discover trends, identify anomalies, start asking discover trends, identify anomalies, start asking questions, and together seek answers.”questions, and together seek answers.”

Contact InformationContact Information

Peggy Dutcher, CoordinatorPeggy Dutcher, Coordinator

Professional DevelopmentProfessional Development

OEAAOEAA

Email: dutcherp@mi.govEmail: dutcherp@mi.gov

Phone: 517-335-0471Phone: 517-335-0471