online statewide assessment: coming soon to a school near...

51
Online Statewide Assessment: Coming Soon to a School Near You! David Judd Director; Office of Systems, Psychometrics & Measurement Research Bureau of Assessment and Accountability April 30, 2013 MERA Conference

Upload: others

Post on 15-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Online Statewide Assessment: Coming Soon to a School Near You!

David Judd Director; Office of Systems, Psychometrics & Measurement Research

Bureau of Assessment and Accountability

April 30, 2013

MERA Conference

Page 2: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Today’s Topics

• Student Voice

• Kindergarten Entry Assessment

• Statewide assessment overview

• Next-generation assessment accessibility

• Statewide systems supporting online assessment & reporting

• Technology readiness

• Fall 2013 Online Testing Opportunities2

Page 3: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

MEAP Social Studies Online PilotStudent Comments

“I enjoyed taking the online test a lot more”

“It was very fun”

“Make the math one too!”

“I would like to use computers for the MEAP again”

“I LOVED TAKING IT ONLINE !!!!!!!!!”

“I feel like it was better to take it online”

“I wish we took all MEAP tests online, it was faster”

“The online MEAP test was a good experience!!!!!”

“Can we take all the MEAP tests online?”

Page 4: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

“The computer tend to make me tired and sleepy”

“I did not like how I could see other computers and I would see how far (the other students) were.”

“Make the words bigger”

“It could use some more color”

“You should tell the kids their score at the end”

“Fix the highlighter tool!”

MEAP Social Studies Online PilotStudent Comments

Page 5: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

MEAP Social Studies Online Pilot Online or Paper?

Page 6: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

MEAP Social Studies Online Pilot What distractions did students face?

Page 7: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

K-2 Cognitive LabsParticipation

• 18 Second Grade Teachers

• 29 First Grade Teachers

• 22 Kindergarten Teachers

• Feedback focused on…– Overall sentiment

– Technology-related issues

– Test content and format

– Implementation issues

Page 8: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

K-2 Cognitive LabsFeedback

• Teachers need to… – prepare students with computer skills– determine logistics of administering an online assessment

to a large group of students

• Some elements were difficult for students…– Scrolling– Drop-down boxes

• System needs to…– highlight words as they are being read– allow student to turn audio off

Page 9: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

K-2 Cognitive LabsFeedback

• Higher student engagement

• Acknowledges different learning styles

• Human voice audio was very helpful

• Reduction in test anxiety

• Necessary life skills

Page 10: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Kindergarten Entry Assessment (KEA)

Legislation provided $3.25 million to implement a KEA

Requires PD for educators Must include a system for data entry and

integration with P-20 LDS Spotlights conducted fall 2012 to understand

state of the art Request for Proposals released March 2013 Significant pilot testing expected fall 2013

10

Page 11: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

The Assessment Challenge

How do we get from here... ...to here?

All students

leave high school

career and

college ready

Common Core

State Standards

specify K-12

expectations for

career and

college readiness

...and what can an

assessment system

do to help?

11

Page 12: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Transitions

– MEAP SBAC (Smarter Balanced Assessment Consortium)

– MME SBAC

– MEAP-Access (for students with disabilities for whom both MEAP/MME

and MI-Access are inappropriate) SBAC

– MI-Access (for students with significant cognitive disabilities) DLM (Dynamic Learning Maps)• Functional Independence (for students with mild cognitive disabilities)

• Supported Independence (for students with moderate cognitive disabilities)

• Participation (for students with severe cognitive disabilities)

– Michigan English Language Proficiency Assessment (ELPA) WIDA/ASSETS (World Class Instructional Design and Assessment)12

Page 13: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Additions

• Interim Assessments

– Optional & Online only

– Primarily grades and content areas not covered by a consortia

• Science

• Social Studies

• MMC courses

• K-2 ELA & Mathematics

• Model assessment (e.g., Arts)

• Kindergarten Entry Assessment13

Page 14: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

● 25 states representing 40% of K-12 students in U.S.

● 21 governing, 4 advisory states

● Washington state is fiscal agent

● WestEd provides project management services

SBAC - A Consortium of States

14

Page 15: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

A Balanced Assessment System

Common

Core State

Standards

specify

K-12

expectations

for career

and college

readiness

All students

leave

high school

career and

college

ready

Teachers and

schools have

information and

tools they need

to improve

teaching and

learningInterim assessments

Flexible, open, used for actionable

feedback

Summative assessments

Benchmarked to career and college

readiness

Teacher resources for formative

assessment practices

to improve instruction

15

Page 16: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

A Balanced Assessment System

Two types of Interim Assessments- Summative Clone (mimics accountability test blueprint)- Interim Cluster (deeper dive into assessment targets)

Page 17: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Smarter Balanced Sample Items

● October 9th , 2012 release of online tool holding 50+ items

−Mathematics

−ELA

−Samples from all 7 grades (3-8 and 11)

● Item type conversations

● New level of rigor expected by the Common Core State Standards

17

Page 18: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Smarter Balanced Sample Items

18

Page 19: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Smarter Balanced Sample Items

19

Page 20: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

20

Page 21: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

21

Page 22: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Performance Tasks● New item type for Michigan

● Measure concepts other item types do not cover very well (e.g., research)

● Up to an estimated 125 minutes needed

● Performance Tasks may include a Classroom Activity

● Six sample Performance Tasks

● Teacher Preparation & Resources

● Specifications

● Scoring Rubrics

22

Page 23: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Smarter Balanced Pilot Test – Spring 2013

• Scientific Sample– Over 700 Michigan schools agreed to participate

– Student survey responses being collected

• Volunteer Schools (Practice Tests)– Grades 3-8 and 11 in ELA and Mathematics

– Similar to planned assessment experience

– Available May 29, 2013 and through the 2013-14 school year

• Practice Test Accommodations– Text-to-speech

– Item-level pop-up Spanish glossaries for construct irrelevant terms (math tests only)

– Braille

– American Sign Language (ASL)23

Page 24: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Smarter Balanced Pilot Test – Spring 2013

• Lessons Learned

– Item development

– Systems interoperability

– Scoring needs

– Translation methodology

24

Page 25: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Smarter Balanced Next Steps

• Item development for spring 2014 field test

• Hire a scoring service provider and build Artificial Intelligence (AI) scoring engine

• Continue developing:

– Computer Adaptive Testing (CAT) specifications

– Reporting system

– Digital library

– Interim system

25

Page 26: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Find Out More

Smarter Balanced can be found online at:

SmarterBalanced.org

26

Page 27: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

DLM – Dynamic Learning Maps

www.dynamiclearningmaps.org

27

Page 28: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

DLM Timeline

28

Page 29: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

The Common Core Essential Elements are specific statements of the content and skills that are

linked to the Common Core State Standards grade level specific expectations for students with

significant cognitive disabilities.

Page 30: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Common Core Essential Elements: Design Priorities

• Define essential differences from grade to grade in

– cognitive demand

– content knowledge

– skills-based expectations

• Identify the key elements essential for each grade level.

– Not necessarily a one-to-one relationship with Common Core State Standards

• Align Essential Elements across and between grades

Page 31: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

CCSS Grade-Level

Standards

Common Core

Essential ElementsRange of Complexity

Represent and solve problems involving multiplication and division.

3.OA.1. Interpret products of whole numbers, e.g., interpret 5 × 7 as the total number of objects in 5 groups of 7 objects each. For example, describe a context in which a total number of objects can be expressed as 5 × 7.

3.OA.2. Interpret whole-number quotients of whole numbers, e.g., interpret 56 ÷ 8 as the number of objects in each share when 56 objects are partitioned equally into 8 shares, or as a number of shares when 56 objects are partitioned into

EE3.OA.1-2. Use

repeated addition

and equal groups to

find the total

number of objects

to find the sum.

Students will:EE3.OA.1-2. Use repeated addition to find the total number of objects arranged in a square or rectangular array.Ex. Using tiles in a template, identify the total number of tiles by adding the tiles in the template.Ex. Fill space of squares and rectangles with 1-inch tiles, add tiles in rows or columns to determine total number of tiles it takes to fill the shape (square/rectangle).Ex. Use an abacus to find the total.Students will:

EE3.OA.1-2. Use repeated addition and equal groups to find the total number of objects to find the sum.Ex. Two birds + two Birds + two birds = six birds.Ex. Given a repeated addition number sentence, use a number line to find the sum.

Third Grade Mathematics Standards: Standards: Operations and Algebraic Thinking

Page 32: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

CCSS Grade-Level

Standards

Common Core

Essential ElementsRange of Complexity

Phonics and Word

Recognition.

RF.5.3. Know and apply

grade-level phonics and

word analysis skills in

decoding words.

a. Use combined

knowledge of all

letter-sound

correspondences,

syllabication

patterns, and

morphology (e.g.,

roots and affixes)

to read accurately

unfamiliar

multisyllabic

words in context

and out of

context.

EERF.5.3. Know and

apply phonics and

word analysis skills

in decoding words.

a. Decode two-

syllable words.

Students will:

EERF.5.3.a. Decode up to multiple-syllable words.

Ex. Read text comprised of frequently encountered two- and three-syllable words.

Students will:

EERF.5.3.a. Decode two-syllable words.

Ex. Read text comprised of single-syllable words with accuracy.

Ex. Use decoding skills to match familiar words with picture-based representations

of the words.

Students will:

EERF.5.3.a. Identify the beginning and ending consonant sounds of familiar words.

Ex. Asked what the first and last sound in mat is, respond with /m/ and /t/ sound.

Ex. Represents the initial and final sound in efforts to spell familiar words.

Ex. Given a word card, the student will indicate (point to) the letter that is at the

beginning of the word and the letter at the end of the word.

Students will:

EERF.5.3.a. Identify initial letter in own name.

Ex. Selects first letter of own name from an array of letters.

Ex. State first letter of own name when asked.

Ex. Given three student names, can pick out own name based on the first letter.

Fifth Grade English Language Arts Standards: Reading (Foundational Skills)

Page 33: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Technology Features for DLM

• Touch screen technology• Audio via sound files• American Sign Language video• Pop-up context-dependent

dictionaries/glossaries• Text and image magnification• On-screen note taking• Color overlays• IntelliKeys™ keyboard accessibility• Masking• Text readers• Refreshable braille displays

Page 34: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

As a result you can expect some changes in the future…

• Accommodations and selection process

• Development of Personal Needs Profiles

• Move towards consortium developed Participation Guidelines

• Accommodations Monitoring

Page 35: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

ASSETS – Assessment Services Supporting ELs through

Technology Systems (ASSETS)

35http://assets.wceruw.org/

Page 36: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

ASSETS Timeline

36

Page 37: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Alternate ACCESS• Currently available

– K (Future availability)

– Grades 1-2

– Grades 3-5

– Grades 6-8

– Grades 9-12

• Designed for ELs who are also SWDs

• Based on alternate performance indicators and alternate proficiency levels

• Listening, Reading, Writing, and Speaking

• Michigan plans to offer in 201337

Page 38: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Technology Readiness Progress

• Bruce Umpstead, OEII & Dave Judd, BAA

• Spring 2015, online assessment to the extent possible

• Smarter Balanced and Michigan accountability assessments will have paper/pencil version at least through spring 2017

• Michigan interim assessments are optional so no paper/pencil version

38

Page 39: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Accessibility and Online Testing

• Technology-based assessment platforms offer new opportunities and ways for accommodations to be provided to students who need them

• Requires a new look at defining

– Accommodations

– Embedded supports/tools

– Good testing practices

39

Page 40: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Good Testing PracticesTeachers must take care to ensure students have opportunities to become familiar with technological aspects of online testing

• Allow time for practice tests using planned assessment platform

• Provide practice time using accommodations and embedded tools

• Begin to use/increase the use of technology in instruction

• Evaluate use of tools/accommodations

40

Page 41: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify
Page 42: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify
Page 43: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify
Page 44: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify
Page 45: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify
Page 46: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

46

What we’ve learned about

TECHNOLOGY READINESS

Page 47: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Device Readiness

• Device Specificationso Most devices meet minimum requirements

o Form factors supported:

- Desktop

- Laptop

- Netbook

- Tablet

• Number of Deviceso Logistical challenges with limited testing devices

o Pilot opportunities allow dry-run of logistics

Page 48: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Network Readiness

• Speed/Capacity• Actual speed varies continually

• Current utilization levels need to be considered

• Reliability• Wireless access points potentially over-utilized

• Consider age and performance of networking equipment

Page 49: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Don’t Feel Ready for Online Testing?

• Paper and Pencil versions of Summative tests will be available through Spring 2017

• So, why make the shift to online testing now?

Page 50: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Advantages of Online Testing

• Retake opportunity

• More informative reports

• Quicker reporting

• No materials to inventory, monitor, and return

• Optional interim assessments online only

• Adaptive testing capability

• More efficient scoring

• Increased student engagement

• Improved security

Page 51: Online Statewide Assessment: Coming Soon to a School Near You!merainc.org/wp-content/uploads/2013/11/Status-of-Online... · 2016-06-17 · Ex. Using tiles in a template, identify

Michigan Online Testing Opportunities for Fall 2013

51

MI-Access SS Cog Lab

5th and 8th Grade MEAPScience

6th and 9th Grade MEAPSocial Studies

K-2 Interim AssessmentsELA and Math

3 - HS Interim AssessmentsScience & Social Studies

Kindergarten Entrance Assessment

September October November